Map generation device

Abstract

A map generation device (60) is provided with: an extraction unit (171) that extracts a feature point from detection information detected by a vehicle-mounted detector (1a) that detects the situation around a host vehicle; a selection unit (172) that selects, from among the plurality of feature points extracted by the extraction unit (171), a feature point to be used for calculation by the calculation unit (173); a calculation unit (173) that, on the basis of the plurality of pieces of detection information, calculates the three-dimensional position of the same feature point included in the plurality of pieces of detection information for each of the plurality of different feature points selected by the selection unit (172), using the position and orientation of the in-vehicle detector (1a); and a generation unit (174) that generates a map including information of each three-dimensional position using the three-dimensional positions of the plurality of different feature points calculated by the calculation unit (173), the selection unit (172) selects feature points other than feature points of a predetermined surface feature, and the generation unit (174) adds information of points corresponding to feature points not selected by the selection unit (172) to the generated map.

Description

Technical Field

[0001] The present invention relates to a map generating device for generating a map for estimating a position of a vehicle. Background Art

[0002] As such an apparatus, there is conventionally known an apparatus configured to create a map using feature points extracted from an image captured by a camera mounted on a traveling vehicle (see, for example, Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent document 1: Japanese Patent Application Publication No. 2019-174910. Summary of the invention

[0006] Problem that the invention aims to solve

[0007] In the prior art, when a vehicle repeatedly drives to correct a map, the accuracy of the information in the map may sometimes be destroyed.

[0008] Solutions for solving problems

[0009] A map generating device according to a technical solution of the present invention comprises: an extraction unit, which extracts feature points from detection information detected by a vehicle-mounted detector that detects the surrounding conditions of the vehicle; a selection unit, which selects feature points used for calculation in a later-described calculation unit from a plurality of feature points extracted by the extraction unit; a calculation unit, which calculates the three-dimensional position of the same feature point included in a plurality of detection information respectively for a plurality of different feature points selected by the selection unit using the position and posture of the vehicle-mounted detector based on a plurality of detection information; and a generation unit, which uses the three-dimensional positions of the plurality of different feature points calculated by the calculation unit to generate a map including information on each three-dimensional position, wherein the selection unit selects feature points other than feature points of specified features, and the generation unit adds information on points corresponding to feature points not selected by the selection unit to the generated map.

[0010] Effects of the Invention

[0011] According to the present invention, a map required for safe vehicle control can be appropriately generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a block diagram schematically showing the overall structure of a vehicle control system according to an embodiment of the present invention.

[0013] Figure 2 This is a block diagram showing a main part configuration of a map generating device according to an embodiment.

[0014] Figure 3AThis is a diagram showing an example of a camera image.

[0015] Figure 3B : is a figure which shows an example of the extracted feature points.

[0016] Figure 3C : is a figure which shows an example of the selected characteristic points.

[0017] Figure 4A This is a flowchart for explaining an example of processing of a program executed by the controller.

[0018] Figure 4B This is a flowchart for explaining an example of processing of a program executed by the controller.

[0019] Figure 5A This is a schematic diagram for explaining information included in the environment map at the time when the process of step S70 is completed.

[0020] Figure 5B This is a schematic diagram for explaining information included in the environment map at the time when the process of step S80 is completed. DETAILED DESCRIPTION

[0021] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0022] The map generation device of the embodiment of the present invention can be applied to a vehicle with an automatic driving function, that is, an automatic driving vehicle. It should be noted that the vehicle to which the map generation device of the present embodiment is applied is sometimes distinguished from other vehicles and referred to as the present vehicle. The present vehicle can be any of an engine vehicle having an internal combustion engine (engine) as a driving source, an electric vehicle having a driving motor as a driving source, and a hybrid vehicle having an engine and a driving motor as driving sources. The present vehicle can be driven not only in an automatic driving mode without the need for the driver to perform driving operations, but also in a manual driving mode based on the driver's driving operations.

[0023] First, the schematic structure of the vehicle related to autonomous driving will be described. Figure 1 1 is a block diagram schematically showing the overall structure of a vehicle control system 100 of a vehicle having a map generating device according to an embodiment. Figure 1 As shown, the vehicle control system 100 mainly includes a controller 10 and an external sensor group 1, an internal sensor group 2, an input and output device 3, a positioning unit 4, a map database 5, a navigation device 6, a communication unit 7 and an actuator AC for driving, which are respectively connected to the controller 10 so as to be communicable therewith.

[0024] The external sensor group 1 is a general term for a plurality of sensors (external sensors) that detect external conditions as information about the surroundings of the vehicle. For example, the external sensor group 1 includes: a laser radar that measures the distance from the vehicle to surrounding obstacles by measuring scattered light generated by omnidirectional irradiation light relative to the vehicle, a radar that detects other vehicles or obstacles around the vehicle by irradiating electromagnetic waves and detecting reflected waves, and a camera mounted on the vehicle and having an imaging element (image sensor) such as a CCD (charge coupled device) or a CMOS (complementary metal oxide semiconductor) to capture images of the surroundings of the vehicle (front, rear, and sides).

[0025] The internal sensor group 2 is a general term for a plurality of sensors (internal sensors) for detecting the driving state of the vehicle. For example, the internal sensor group 2 includes a vehicle speed sensor for detecting the vehicle speed, an acceleration sensor for detecting the acceleration in the front-rear direction and the acceleration in the left-right direction (lateral acceleration) of the vehicle, a rotation speed sensor for detecting the rotation speed of the driving source, and a yaw rate sensor for detecting the angular velocity of the center of gravity of the vehicle around the vertical axis. Sensors for detecting the driver's driving operation in the manual driving mode, such as the operation of the accelerator pedal, the operation of the brake pedal, and the operation of the steering wheel, are also included in the internal sensor group 2.

[0026] The input / output device 3 is a general term for devices for the driver to input commands or output information to the driver. For example, the input / output device 3 includes various switches for the driver to input various commands by operating operating components, a microphone for the driver to input commands by voice, a display that provides information to the driver by displaying images, a speaker that provides information to the driver by sound, etc.

[0027] The positioning unit (GNSS unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. Positioning satellites are artificial satellites such as GPS (Global Positioning System) satellites and Quasi-Zenith Satellites. The positioning unit 4 measures the current position (latitude, longitude, altitude) of the vehicle using the positioning information received by the positioning sensor.

[0028] The map database 5 is a device for storing general map information used in the navigation device 6, and is composed of, for example, a hard disk or a semiconductor element. The map information includes the location information of roads, the information of the road shape (curvature, etc.), and the location information of intersections or forks. It should be noted that the map information stored in the map database 5 is different from the high-precision map information stored in the storage unit 12 of the controller 10.

[0029] The navigation device 6 is a device that searches for a target path on the road leading to the destination input by the driver and guides the driver along the target path. The input of the destination and the guidance along the target path are performed via the input / output device 3. The target path is calculated based on the current position of the vehicle measured by the positioning unit 4 and the map information stored in the map database 5. The current position of the vehicle can also be measured using the detection value of the external sensor group 1, and the target path can also be calculated based on the current position and the high-precision map information stored in the storage unit 12.

[0030] The communication unit 7 communicates with various servers (not shown) via a network including a wireless communication network represented by the Internet, a mobile phone network, etc., and obtains map information, driving history information, traffic information, etc. from the server regularly or at any time. Not only can the driving history information be obtained, but the driving history information of the vehicle can also be sent to the server via the communication unit 7. The network includes not only a public wireless communication network, but also a closed communication network set up for each specified management area, such as a wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc. The obtained map information is output to the map database 5 and the storage unit 12, and the map information is updated.

[0031] The actuator AC is a driving actuator for controlling the driving of the vehicle. When the driving source is an engine, the actuator AC includes a throttle actuator for adjusting the opening of the throttle valve of the engine (throttle opening). When the driving source is a driving motor, the driving motor is included in the actuator AC. The brake actuator that operates the braking device of the vehicle and the steering actuator that drives the steering device are also included in the actuator AC.

[0032] The controller 10 includes an electronic control unit (ECU). More specifically, the controller 10 includes a computer having a computing unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM (read-only memory) and a RAM (random access memory), and other peripheral circuits not shown in the figure, such as an I / O interface. It should be noted that although multiple ECUs with different functions, such as an engine control ECU, a travel motor control ECU, and a brake device ECU, can be independently provided, for the sake of convenience, in Figure 1 , the controller 10 is shown as a collection of these ECUs.

[0033] The storage unit 12 stores high-precision detailed map information (referred to as high-precision map information). The high-precision map information includes road location information, road shape (curvature, etc.) information, road slope information, intersection or fork location information, types of road markings such as white lines or their location information, lane number information, lane width and location information of each lane (information on the center position of the lane, lane position boundary line), location information of landmarks (buildings, traffic lights, signs, etc.) as marks on the map, and road surface contour information such as the bumps and depressions of the road surface. In the embodiment, the center line, lane boundary line, lane outer line, etc. are collectively referred to as road markings.

[0034] The high-precision map information stored in the storage unit 12 includes map information obtained from outside the vehicle via the communication unit 7 (referred to as external map information) and a map produced by the vehicle itself using the detection values ​​of the external sensor group 1 or the detection values ​​of the external sensor group 1 and the internal sensor group 2 (referred to as internal map information).

[0035] External map information is information about a map (called a cloud map) obtained through a cloud server, for example, and internal map information is information about a map (called an environmental map) composed of three-dimensional point cloud data generated by mapping using technologies such as SLAM (Simultaneous Localization and Mapping). External map information is shared by this vehicle and other vehicles, while internal map information is unique map information of this vehicle (for example, map information that this vehicle alone possesses). Environmental maps are created by the vehicle itself on roads that the vehicle has not traveled on, newly built roads, etc. It should be noted that internal map information can also be provided to server devices and other vehicles via the communication unit 7.

[0036] The storage unit 12 stores information such as the travel trajectory information of the host vehicle, various control programs, and threshold values ​​used in the programs, in addition to the high-precision map information described above.

[0037] The calculation unit 11 includes a vehicle position recognition unit 13 , an outside world recognition unit 14 , an action plan generation unit 15 , a travel control unit 16 , and a map generation unit 17 as functional components.

[0038] The vehicle position recognition unit 13 recognizes (may also be referred to as estimating) the position of the vehicle on the map (vehicle position) based on the vehicle position information obtained by the positioning unit 4 and the map information in the map database 5 .

[0039] The vehicle position can also be recognized (estimated) using the high-precision map information stored in the storage unit 12 and the surrounding information of the vehicle detected by the external sensor group 1 , thereby recognizing the vehicle position with high precision.

[0040] The movement information (moving direction, moving distance) of the vehicle is calculated based on the detection values ​​of the internal sensor group 2, thereby identifying the position of the vehicle. It should be noted that when the position of the vehicle can be measured using an external sensor installed on or beside the road, the position of the vehicle can also be identified by communicating with the sensor via the communication unit 7.

[0041] The external recognition unit 14 recognizes the external conditions around the vehicle based on the signals from the external sensor group 1 such as laser radar, radar, camera, etc. For example, the position, speed, acceleration of surrounding vehicles (front vehicles or rear vehicles) traveling around the vehicle, the position of surrounding vehicles parked or parked around the vehicle, and the position and status of other objects are recognized. Other objects include signs, traffic lights, road markings or stop lines, buildings, guardrails, telephone poles, signs, pedestrians, bicycles, etc. The status of other objects includes the color of the traffic light (red, green, yellow), the moving speed and direction of pedestrians or bicycles, etc. Part of the stationary objects among other objects constitute landmarks that serve as indicators of positions on the map, and the external recognition unit 14 also recognizes the position and category of the landmarks.

[0042] The action plan generation unit 15 generates a driving trajectory (target trajectory) of the vehicle from the current moment to a predetermined time based on, for example, the target path calculated by the navigation device 6, the high-precision map information stored in the storage unit 12, the vehicle position identified by the vehicle position recognition unit 13, and the external conditions identified by the external recognition unit 14. When there are multiple trajectories that are candidates for the target trajectory on the target path, the action plan generation unit 15 selects the best trajectory that complies with laws and regulations and meets the standards of efficient and safe driving, and uses the selected trajectory as the target trajectory. Then, the action plan generation unit 15 generates an action plan corresponding to the generated target trajectory. The action plan generation unit 15 generates various action plans corresponding to overtaking driving for overtaking a preceding vehicle, lane change driving for changing driving lanes, following driving for following a preceding vehicle, lane keeping driving for maintaining a lane so as not to leave the driving lane, decelerating driving or accelerating driving. When generating a target trajectory, the action plan generation unit 15 first determines the driving mode and generates the target trajectory based on the driving mode.

[0043] The driving control unit 16 controls each actuator AC in the automatic driving mode so that the vehicle travels along the target trajectory generated by the action plan generating unit 15. More specifically, the driving control unit 16 calculates the required driving force for obtaining the target acceleration per unit time calculated by the action plan generating unit 15, taking into account the driving resistance determined by the road slope, etc. in the automatic driving mode. And, for example, the actuator AC is feedback-controlled in such a way that the actual acceleration detected by the internal sensor group 2 becomes the target acceleration. That is, the actuator AC is controlled so that the vehicle travels at the target vehicle speed and target acceleration. It should be noted that in the manual driving mode, the driving control unit 16 controls each actuator AC according to the driving instructions (steering operation, etc.) from the driver obtained by the internal sensor group 2.

[0044] While driving in manual driving mode, the map generation unit 17 uses the detection values ​​detected by the external sensor group 1 to generate an environmental map of the surrounding area of ​​the road that the vehicle has traveled as internal map information. For example, from the multi-frame camera images obtained by the camera, the edges representing the outline of the object are extracted based on the brightness and color information of each pixel, and the feature points are extracted using the edge information. Feature points are, for example, the intersection of edges, corresponding to the corners of buildings, the corners of road signs, etc. The map generation unit 17 estimates the position and posture of the camera in a manner that the same feature point converges to one point between multiple frame camera images based on the algorithm of SLAM technology, and calculates the three-dimensional position of the feature point at the same time. By performing this calculation process on multiple feature points respectively, an environmental map consisting of three-dimensional point cloud data is generated.

[0045] It should be noted that, instead of a camera, data obtained by radar or lidar may be used to extract feature points of objects around the vehicle and generate an environment map.

[0046] In addition, when the map generating unit 17 generates an environmental map, if it is determined through template matching processing, etc. that the camera image contains specified features (such as road markings, traffic lights, signs, etc.) having feature points that are not used in the calculation of the above-mentioned three-dimensional position, the position information of the points corresponding to the feature points of the features based on the camera image are added to the environmental map and recorded in the storage unit 12.

[0047] The vehicle position recognition unit 13 performs the position recognition processing of the vehicle in parallel with the map making processing of the map generating unit 17. That is, the position of the vehicle is estimated based on the change in the position of the feature point over time. The map making processing and the position recognition (estimation) processing are performed simultaneously according to the algorithm of the SLAM technology. The map generating unit 17 can generate an environmental map not only when driving in a manual driving mode, but also when driving in an automatic driving mode. In the case where the environmental map has been generated and stored in the storage unit 12, the map generating unit 17 can also update the environmental map based on the feature points (also referred to as new feature points) newly extracted from the newly acquired camera image.

[0048] However, the feature points used in generating an environmental map using SLAM technology need to be unique feature points that are easily distinguished from other feature points. In view of this, in actual vehicle control, information on features such as road markings needs to be included in the environmental map. In an embodiment, an environmental map containing information required for vehicle control is appropriately generated by configuring a map generating device that performs the following processes (1) to (3).

[0049] (1) Feature points used in generating the environment map are selected from the feature points extracted from the camera image, which are unique and can be easily distinguished from other feature points. This is because if the feature points are not unique, it is difficult to track the same feature points between multiple camera images. Therefore, on the one hand, it is preferred to select unique feature points based on edge information such as window frames of buildings, and on the other hand, avoid selecting feature points based on edge information of prescribed features such as road markings, signs, and signals, which are difficult to track the same feature points between multiple camera images.

[0050] (2) Adding information useful for identifying (estimating) the vehicle's position to the environment map. According to (1) above, since the environment map does not contain information such as road lines required for identifying the vehicle's position, the road lines and other information are then added (also called filled) to the environment map.

[0051] (3) When the environmental map is corrected, the information added in (2) above is re-appended. Generally speaking, in SLAM technology, since the position of the vehicle is identified while the vehicle is moving, errors are accumulated. For example, when the vehicle is traveling on a road that is closed in the shape of a "mouth", the positions of the starting point and the end point become inconsistent due to accumulated errors. Therefore, when the position identified as the vehicle is on the past driving trajectory, a loop closure process is performed, in which the position of the vehicle identified using feature points (called new feature points) extracted from a newly acquired camera image at the same driving location as in the past and the position of the vehicle identified in the past using feature points extracted from a camera image acquired during past driving are made to have the same coordinates. In an embodiment, the loop closure process is referred to as correction of the environmental map, and the information of the three-dimensional position contained in the environmental map is corrected. At this time, the information added in (2) above is deleted and re-appended to the corrected environmental map.

[0052] The map generating device that performs the above-mentioned processes (1) to (3) will be described in more detail.

[0053] Figure 2 1 is a block diagram showing the main structure of the map generating device 60 according to the embodiment. The map generating device 60 is used to control the driving behavior of the host vehicle, and is composed of Figure 1 As part of the vehicle control system 100. Figure 2 As shown, the map generating device 60 has a controller 10, a camera 1a, a radar 1b, and a laser radar 1c.

[0054] Camera 1a Configuration Figure 1 The camera 1a is a part of the external sensor group 1. The camera 1a can be either a single-lens camera or a stereo camera, and captures the surroundings of the vehicle. The camera 1a is, for example, installed at a predetermined position in front of the vehicle, and continuously captures the space in front of the vehicle at a predetermined frame rate, and sequentially outputs frame image data (referred to as camera images) as detection information to the controller 10.

[0055] Figure 3A 1 is a diagram showing an example of a frame of camera image acquired by camera 1a. Camera image IM includes another vehicle V1 traveling in front of the vehicle, another vehicle V2 traveling in the right lane of the vehicle, traffic lights SG around the vehicle, pedestrians PE, traffic signs TS1, TS2, buildings BL1, BL2, BL3 around the vehicle, lane outer lines OL, lane boundary lines SL, etc.

[0056] Figure 2The radar 1b is mounted on the vehicle and detects other vehicles and obstacles around the vehicle by irradiating electromagnetic waves and detecting reflected waves. The radar 1b outputs the detection value (detection data) as detection information to the controller 10. The laser radar 1c is mounted on the vehicle and measures the scattered light generated by the omnidirectional irradiation light of the vehicle to detect the distance from the vehicle to the surrounding obstacles. The laser radar 1c outputs the detection value (detection data) as detection information to the controller 10.

[0057] The controller 10 includes a calculation unit 11 and a storage unit 12. The calculation unit 11 has an information acquisition unit 141, an extraction unit 171, a selection unit 172, a calculation unit 173, a generation unit 174, a determination unit 175, and a vehicle position recognition unit 13 as functional components.

[0058] The information acquisition unit 141 includes, for example, Figure 1 The extraction unit 171, the selection unit 172, the calculation unit 173, the generation unit 174 and the determination unit 175 are included in, for example, Figure 1 In the map generating unit 17.

[0059] Furthermore, the storage unit 12 includes a map storage unit 121 and a trajectory storage unit 122 .

[0060] The information acquisition unit 141 acquires information for controlling the driving action of the vehicle from the storage unit 12 (map storage unit 121). More specifically, the information acquisition unit 141 reads landmark information included in the environment map from the map storage unit 121, and further acquires information on the positions of road lines indicating the road on which the vehicle is traveling and the extending directions of these road lines (hereinafter referred to as road line information) from the landmark information.

[0061] It should be noted that, when the road line information does not include information indicating the extending direction of the road line, the information acquisition unit 141 may also calculate the extending direction of the road line based on the position of the road line. In addition, information indicating the position and extending direction of the road line on the road on which the vehicle is traveling may be acquired from the road map information stored in the map storage unit 121, the white line map (information indicating the position of white, yellow, etc. road lines), and the like.

[0062] The extraction unit 171 extracts the camera image IM (in Figure 3A The edge showing the outline of the object is extracted (as exemplified in FIG. 1 ), and the feature point is extracted using the edge information. As described above, the feature point is, for example, the intersection point of the edge. Figure 3B It is an example based on Figure 3A FIG. 1 is a diagram showing feature points of a camera image IM extracted by the extraction unit 171. Black circles in the diagram represent feature points.

[0063] The selection unit 172 selects feature points for calculating the three-dimensional position from the feature points extracted by the extraction unit 171. In the embodiment, feature points included in features other than predetermined features (e.g., road markings, traffic lights, traffic signs, etc.) are selected as unique feature points that are easily distinguished from other feature points. Figure 3C It is an example based on Figure 3B A diagram of feature points selected by the selection unit 172. Black circles in the diagram represent feature points. The illustrated predetermined features are only examples, and at least one may be excluded.

[0064] The calculation unit 173 estimates the position and posture of the camera 1a so that the same feature point converges to one point between the multiple frames of camera images IM, and calculates the three-dimensional position of the feature point. The calculation unit 173 calculates the three-dimensional positions of the multiple different feature points selected by the selection unit 172.

[0065] The generation unit 174 generates an environment map composed of three-dimensional point cloud data including information on each three-dimensional position, using the three-dimensional positions of the plurality of different feature points calculated by the calculation unit 173 .

[0066] The determination unit 175 determines whether the environment map generated by the generation unit 174 has been successfully completed. The details of the determination will be described later. The determination unit 175 determines whether the map generated by the generation unit 174 has been successfully completed based on the difference between the position of the new feature point of the above-mentioned predetermined feature extracted based on the camera image IM newly acquired by the camera 1a and the position of the point added to the environment map stored in the map storage unit 121.

[0067] The vehicle position recognition unit 13 estimates the vehicle position on the environment map based on the environment map stored in the map storage unit 121 .

[0068] First, the vehicle position recognition unit 13 estimates the position of the vehicle in the vehicle width direction. In detail, the vehicle position recognition unit 13 uses machine learning (DNN (Deep Neural Network) and other techniques) to recognize the road lines contained in the camera image IM newly acquired by the camera 1a. The vehicle position recognition unit 13 recognizes the position and extension direction of the road lines contained in the camera image IM on the environmental map based on the road line information obtained from the landmark information contained in the environmental map stored in the map storage unit 121. In addition, the vehicle position recognition unit 13 estimates the relative positional relationship between the vehicle and the road lines in the vehicle width direction (positional relationship on the environmental map) based on the position and extension direction of the road lines on the environmental map. In this way, the position of the vehicle in the vehicle width direction on the environmental map is estimated.

[0069] Next, the vehicle position recognition unit 13 estimates the position of the vehicle in the traveling direction. Specifically, the vehicle position recognition unit 13 estimates the position of the vehicle in the traveling direction from the camera image IM ( Figure 3A ) identifies a landmark (e.g., building BL1), and identifies feature points on the landmark from the feature points extracted by the extraction unit 171. Furthermore, the vehicle position recognition unit 13 estimates the distance from the vehicle to the landmark in the direction of travel based on the position of the feature points of the landmark captured in the camera image IM. It should be noted that the distance from the vehicle to the landmark can also be calculated based on the detection value of the radar 1b or the laser radar 1c.

[0070] The vehicle position recognition unit 13 searches for feature points corresponding to the landmark in the environment map stored in the map storage unit 121. In other words, it recognizes feature points matching the feature points of the landmark recognized from the newly acquired camera image IM from among the plurality of feature points (point cloud data) constituting the environment map.

[0071] Next, the vehicle position recognition unit 13 estimates the position of the vehicle in the traveling direction on the environment map based on the position of the feature point on the environment map corresponding to the feature point of the landmark and the distance from the vehicle to the landmark in the traveling direction.

[0072] As described above, the vehicle position recognition unit 13 recognizes the position of the vehicle on the environment map based on the estimated position of the vehicle in the vehicle width direction and the traveling direction on the environment map.

[0073] The map storage unit 121 stores information on the environment map generated by the generation unit 174 .

[0074] The trajectory storage unit 122 stores information indicating the travel trajectory of the vehicle. The travel trajectory indicates, for example, the position of the vehicle on the environment map recognized by the vehicle position recognition unit 13 during travel.

[0075] <Description of Flowchart>

[0076] Reference Figure 4A and Figure 4B A flowchart showing the process of Figure 2 An example of a process performed in the controller 10. Figure 4A The process before the environment map is created is shown, and is started in the manual driving mode, for example, and is repeated at a predetermined cycle. Figure 4B Shown with Figure 4A The map production process is performed in parallel. In addition, Figure 4B After the environment map is created, the automatic driving mode is started, for example, and the process is repeated at a predetermined cycle.

[0077] exist Figure 4AIn step S10 , the controller 10 obtains the camera image IM as detection information from the camera 1 a and proceeds to step S20 .

[0078] In step S20 , the controller 10 extracts feature points from the camera image IM through the extraction unit 171 and proceeds to step S30 .

[0079] In step S30, the controller 10 selects a feature point through the selection unit 172 and proceeds to step S40. As described above, by selecting feature points included in features other than road markings, traffic lights, and traffic signs, unique feature points that are easily distinguishable from other feature points can be selected.

[0080] In step S40 , the controller 10 calculates the three-dimensional positions of a plurality of different feature points respectively through the calculation unit 173 , and then proceeds to step S50 .

[0081] In step S50 , the controller 10 generates an environment map composed of three-dimensional point cloud data including information on the three-dimensional positions of a plurality of different feature points through the generation unit 174 , and proceeds to step S60 .

[0082] In step S60, the controller 10 obtains the position information of the feature points extracted in step S20 that are not selected in step S30, in other words, obtains the position information (the distance from the vehicle to the feature) of the above-mentioned specified feature (road marking, signal, traffic sign, etc.), and proceeds to step S70. The position information is obtained by estimating the distance from the vehicle to the feature based on the position of the feature point of the feature captured in the camera image IM. It should be noted that the distance from the vehicle to the feature can also be obtained based on the detection value of the radar 1b or the laser radar 1c.

[0083] In step S70, the controller 10 adds information of points corresponding to the feature points of the above-mentioned features to the point cloud data of the environment map, and proceeds to step S80. Due to such a configuration, information of features such as road markings is filled into the environment map. By adding information of road markings, signal lights, and traffic signs to the environment map, information of the positions of road markings, signal lights, and traffic signs seen from the position of the vehicle estimated based on the information of the environment map can be provided to the vehicle based on the information of the environment map.

[0084] In step S80 , when the controller 10 recognizes that the position where the host vehicle is traveling is located on the past travel trajectory, the controller 10 corrects the three-dimensional position information included in the environment map through the above-mentioned loop closure processing, and proceeds to step S90 .

[0085] In step S90, the controller 10 records the information of the environment map in the map storage unit 121 of the storage unit 12, and ends. Figure 4Aprocessing.

[0086] exist Figure 4B In step S210 , the controller 10 obtains the camera image IM as detection information from the camera 1 a and proceeds to step S220 .

[0087] In step S220, the controller 10 extracts new feature points from the camera image IM through the extraction unit 171, and proceeds to step S230. Figure 4B Even if the feature points extracted in the process are the same as those in Figure 4A In the case of the feature points extracted during the process, the points on the same object are also called new feature points.

[0088] In step S230, the controller 10 selects new feature points through the selection unit 172 and proceeds to step S240. In step S230, new feature points based on edge information of specified features (road markings, signs, and signals, etc.) and new feature points based on edge information of buildings, etc., which are not specified features, are selected.

[0089] In step S240 , the controller 10 recognizes (estimates) the vehicle position based on the environment map by the vehicle position recognition unit 13 , and the process proceeds to step S250 .

[0090] In step S250, the controller 10 calculates the position difference and proceeds to step S260. The position difference is the difference between the position of the new feature point of the specified feature selected in step S230 and the position of the point corresponding to the feature point of the specified feature added to the environment map in step S70. The position information of the new feature point of the specified feature is obtained by, for example, estimating the distance from the vehicle to the road line, etc., based on the position of the road line, etc. captured in the camera image IM. It should be noted that the distance from the vehicle to the road line, etc. can also be obtained based on the detection value of the radar 1b or the laser radar 1c.

[0091] In step S260, the controller 10 determines whether the environment map is complete. If the position difference is less than the predetermined allowable value, the controller 10 determines that step S260 is affirmative (S260: Yes) and proceeds to step S270. In this case, it is considered that the environment map is completed. Figure 4B The environmental map of the area traveled during the processing is completed, and the environmental map can be used in vehicle control under autonomous driving in this area.

[0092] On the other hand, if the position difference exceeds the predetermined allowable value, the controller 10 determines that step S260 is negative (S260: No), and proceeds to step S280. Figure 4B The environmental map of the area traveled during the processing is not complete, and the environmental map cannot be used for vehicle control under autonomous driving in this area.

[0093] In step S280 , the controller 10 deletes the information added to the environment map in step S70 , and re-adds the position information of the new feature point of the predetermined feature selected in step S230 to the environment map, and then proceeds to step S270 .

[0094] According to the above-described embodiment, the following effects can be obtained.

[0095] (1) The map generating device 60 comprises: an extracting unit 171 for extracting feature points from a camera image IM as detection information detected by a camera 1a as a vehicle-mounted detector for detecting the surrounding conditions of the own vehicle; a selecting unit 172 for selecting feature points to be used for calculation in a calculating unit 173 from among a plurality of feature points extracted by the extracting unit 171; a calculating unit 173 for calculating, based on the plurality of frames of camera images IM, the three-dimensional positions of the same feature point included in the plurality of frames of camera images IM respectively for a plurality of different feature points selected by the selecting unit 172 using the position and posture of the camera 1a; and a generating unit 174 for generating an environment map including information on the respective three-dimensional positions using the three-dimensional positions of the plurality of different feature points calculated by the calculating unit 173, wherein the selecting unit 172 selects feature points other than feature points of specified features, and the generating unit 174 adds information on points corresponding to feature points not selected by the selecting unit 172 to the generated environment map.

[0096] Due to such a configuration, by giving priority to selecting unique feature points that are easy to track between multiple frame camera images IM (for example, feature points based on edge information of window frames of buildings, etc.), and avoiding selecting feature points that are difficult to track between multiple frame camera images IM (for example, feature points based on edge information of specified features such as road markings, signs and signals), the number of feature points used to generate the environment map can be suppressed, and information such as road markings that is useful for identifying (estimating) the position of the vehicle can be included in the environment map.

[0097] In this way, an environment map required for safe vehicle control can be appropriately generated.

[0098] (2) The map generating device 60 of (1) above further comprises: a map storage unit 121 for storing the generated environmental map; a vehicle position recognition unit 13, which serves as a position estimation unit and estimates the position of the vehicle by comparing new feature points extracted by the extraction unit 171 from the camera image IM newly detected by the camera 1a with feature points in the environmental map stored in the map storage unit 121; and a trajectory storage unit 122 for storing the past driving trajectory of the vehicle, wherein the generating unit 174 corrects the information of the environmental map when the position of the vehicle is located on the driving trajectory stored in the trajectory storage unit 122 so that the position of the vehicle estimated by the vehicle position recognition unit 13 using the new feature points is consistent with the position of the vehicle estimated by the vehicle position recognition unit 13 when the vehicle was traveling in the past.

[0099] With this configuration, loop closure processing can be appropriately performed to correct information included in the environment map. As a result, an environment map required for safe vehicle control can be appropriately generated.

[0100] (3) The map generating device 60 of (2) above is further provided with a determination unit 175, which determines whether the environmental map is complete based on the difference between the position of new feature points such as road markings, signs and signals as specified features obtained based on the camera image IM newly detected by the camera 1a and the position of points corresponding to feature points (feature points such as road markings, signs and signals) added to the environmental map stored in the map storage unit 121 and not selected by the selection unit 172.

[0101] With this configuration, the controller 10 can appropriately determine whether the environment map is complete. Figure 5A and 5B To explain the reason.

[0102] Figure 5A is a schematic diagram for explaining information included in the environment map at the time when the processing of step S70 is completed. Figure 5B Schematic diagram for explaining information included in the environment map at the time when the processing of step S80 is completed. Figure 5A In FIG. 1 , circular marks denoted by reference numerals FP1 to FP12 represent feature points constituting the environment map, and graphics denoted by reference numerals T1 to T8 represent points added to the environment map in the process of step S70 (points corresponding to road lines in the camera image IM).

[0103] Through the correction process of step S80, among the feature points constituting the environment map, Figure 5A The characteristic points denoted by reference numerals FP5 to FP7 are respectively Figure 5BThe positions of the reference numerals FP5 to FP7 in the figure are moved. The points T3 and T4 corresponding to the road marking lines, which are added with the positions of the nearest feature points FP5 to FP7 as reference positions, move to the same positions as the positions of the feature points FP5 to FP7. Figure 5B The positions indicated by reference numerals T3 and T4 in the figure move.

[0104] When the positions of the points T3 and T4 corresponding to the road markings have moved beyond the predetermined allowable range, the determination unit 175 determines that the environment map generated by the generation unit 174 is incomplete. In this case, it is necessary to re-add the points T3′ and T4′ corresponding to the road markings newly obtained based on the camera image IM using the positions of the moved feature points FP5 to FP7 as reference positions.

[0105] On the other hand, when the positions of the points T3 and T4 corresponding to the road markings have moved to positions within a predetermined allowable range, the determination unit 175 determines that the environment map generated by the generation unit 174 is complete. In this case, it is not necessary to re-add the points T3′ and T4′ corresponding to the road markings newly acquired based on the camera image IM using the positions of the moved feature points FP5 to FP7 as reference positions.

[0106] As described above, the controller 10 can appropriately determine whether the environment map is completed.

[0107] (4) In the map generation device 60 of (3) above, when the determination unit 175 determines that the environmental map is not complete (step S260 is negative), the generation unit 174 replaces the information of the points (T3 and T4) corresponding to the feature points (feature points such as road markings, signs and signals) not selected by the selection unit 172, and adds the information of the points (T3′ and T4′) corresponding to the new feature points such as road markings, signs and signals that are specified features to the corrected map.

[0108] Due to such a configuration, when the points T3 and T4 corresponding to the road lines move beyond the permissible range as the positions of the feature points FP5 to FP7 move due to the correction of the information of the environment map, the points T3′ and T4′ corresponding to the road lines newly acquired based on the camera image IM can be added to the environment map again. Thus, based on the information of the corrected environment map, the information of the positions of the road lines, signals, and traffic signs seen from the position of the vehicle estimated based on the information of the corrected environment map can be provided to the vehicle.

[0109] (5) In the map generating device 60 of (1) to (4) above, the selecting unit 172 does not select at least one feature point of the road marking, the signal, and the traffic sign.

[0110] Due to such a configuration, by giving priority to selecting unique feature points that are easy to track between multiple frame camera images IM (for example, feature points based on edge information of window frames of buildings, etc.), and avoiding selecting feature points based on edge information of road markings, traffic signs, and signals, etc. that are difficult to track between multiple frame camera images IM, the number of feature points used to generate the environment map can be suppressed, and information such as road markings that is useful for identifying (estimating) the position of the vehicle can be included in the environment map.

[0111] The above-described embodiment can be modified in various forms. Modifications will be described below.

[0112] (Variant 1)

[0113] As examples of specific features that the selection unit 172 does not select based on the feature points of the camera image IM, road markings, traffic signs, and traffic lights are exemplified. However, as long as the object is difficult to track between multiple frames of camera images IM, it can also be configured not to select feature points for other features other than the above-mentioned features.

[0114] (Variant 2)

[0115] In the implementation mode, for the sake of simplicity and ease of explanation, Figure 4A The processing shown in the figure is described as processing before the environment map is created. However, even after the environment map is created (after it is determined that the environment map is complete), it is possible to Figure 4B The vehicle position recognition processing is performed in parallel Figure 4A By performing processing even after the environment map is completed, for example, when the road environment changes, the information can be appropriately added to the environment map.

[0116] The above description is merely an example, and the above embodiment and modified examples do not limit the present invention unless the features of the present invention are impaired. The above embodiment and modified examples may be arbitrarily combined with one or more of them, and the modified examples may be combined with each other.

[0117] Description of Reference Numerals

[0118] 1a: Camera;

[0119] 1b: radar;

[0120] 1c: LiDAR;

[0121] 10: Controller;

[0122] 11: Operation unit;

[0123] 12: Storage unit;

[0124] 13: Vehicle position recognition unit;

[0125] 14: External identification unit;

[0126] 17: Map generation department;

[0127] 60: map generation device;

[0128] 121: map storage unit;

[0129] 122: track storage unit;

[0130] 171: Extraction department;

[0131] 172: Selection Department;

[0132] 173: Computing Department;

[0133] 174: Generation Department;

[0134] 175: Judgment Department;

[0135] BL1~BL3: buildings;

[0136] FP1~FP12: feature points;

[0137] IM: camera image;

[0138] OL: outer lane line;

[0139] SG: signal machine;

[0140] SL: Lane boundary line;

[0141] T1~T8, T3′, T4′: points;

[0142] TS1, TS2: traffic signs;

[0143] V1, V2: other vehicles.

Claims

1. A map generating device, characterized in that: have: an extraction unit that extracts feature points from detection information detected by an on-vehicle detector that detects a condition around the vehicle; a selection unit that selects a feature point used for calculation in a calculation unit described later from among the plurality of feature points extracted by the extraction unit; a calculation unit that calculates the three-dimensional position of the same feature point included in the plurality of detection information respectively for the plurality of different feature points selected by the selection unit based on the plurality of detection information using the position and posture of the vehicle-mounted detector; as well as a generating unit that generates a map including information of each of the three-dimensional positions using the three-dimensional positions of the plurality of different feature points calculated by the calculating unit, The selection unit selects the feature points other than the feature points of a predetermined feature, and the generation unit adds information on points corresponding to the feature points not selected by the selection unit to the generated map.

2. The map generating device according to claim 1, characterized in that: Also available: A map storage unit, which stores the generated map; a position estimating unit that estimates the position of the host vehicle by comparing the new feature points extracted by the extraction unit from the detection information newly detected by the on-vehicle detector with the feature points in the map stored in the map storage unit; and a track storage unit storing the past driving track of the host vehicle, When the position of the vehicle being driven is on the driving trajectory stored in the trajectory storage unit, the generating unit corrects the information of the map so that the position of the vehicle estimated by the position estimating unit using the new feature point is consistent with the position of the vehicle estimated by the position estimating unit during past driving.

3. The map generating device according to claim 2, characterized in that: It also includes a determination unit that determines whether the map is complete based on the difference between the position of a new feature point of the specified feature obtained by the detection information newly detected by the vehicle-mounted detector and the position of a point corresponding to the feature point added to the map stored in the map storage unit and not selected by the selection unit.

4. The map generating device according to claim 3, characterized in that: When the determination unit determines that the map is incomplete, the generation unit adds information of points corresponding to new feature points of the predetermined feature to the corrected map, instead of information of points corresponding to the feature points not selected by the selection unit.

5. The map generating device according to claim 1, characterized in that: The selection unit does not select the feature point of at least one of a road marking, a signal, and a traffic sign.

Images