Map systems and autonomous vehicles

Abstract

Obtain constantly changing map information. [Solution] The map system 100 generates a continuously changing dynamic map using information from the moving object 12. The integrated server generates a dynamic map by overlaying the map information, dynamic information, and static information based on the position on the map, using the map information as a reference, based on the real-time surrounding conditions including surrounding images obtained from the moving object 12, dynamic information about moving objects and static information about stationary objects, quasi-static information including signal information and traffic information obtained from road infrastructure, and pre-stored map information.

Description

Technical Field

[0001] The present disclosure relates to a map system that generates a dynamically changing dynamic map using information from a moving object, and an autonomous vehicle using the generated dynamic map.

Background Art

[0002] It is known to perform autonomous driving of a vehicle using a captured image of an in-vehicle camera, and there is also a proposal for creating a map using an imaging image of an in-vehicle camera.

[0003] In Patent Document 1, the three-dimensional position of the same feature point included in a plurality of captured images is calculated using the position and orientation of an in-vehicle camera. Next, the upper speed limit of the vehicle is controlled so that the same feature point is included in a plurality of captured images, using the position and orientation of the in-vehicle camera and the azimuth angle to the position of the calculated feature point. Then, a map including information on each pre-dimensional position is created using the three-dimensional positions of a plurality of different feature points respectively calculated from a plurality of captured images.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, Patent Document 1 relates to map making using an automobile, and creates a map based on a captured image of a camera mounted on the automobile, but there is no idea of reflecting moving objects such as pedestrians and bicycles in the map. Also, due to the influence of a preceding vehicle or the like, data loss occurs at locations that are blind spots of the in-vehicle camera. Furthermore, infrastructure information such as the state of a traffic signal cannot be acquired. Therefore, a real-time dynamic map that reflects the ever-changing situation has not been obtained. [Means for solving the problem]

[0006] The map system described herein is a map system that generates a continuously changing dynamic map using information from a moving object, and generates a dynamic map by overlaying the map information, dynamic information and static information based on the position on the map, based on real-time surrounding conditions including surrounding images obtained from the moving object, dynamic information about moving objects and static information about stationary objects, quasi-static information including signal information and traffic information obtained from road infrastructure, and pre-stored map information.

[0007] Map information is a three-dimensional map, and it is advisable to modify the map information using static information obtained from moving objects.

[0008] Furthermore, it is advisable to overlay quasi-dynamic information, including accident information, road closure information, traffic congestion information, and local weather information, with quasi-static information, including traffic regulation information, onto map data.

[0009] It is preferable to provide a dynamics map of the area in front of a moving object.

[0010] Furthermore, the autonomous vehicle according to the present invention is an autonomous vehicle that uses a dynamic map generated by the map system described above, determines a route from the current location to the destination, uses the dynamic map to acquire and update information on pedestrians, other vehicles, and traffic information along the route, and confirms the route within a few minutes of arriving at the destination. [Effects of the Invention]

[0011] According to this embodiment, a real-time dynamics map that reflects the constantly changing situation can be obtained, and appropriate automated driving can be performed using the obtained dynamics map. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the configuration of the map system 100 related to this disclosure. [Figure 2] This is a block diagram showing the configuration of an autonomous vehicle, which is one type of mobile device. [Figure 3] This is a block diagram showing the configuration of the integrated server. [Figure 4] This is a flowchart showing the processing of static information in an integrated server. [Figure 5] This is a flowchart showing the steps for creating a dynamics map. [Figure 6] This diagram shows an example of vehicle-to-vehicle communication. [Modes for carrying out the invention]

[0013] The embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are not limiting to this disclosure, and configurations formed by selectively combining multiple examples are also included in this disclosure.

[0014] "Configuration of the map system" Figure 1 is a block diagram showing the configuration of the map system 100 related to this disclosure. The integrated server 10 creates a real-time dynamics map based on various input data. The integrated server 10 is supplied with map information as basic information. This map information is supplied from various map information sources that have been used conventionally, and is created based on aerial photographs and various surveys. This map information is updated at a predetermined frequency, but not in real time. In this example, the map information is 3D detailed map information. The integrated server 10 can be configured as a data server on the cloud and can communicate with the mobile unit 12 and the information center 14, which will be described later, at any time using various communication methods.

[0015] The integrated server 10 acquires the surrounding information of a number of moving objects 12. The moving object 12 is equipped with a surrounding information acquisition device such as, for example, a camera, a LiDAR (Light Detection and Ranging), an ultrasonic sensor, etc., and acquires the surrounding conditions. Also, examples of the moving object 12 include automobiles, bicycles, and people, and automobiles are the moving objects that serve as information sources. This is because many automobiles are equipped with navigation devices and can thus grasp their own vehicle positions and communicate with the outside.

[0016] Then, the moving object 12 acquires static information 20, dynamic information 22, and semi-dynamic information 24, and supplies these information to the integrated server 10. Examples of the static information 20 acquired by the moving object 12 include (1) roads, sign images, and (2) building 3D data, etc. Also, examples of the dynamic information 22 acquired by the moving object 12 include (1) labels (people such as automobiles (other vehicles), bicycles, pedestrians, etc.), (2) positions (coordinates), (3) moving directions and speeds, etc. Furthermore, examples of the semi-dynamic information 24 include (1) traffic jam information, (2) road closure information, (3) local weather data (rain, clear, etc.), etc.

[0017] Also, on the road side, there is an information center 14 that includes various information collection facilities and facilities for acquiring road information, etc. It collects various information and supplies it to the integrated server 10. Examples of the dynamic information 22 include signal information (information about the lighting status of traffic signals at intersections), etc., examples of the semi-dynamic information 24 include accidents, traffic jams, weather forecasts, etc., and examples of the quasi-static information 26 include traffic information such as traffic regulations.

[0018] Here, the dynamic information 22 refers to information that needs to be updated about every 0.1 seconds, the semi-dynamic information 24 refers to information that needs to be updated in a few hours to one day, the quasi-static information 26 refers to information that needs to be updated in a few days to one month, and the static information 20 refers to information that is updated at a higher frequency, but it is not necessary to strictly distinguish them.

[0019] Then, the integrated server 10 combines the various supplied information with the map information to create a dynamic map that changes in real time.

[0020] "Configuration of Autonomous Vehicle" Figure 2 is a block diagram showing the configuration of an autonomous vehicle 30 which is one of the moving bodies 12. The surrounding information acquisition device 32 is a camera or the like as described above, and at least obtains an image of the surroundings of the host vehicle. Note that camera images such as traffic jams, road closures, and rain conditions can also be acquired.

[0021] The signal processing unit 34 performs signal processing such as specifying the coordinates of each object existing in the obtained surrounding image, such as latitude, longitude, and altitude, from the surrounding video, position identification function (GPS (Global Positioning System) or position identification from video), and time data.

[0022] Then, the communication unit 36 supplies the information subjected to signal processing to the integrated server 10. This data communication may use a communication line such as the Internet using a wireless line, or may be direct data communication between the vehicle and an infrastructure spot provided on the road side (for example, communication by ETC 2.0).

[0023] In addition, a real-time dynamic map that changes moment by moment is supplied to the communication unit 36 from the integrated server 10. In particular, the integrated server 10 identifies the autonomous vehicle 30 by its vehicle ID and the like, and recognizes the position, speed, etc. of the autonomous vehicle 30, so it supplies a dynamic map of the necessary range in front of the autonomous vehicle 30.

[0024] An autonomous driving processing unit 38 is connected to the communication unit 36, and the autonomous driving processing unit 38 uses the dynamic map to control the running of the host vehicle. For example, it calculates the optimal route from the current location to the destination using the dynamic map. In addition, the dynamic map also includes information such as pedestrians, other moving vehicles, and bicycles, and uses this information to perform driving control such as appropriate steering control and running speed control.

[0025] For example, the mobile unit 12 receives dynamic maps created by the integrated server 10, which overlay quasi-static and dynamic information onto a 3D map, within a 1km radius in the direction of travel of the mobile unit 12. Therefore, the mobile unit 12 can use the continuously supplied dynamic maps to perform appropriate driving control according to the surrounding conditions.

[0026] "Integrated Server Configuration" Figure 3 is a block diagram showing the configuration of the integrated server 10. The information processing unit 40 receives various types of information, such as static and dynamic information, supplied from the mobile unit 12 and the information center 14, and formats them into a format usable by the integrated server 10. The information from the information processing unit 40 is supplied to the dynamics map creation unit 42, where a dynamics map is created. This dynamics map creation unit 42 creates a global dynamics map for the entire range from which the information is supplied.

[0027] Furthermore, the dynamics map creation unit 42 is connected to an individual dynamics map creation unit 44, which creates a dynamics map for an individual autonomous vehicle 30. That is, it creates a dynamics map that changes moment by moment in front of the autonomous vehicle 30 (for example, 1 km ahead) as the vehicle is about to travel.

[0028] Then, the transmission unit 46 transmits the individual dynamics map to the corresponding autonomous vehicle 30.

[0029] "Processing of static information" Figure 4 is a flowchart showing the processing of static information in the integrated server 10. First, the information processing unit 40 acquires information such as time, position, direction of movement, and speed from the moving object 12 (S11). The information processing unit 40 supplies the acquired information to 42 (S12).

[0030] In the dynamics map creation unit 42, first, still images are extracted from video footage such as a camera, and information such as date and time, direction of movement, and speed is added to the images (S13). Next, images of the same location are selected based on the location information (coordinates) (S14).

[0031] Some images should not be used. Therefore, images are selected and checked for reflections of other vehicles, whether the image is invalid due to rain, etc., and whether there is blur due to speed, etc. (S15). Then, using the usable images, 3DCG (three-dimensional computer graphics) is created using photogrammetry (photogrammetry) (S16). Here, the 3DCG created by photogrammetry does not have the captured images pasted onto it.

[0032] Next, based on the date and time information, an image to be pasted onto the CG is selected (S17). For example, an image of what is visible in front of the location is pasted, but in this case, an image from the morning is pasted if it is morning, and an image from the evening is pasted if it is evening.

[0033] In this way, the created 3DCG is used as real-time static information for the dynamics map (S18). That is, it is used as 3D map data for the dynamics map. Note that the 3D map data can also be created from 3D data acquired by LiDAR or other means, rather than photogrammetry.

[0034] In this type of system, the more mobile devices 12 providing information there are, the more constantly the acquired information is updated. By creating 3DCG as static information from multiple images released every second, data such as construction progress and accident information can be updated in addition to buildings, allowing for the acquisition of a real-time dynamics map.

[0035] "Processing of static information" Figure 5 is a flowchart showing the procedure for creating a dynamics map.

[0036] First, static information is combined with pre-prepared 2D map information (S21). 2D map information can be obtained from various sources; it is advisable to select and store the appropriate information. The static information to be combined is generated by processes such as those shown in Figure 4.

[0037] Next, quasi-static information such as construction forecasts and weather forecasts is added (S22). This quasi-static information is obtained from information centers, etc.

[0038] Furthermore, quasi-dynamic information such as construction status, accident status, and local weather is added (S23). This quasi-dynamic information is obtained from the information center 14 and the mobile device 12.

[0039] Then, dynamic information such as vehicles, bicycles, and people is added (S24). In this way, a dynamic map that changes in real time can be created.

[0040] Furthermore, this additional information is overlaid based on location, such as latitude and longitude. The text display can also be made identifiable using callouts or other visual aids.

[0041] "Vehicle-to-vehicle communication" Figure 6 shows an example of car-to-car communication. In this way, mobile units 12 can communicate directly with each other and exchange information. This allows information to be obtained from a nearby vehicle when communication with the integrated server 10 is lost.

[0042] For example, it is advisable to anticipate communication interruptions and store a certain amount of information as a cache on the vehicle side. In this case, the information of the mobile unit 12 can be stored as predicted values. In this way, if there is a mobile unit 12 nearby that can acquire and provide information, they can share information with each other via vehicle-to-vehicle communication.

[0043] "Other configurations" Furthermore, information on pedestrians, cyclists, and other vehicles can be provided only with location, speed, and attributes, ensuring that individuals cannot be identified.

[0044] Static CG for dynamic smaps can be acquired in advance using drones or similar equipment. However, it's best to update them with information obtained from vehicles whenever possible.

[0045] For creating dynamics maps, information captured by small mobility devices or pedestrians can also be used. In this case, location information, camera footage, speed, and shooting direction information are required.

[0046] Dynamic SMAP primarily uses footage shot during the day, but nighttime footage can be processed by software to output nighttime information.

[0047] The Dynamic SMAP video will display images, but even just the color information will suffice.

[0048] The weather in Dynamic Smap can also be created using the software's rendering function, allowing you to create rain, cloudy skies, snowy roads, and other weather conditions.

[0049] Real-time dynamic maps can be viewed and used not only by vehicles, but also by mobile devices that cannot provide images from cameras, smartphones, mobile phones, and computers.

[0050] "Effects of the Embodiment" According to the map system of this embodiment, by obtaining dynamic information that needs to be updated in about 0.1 seconds (cars, pedestrians, bicycles, traffic lights, etc.), static information that is 3D map information (lanes, signs, buildings), semi-dynamic information that needs to be updated in a few hours to a day (accidents, road closures, congestion, local weather information), and semi-static information that needs to be updated in a few days to a month (traffic information such as traffic regulations), a dynamic map can be obtained that covers all the information necessary for autonomous driving.

[0051] Then, by using a dynamics map to calculate the optimal route from the current location to the destination, the route can be determined while taking into account traffic restrictions, congestion, and accident information.

[0052] Furthermore, by using dynamic maps to acquire information on pedestrians, bicycles, other vehicles, and traffic regulations within a few kilometers of the vehicle's direction of travel along the route before starting to drive, and by using dynamic maps every few seconds to acquire information on pedestrians, bicycles, other vehicles, and traffic regulations within a few kilometers of the vehicle's direction of travel along the route, and by reconfirming the route every few minutes until arriving at the destination, appropriate autonomous driving becomes possible.

[0053] Thus, the map system of this embodiment can supply vehicles with a dynamic map (3D map + traffic flow + traffic signals + construction information + pedestrian and bicycle information, etc.) that changes in real time. Furthermore, information obtained from the vehicle itself and other vehicles can be reflected in the dynamic map in real time. By performing autonomous driving using this information, it is possible to drive safely using information about people and other vehicles that the vehicle itself cannot detect. In addition, by using the real-time dynamic map in simulations, it becomes possible to verify whether the autonomous driving algorithm is working correctly.

[0054] Thus, the dynamics map of this embodiment can greatly contribute to the realization of Level 5 (LV5) autonomous driving (on public roads). [Explanation of Symbols]

[0055] 10 Integrated server, 12 Mobile unit, 14 Information center, 20 Static information, 22 Dynamic information, 24 Semi-dynamic information, 26 Semi-static information, 30 Autonomous vehicle, 32 Surrounding information acquisition device, 34 Signal processing unit, 36 Communication unit, 38 Autonomous driving processing unit, 40 Information processing unit, 42 Dynamics map creation unit, 44 Individual dynamics map creation unit, 46 Transmission unit, 100 Map system.

Claims

1. A map system that generates a continuously changing dynamic map using information from a moving object, Based on real-time surrounding conditions including surrounding images obtained from a moving object, dynamic information about moving objects and static information about stationary objects, quasi-static information including signal information and traffic information obtained from road infrastructure, and pre-stored map information, the system overlays the map information, dynamic information, and static information based on the position on the map to generate a dynamic map. Map system.

2. A map system according to claim 1, Map information is a three-dimensional map, and the map information is modified using static information obtained from moving objects. Map system.

3. A map system according to claim 1, Furthermore, semi-dynamic information, including accident information, road closure information, traffic congestion information, and local weather information, and semi-static information, including traffic regulation information, are overlaid on map information. Map system.

4. A map system according to claim 1, To provide a dynamics map of the area in front of a moving object, Map system.

5. An autonomous vehicle using a dynamics map generated by a map system according to any one of claims 1 to 4, Determine the route from your current location to your destination. Using dynamic maps, information on pedestrians, other vehicles, and traffic along the route is acquired and updated. Check the route in the few minutes before arriving at your destination. Self-driving vehicles.

Images