Vehicle position generation device, vehicle, and server device

Abstract

To improve the reliability of a generated vehicle position.SOLUTION: A vehicle position generation device comprises: a memory that records map data including information on at least positions of a plurality of utility poles 68 arranged to stand along roads 1 and 2 on which a vehicle 3 travels; an imaging device that captures an image of a forward side in a traveling direction of the vehicle 3; and a control section that acquires information from the memory and the imaging device to process the information. The control section generates information on the utility poles 68 included in the image captured by the imaging device, identifies, in the map data, positions of the utility poles 68 on which the information has been generated, and generates a position of the vehicle 3 on the basis of the positions of the utility poles identified in the map data, and the captured image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle position generation device, a vehicle, and a server device. [Background technology]

[0002] Research and development is underway to implement driving control that supports the driver's driving operation in vehicles such as automobiles, or to implement driving control through automated driving. Although automated driving basically includes driving that does not depend on the driver's driving operation, it may also include the driving assistance described above. In such vehicle travel control, it is necessary to obtain the current position of the vehicle and generate the direction in which the vehicle will travel from the current position. In addition, in vehicle travel control, by repeatedly obtaining the current position of the vehicle, it is possible to continuously control the travel of the vehicle. The current position of the vehicle can be generated, for example, by a GNSS receiver installed in the vehicle. The GNSS receiver receives radio waves from multiple GNSS satellites, allowing the vehicle's cruise control device, for example, to calculate and obtain the current position of the vehicle based on the information contained in the radio waves. However, in vehicles such as automobiles, there are times when the GNSS receiver cannot receive signals from multiple GNSS satellites well, for example, in driving conditions such as tunnels, canyons between buildings, or roads in forests. In such cases, the vehicle may not be able to obtain its current position or a reliable current position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-038361 [Patent Document 2] International Publication No. 2016 / 093028 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Documents 1 and 2, for example, an image of a signboard is captured from a vehicle, and the position of the vehicle is corrected based on the captured image. However, the signboards that are the subject of image capture in Patent Documents 1 and 2 are not necessarily installed on roads on which vehicles travel. In this case, it is expected that the techniques in Patent Documents 1 and 2 will be difficult to use widely and generally for roads on which vehicles travel. In addition, for example, zebra lines for pedestrian crossings, stop lines, traffic marks, and the like may be drawn on the road surface. It is conceivable to capture images of these and correct the position of the vehicle based on the captured images. However, the road surface may be covered with snow, soil, or the like. In this case, even if the vehicle captures an image of the road surface, it is unable to obtain information for correcting the position of the vehicle. Road signs and traffic lights are also provided on roads. These road signs, traffic lights, and other structures are provided in the air above the road so that they are easily visible to vehicle drivers. In this case, road signs and traffic lights provided in the air are difficult for drivers to see in the morning or evening when the sun is low in the sky. It is also difficult to clearly identify the indications of road signs and traffic lights even in images captured from a vehicle. Furthermore, if snow accumulates on road signs or traffic lights, it may become impossible to recognize the road signs or traffic lights themselves.

[0005] In this way, when generating a vehicle position, it is required to improve the accuracy of the generated vehicle position. [Means for solving the problem]

[0006] A vehicle position generation device according to one aspect of the present invention includes a memory that records map data including at least position information of a plurality of standing structures arranged in a line along a road on which a vehicle is traveling, an imaging device that images a forward direction in which the vehicle is traveling, and a control unit that acquires and processes information from the memory and the imaging device, The memory is capable of recording information about an erected structure generated based on a past captured image, and the control unit uses the information about the erected structure included in the captured image of the imaging device to generate a position of the vehicle based on the information about the erected structure included in the captured image of the imaging device.and generating a position of the vehicle based on the position of the standing structure identified in the map data and the captured image. The position of the vehicle is generated by a process selected from a single image position generation process and a multiple image position generation process that generates the position of the vehicle based on information about standing structures contained in the image captured by the imaging device and information about past standing structures recorded in the memory.

[0007] A vehicle according to one aspect of the present invention includes a driving control device capable of controlling driving using the position generated by the vehicle position generation device described above.

[0008] A server device according to one embodiment of the present invention includes a driving control unit that generates driving control information for controlling the driving of a vehicle, and a transmitting device that transmits the driving control information generated by the driving control unit to the vehicle, and the driving control unit generates the driving control information assuming that the vehicle is located at a position generated by the vehicle position generation device described above. [Effects of the Invention]

[0009] In the present invention, the map data stored in the memory includes at least information on the positions of upstanding structures erected relative to the road on which the vehicle is traveling. The control unit, which acquires and processes information from the memory and the imaging device, generates information on the upstanding structures included in the captured image of the imaging device and identifies the generated positions of the upstanding structures in the map data. The control unit then calculates the position of the vehicle based on the positions of the upstanding structures identified in the map data and the captured image. The standing structures identified to generate the position may be, for example, utility poles or other poles arranged in a row along the road on which the vehicle travels. Unlike signs and other objects, such standing structures are widely and generally used along roads on which vehicles travel and can be imaged regardless of where the vehicle is traveling on the road. Furthermore, standing structures are less likely to be entirely covered by snow or soil, unlike zebra stripes, stop lines, traffic marks, and other objects painted on the road surface. Furthermore, standing structures can be imaged, for example, at least in part near the road surface, even in the morning or evening when the sun is low in the sky. As a result, when the vehicle travels in a location where standing structures may be included in an image captured by the imaging device, the control unit can generate information about the standing structures included in the image with high reliability from the captured image. Moreover, because the standing structures are arranged in a row along the road on which the vehicle travels, the control unit can generate the vehicle's position by correcting the position each time each standing structure is imaged. In this way, the present invention can improve the accuracy of the generated vehicle position. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram of the driving environment of a car. [Figure 2] FIG. 2 is an explanatory diagram of a control system for an automobile according to the first embodiment of the present invention. [Figure 3] FIG. 3 is a basic configuration diagram of the driving control device of FIG. [Figure 4] FIG. 4 is a flowchart of the driving control of the control system of the automobile of FIG. [Figure 5] FIG. 5 is a flowchart of the driving control of the automatic driving of FIG. [Figure 6] FIG. 6 is a flowchart of the control of generating the current position in FIG. [Figure 7] FIG. 7 is a flowchart of the control of generating the current position based on the captured image of FIG. [Figure 8] FIG. 8 is a flowchart of the generation control of the current position based on the single image of FIG. [Figure 9] FIG. 9 is an explanatory diagram of main information that can be used in generating and controlling the current position by the single image position generation process. [Figure 10] FIG. 10 is an explanatory diagram of a method for generating the position and orientation of an automobile according to the first generation control of FIG. 8 in the single image position generation process. [Figure 11] FIG. 11 is an explanatory diagram of a method for generating the position and orientation of the automobile according to the second generation control of FIG. [Figure 12] FIG. 12 is an explanatory diagram of a method for generating the position and orientation of the automobile according to the third generation control of FIG. [Figure 13] FIG. 13 is a flowchart of the generation control of the current position based on the multiple images of FIG. [Figure 14] FIG. 14 is an explanatory diagram of a method for generating the position of an automobile by the multiple image position generation process. [Figure 15] FIG. 15 is an explanatory diagram of a server device for controlling the running of an automobile according to the second embodiment of the present invention. [Figure 16] FIG. 16 is a flowchart of the server running control by the server device of FIG. [Figure 17] FIG. 17 is a flowchart of the selection control of a structure to be used for control when a plurality of structures are extracted from a captured image. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] [First embodiment] FIG. 1 is an explanatory diagram of the driving environment of an automobile 3. As shown in FIG. In Fig. 1, a plurality of automobiles 3 to which the present invention can be applied are traveling on a road. The automobiles 3 are an example of a vehicle. The vehicle may be one that runs on the driving force of an engine, or one that runs on the driving force of a motor that uses battery power.

[0013] The lower part of FIG. 1 shows an urban road 1. At intersections on the urban road 1, traffic lights 61, zebra zones 62, and stop lines 63 are provided. The traffic lights 61 are provided for roads or lanes entering the intersection. The zebra zones 62 are drawn on the road surface near the intersection to indicate where pedestrians should cross the intersection. The stop lines 63 are drawn on the road surface near the zebra zones 62 so that automobiles 3 should stop before the zebra zones 62. A traffic mark 64 is drawn on the road surface near the zebra zones 62 to warn that a crosswalk is ahead. In addition, road signs 65 indicating destinations and the like are provided near the intersections. The traffic lights 61, road signs 65, and the like are supported by poles erected on the shoulders of the roads or the like and are provided in the air above the road surface so that they are easily visible to drivers of automobiles 3. In urban areas, buildings 69 and the like are built near intersections and the like. A plurality of utility poles 68 are lined up along the road to distribute electricity to buildings 69 and the like. The plurality of utility poles 68 are generally erected in a row at intervals of, for example, several tens of meters on the shoulder of the road or the like.

[0014] The upper part of FIG. 1 shows, for example, a suburban road 2 in a forest. In this case, many trees stand on both sides of the suburban road 2. A sign 66, multiple utility poles 68, and boundary poles 67 are erected beside the suburban road 2. The multiple utility poles 68 are generally erected in a line along the road, such as on the shoulder of the road, at intervals of, for example, several tens of meters. The boundary poles 67 are erected at predetermined intervals on both sides of the road to indicate the position of the shoulder of the road.

[0015] The automobile 3 can travel on urban roads 1 in the lower part of Fig. 1 and suburban roads 2 in the upper part of Fig. 1. The automobile 3 may travel solely based on the driver's operation, may travel with assistance to the driver's operation, or may travel autonomously without assistance from the driver. Driving assistance that assists the driver's operation is a type of autonomous driving.

[0016] FIG. 2 is an explanatory diagram of a control system 10 of the automobile 3 according to the first embodiment of the present invention. The control system 10 in FIG. 2 is provided in the automobile 3 in FIG. 1 and functions as a vehicle position generating device.

[0017] The control system 10 of the automobile 3 in FIG. 2 has multiple control devices, including a cruise control device 15 that performs autonomous driving. FIG. 2 shows multiple control devices, such as a drive control device 11, a steering control device 12, a braking control device 13, an operation detection device 14, a cruise control device 15, a detection control device 16, and an external communication device 17. The control system 10 of the automobile 3 may also include other control devices, such as an air conditioning control device, an occupant monitoring device, a short-range communication device, and an alarm device. The multiple control devices are connected by cables to a central gateway device (CGW) 18 that constitutes a vehicle network. Multiple cables are connected to the central gateway device 18. The multiple control devices may be connected to the central gateway device 18 in a star or bus configuration. The vehicle network may conform to standards such as CAN (Controller Area Network) or LIN (Local Interconnect Network). The vehicle network may also conform to other standards, such as a general-purpose wired communication standard such as a LAN, a wireless communication standard, or a combination of these. Each control device is assigned an ID to distinguish it from other control devices. Each control device may input and output various information using packets with the destination ID and source ID attached. The central gateway device 18 monitors and routes packets on the vehicle network. The central gateway device 18 may check the list and control routing.

[0018] The drive control device 11 controls the drive source and drive force transmission mechanism of the automobile 3. The drive force transmission mechanism may be, for example, a reduction gear, a center differential, etc. The drive force transmission mechanism may be one that individually controls the magnitude of the drive force transmitted to each of the multiple wheels of the automobile 3. The steering control device 12 controls a steering device that changes the direction of a plurality of wheels on the front side of the automobile 3. The traveling direction of the automobile 3 changes according to the direction of the wheels. The brake control device 13 controls a braking device that individually brakes the plurality of wheels of the automobile 3. The braking device may be one that individually controls the magnitude of the braking force that acts on the plurality of wheels of the automobile 3.

[0019] The operation detection device 14 is connected to a plurality of operating members provided on the automobile 3 for the occupant to operate the driving of the automobile 3. The plurality of operating members include, for example, a steering wheel 21, an accelerator pedal 22, a brake pedal 23, and a shift lever 24. The operation detection device 14 detects whether or not each operating member is operated, the amount of operation, etc., and outputs the operation information to the vehicle network.

[0020] A plurality of detection components for detecting the driving state and driving environment of the automobile 3 are connected to the detection control device 16. The plurality of detection components include, for example, a GNSS receiver 25, an outside camera 26, a lidar 27, an acceleration sensor 28, and a distance sensor 29. The GNSS receiver 25 receives radio waves from multiple GNSS satellites (not shown) and generates information on the current position and current time of the automobile 3 equipped with the GNSS receiver 25. The GNSS receiver 25 may be one that can receive radio waves from terrestrial waves and zenith satellites and generate highly accurate information on the current position and current time. The exterior camera 26 captures images of the outside of the automobile 3, which is capable of traveling on roads, etc. The exterior camera 26 may be a monocular camera or a stereo camera. A stereo camera captures multiple images. The automobile 3 may be provided with multiple exterior cameras 26. The multiple exterior cameras 26 may capture images of the front, rear, left, and right sides of the automobile 3 separately to capture images of the surroundings of the automobile 3. The lidar 27 uses a laser to scan the exterior of the automobile 3, which is capable of traveling on roads, and generates spatial information about the exterior of the automobile based on reflected laser waves. The spatial information about the exterior of the automobile includes images of other automobiles and the like around the automobile 3. The exterior camera 26 and the lidar 27 are sensors that detect other automobiles around the automobile 3. The acceleration sensor 28 may be one that detects acceleration in the axial directions of the front-rear, left-right, and up-down directions of the automobile 3. In this case, the acceleration sensor 28 can detect acceleration in the yaw, roll, and pitch directions of the automobile 3. The distance sensor 29 may be one that detects the amount of movement of the automobile 3 based on the amount of rotation of the wheels of the automobile 3, for example. The detection control device 16 outputs the detection information of the various detection components provided in the vehicle to the vehicle network. The detection control device 16 may generate information based on the detection information, such as detection information of other vehicles around the vehicle, and output the information to the vehicle network.

[0021] FIG. 1 shows a plurality of images captured by an exterior camera 26 of an automobile 3. 1 shows an image 70 of an urban area captured by an automobile 3 traveling on an urban road 1, capturing an image of the area ahead of the automobile, i.e., the direction of travel, within the angle of view indicated by the dashed line in the figure, using an external camera 26. In the image 70 of the urban area, in addition to an image 71 of the road surface at an intersection, images 73 of a traffic light, an image 72 of a zebra crossing, and images 74 of multiple utility poles located near the intersection are clearly visible in the image. 1 shows a suburban captured image 80, captured by exterior camera 26 of a car 3 traveling on a suburban road 2, within the angle of view indicated by the dashed line in the figure, of the area ahead of the car. In suburban captured image 80, along with an image 81 of a long, straight road surface, images 82 of signs erected on both sides of the road, images 84 of multiple utility poles, and images 83 of multiple boundary poles are clearly visible in the image. The exterior camera 26 repeatedly captures images of the surroundings of the vehicle 3 while the vehicle 3 is traveling, and outputs the images to the detection control device 16. The detection control device 16 may analyze the images captured by the exterior camera 26, identify the images 71-74, 81-84 included in the captured images, for example, based on their captured shapes, and extract various structures 61-68 included in the captured images. In this case, the detection control device 16 may output information about the structures 61-68 extracted from the captured images, along with the images captured by the exterior camera 26, to, for example, the cruise control device 15 via the vehicle network. Such an exterior camera 26 functions as an imaging device that captures an image of the area ahead, which is the direction in which the automobile 3 is traveling.

[0022] The external communication device 17 establishes a wireless communication path with a base station 30 located outside the automobile 3, for example, near a road. The base station 30 may be a carrier-based one or one for advanced traffic information. The external communication device 17 transmits and receives information to and from a server device 31 connected to the base station 30 via the base station 30. The server device 31 may be provided in a distributed manner corresponding to the base station 30. By providing the base station 30 for 5G communication with the function of the server device 31, the external communication device 17 of the automobile 3 can perform high-speed, large-capacity communication with the server device 31 provided in the base station 30.

[0023] The driving control device 15 controls the driving of the automobile 3 . The driving control device 15 may generate control values ​​for controlling the driving of the automobile 3 and output them to the drive control device 11, the steering control device 12, and the braking control device 13. This allows the automobile 3 to travel on the road under the driving control of the driving control device 15.

[0024] FIG. 3 is a basic configuration diagram of the driving control device 15 of FIG. The driving control device 15 in FIG. 3 includes an input / output device 41, a timer 42, a memory 43, an ECU 44, and an internal bus 45 to which these are connected. The various control devices used in the control system 10 in FIG. 2 may also have the same basic configuration as the driving control device 15 in FIG.

[0025] The input / output device 41 is connected to the vehicle network. The input / output device 41 controls the input / output of information through the vehicle network. For example, the input / output device 41 acquires a packet with an ID corresponding to itself attached thereto from the vehicle network, and outputs the packet to the ECU 44 via the internal bus 45. For example, the input / output device 41 adds a source ID and a destination ID corresponding to itself to the information acquired from the ECU 44 via the internal bus 45, and outputs the information to the vehicle network. The timer 42 measures time and the hour. The hour of the timer 42 may be calibrated by the current hour of the GNSS receiver 25 of the control system 10 of the automobile 3. The memory 43 may be configured, for example, with a non-volatile semiconductor memory, a HDD, a RAM, or the like. The memory 43 stores, for example, programs and data executed by the ECU 44. The memory 43 of the cruise control device 15 may store, in addition to the programs for cruise control, cruise control setting values, detection information of the detection control device 16, operation information of the detection control device 16, high-precision map data 46, and the like. The high-precision map data 46 may be data obtained by the external communication device 17 from the server device 31 and stored.

[0026] The high-precision map data 46 contains richer information than conventional map data for route guidance, which provides directions to a set destination for the automobile 3. The high-precision map data 46 includes, for example, link information on roads and lanes on which the automobile 3 can travel, as well as information on the shapes of roads and lanes and intersections. The high-precision map data 46 may also include information on various structures that can be recognized in images captured by the exterior camera 26 of the automobile 3, such as traffic lights 61, zebra crossings 62, stop lines 63, traffic marks 64, road signs 65, buildings 69, utility poles 68, signs 66, and boundary poles 67. The information on the structures may include the type of structure, information for uniquely identifying the structure or at least the type, the location of the structure, and so forth. Such high-precision map data 46 includes information such as the locations of multiple standing structures erected along the roads on which the automobile 3 travels.

[0027] The ECU 44 reads and executes a program recorded in the memory 43. This realizes a control unit. The ECU 44 generates control values ​​for controlling the running of the automobile 3 and outputs them to the drive control device 11 , the steering control device 12 and the braking control device 13 .

[0028] FIG. 4 is a flowchart of the driving control of the control system 10 of the automobile 3 of FIG. For example, the ECU 44 of the driving control device 15 in FIG. 2 may repeatedly execute driving control of the automobile 3 in FIG. 4 as a control unit. As a result, the control system 10 of the automobile 3 functions as a vehicle position generating device provided in the automobile 3. The driving control of the automobile 3 in FIG. 4 may be executed by the ECU 44 of a control device other than the driving control device 15 provided in the control system 10 of the automobile 3. 4 may be executed by the ECUs 44 of a plurality of control devices provided in the control system 10 of the automobile 3 in cooperation with each other.

[0029] In step ST1, the ECU 44 acquires settings for driving control of the automobile 3. The ECU 44 may acquire information from each part of the control system 10 for driving control of the automobile 3. The ECU 44 may also acquire similar information from the memory 43. The settings for driving control include, for example, a setting for the autonomous driving level.

[0030] In step ST2, the ECU 44 determines whether the driving control to be executed is autonomous driving based on the acquired setting information. Here, autonomous driving may include driving assistance. If the setting is autonomous driving, the ECU 44 proceeds to step ST3. Otherwise, the ECU 44 proceeds to step ST4.

[0031] In step ST3, the ECU 44 executes autonomous driving cruise control including driving assistance. In the cruise control for autonomous driving, the ECU 44 acquires, for example, detection information such as a route to a set destination, high-precision map data 46, and a current position, without relying on driver operation information, and generates a course for the automobile 3 based on the acquired information. The detection information may include a front image captured by the exterior camera 26. The ECU 44 may acquire the route, high-precision map data 46, and the like from the memory 43. Then, for example, the ECU 44 generates a course for safely traveling to the destination along the route from the current position of the automobile 3 generated by the GNSS receiver 25, generates control values ​​for traveling along the generated course, and outputs the control values ​​to the drive control device 11, the steering control device 12, and the braking control device 13. In the cruise control for autonomous driving, the ECU 44 may execute controls such as lane keeping control, preceding vehicle following control, lane changing, merging control at merging sections including merging, obstacle avoidance, and emergency stopping. The ECU 44 may select at least one of these various controls to control traveling along the generated course, and then terminate this control.

[0032] In step ST4, the ECU 44 executes driving control based only on the driver's operation. In this case, the ECU 44 generates a control value corresponding to the driver's operation acquired from the operation detection device 14, and outputs the control value to the drive control device 11, the steering control device 12, and the braking control device 13. Thereafter, the ECU 44 ends this control.

[0033] In this way, the driving control device 15 can perform driving control of the automobile 3 based on the driver's operation, driving control of the automobile 3 that assists the driver's operation, and driving control for automatic driving without the driver's operation. In order for the automobile 3 to perform cruise control for automatic driving, it is important to acquire a highly accurate current position of the automobile 3. If the acquired current position deviates from the actual position of the automobile 3, the actual path of the automobile 3 as a result of cruise control will also deviate. Furthermore, when the automobile 3 continuously executes automatic driving control, it is necessary to repeatedly acquire the current position of the automobile 3 at relatively short intervals. For this reason, the automobile 3 is equipped with a GNSS receiver 25 that can generate highly accurate positions. However, the automobile 3 does not necessarily travel on roads where the GNSS receiver 25 can receive radio waves from multiple GNSS satellites with good reception. For example, the automobile 3 may travel through tunnels, valleys between buildings 69, or roads buried in forests. In these driving conditions, the GNSS receiver 25 may not be able to receive radio waves from multiple GNSS satellites. Even if the automobile 3 is able to receive radio waves from multiple GNSS satellites, the accuracy of the position generated based on the radio waves may be reduced. The automobile 3 may not be able to obtain its current position or a reliable current position. In driving control of the automobile 3, it is required to continuously acquire the current position with an accuracy that can be used for driving control of the automated driving, regardless of the driving conditions of the automobile 3. It is required to improve the accuracy of the position of the automobile 3, regardless of the driving conditions of the automobile 3.

[0034] FIG. 5 is a flowchart of the driving control of the automatic driving of FIG. The ECU 44 of the cruise control device 15 in FIG. 2 may execute the cruise control for the automatic driving in FIG. 5 in step ST3 in FIG.

[0035] In step ST11 , the ECU 44 acquires the latest position information of the automobile 3 from the GNSS receiver 25 .

[0036] In step ST12, the ECU 44 determines whether a new position different from the previous position has been acquired from the GNSS receiver 25. The GNSS receiver 25 basically generates and updates a new position when it is able to receive new radio waves. If the GNSS receiver 25 is unable to receive new radio waves, the GNSS receiver 25 is unable to update the position. Alternatively, the GNSS receiver 25 is unable to generate a meaningful position through the update process. For example, if the newly acquired position is meaningful and different from the position acquired in the previous process, the ECU 44 determines that a new position has been acquired, and proceeds to step ST13. If it is not determined that a new position has been acquired, the ECU 44 proceeds to step ST14 so as not to use the newly acquired position from the GNSS receiver 25 for the current position.

[0037] In step ST13, the ECU 44 determines the accuracy of the position newly acquired from the GNSS receiver 25. Here, the ECU 44 determines whether the error of the position newly acquired from the GNSS receiver 25 is equal to or less than an error threshold. When generating a position, the GNSS receiver 25 often also generates an error range of the position. The threshold of the error range may be, for example, a radius of several tens of centimeters. If the error of the position acquired from the GNSS receiver 25 is equal to or less than the threshold, the ECU 44 proceeds to step ST15. If the error of the acquired position is greater than the threshold, the ECU 44 proceeds to step ST14 so as not to use the position newly acquired from the GNSS receiver 25 for the current position.

[0038] In step ST14, the ECU 44 sets position generation for obtaining the current position. The ECU 44 records, for example, a position generation flag of a predetermined value in the memory 43. The value of the position generation flag may be either a value for executing position generation for obtaining the current position, or a value for not executing position generation. Thereafter, the ECU 44 proceeds to step ST17.

[0039] In step ST15, the ECU 44 determines whether position generation for obtaining the current position is set. The ECU 44 acquires, for example, the value of a position generation flag recorded in the memory 43. If the position generation flag is a set value for position generation, the ECU 44 proceeds to step ST17. The ECU 44 proceeds to step ST17 while the value of the position generation flag is set to the set value for position generation. If the value of the position generation flag is a set value for not executing position generation, the ECU 44 proceeds to step ST16.

[0040] In step ST16, the ECU 44 sets the current position to the position newly acquired from the GNSS receiver 25. After that, the ECU 44 proceeds to step ST18.

[0041] In step ST17, the ECU 44 generates a current position. In this case, the ECU 44 does not use a position newly acquired from the GNSS receiver 25 as the current position as it is, but generates a position calculated by various methods described later as the current position.

[0042] In step ST18, the ECU 44 maps the current position on the high-precision map data 46, generates a course for the automobile 3 to follow from the current position, and generates cruise control values ​​for traveling along the generated course.

[0043] In step ST19, the ECU 44 outputs the generated driving control values ​​to the drive control device 11, the steering control device 12, and the braking control device 13. This allows the automobile 3 to travel by automatic driving.

[0044] FIG. 6 is a flowchart of the control of generating the current position in FIG. The ECU 44 of the driving control device 15 in FIG. 2 may execute the control of generating the current position in FIG. 6 in step ST17 in FIG.

[0045] In step ST21, the ECU 44 generates a GNSS-based position of the vehicle 3 based on the most recent significant position obtained from the GNSS receiver 25. Here, if a new position cannot be acquired from the GNSS receiver 25, the ECU 44 may use the most recent significant position acquired from the GNSS receiver 25 at any previous time. The ECU 44 then generates the position of the vehicle 3 based on the movement history from the most recent significant acquired position. The ECU 44 may acquire the movement distance and movement direction after the most recent significant acquired position from the detection control device 16 or the memory 43. The detection control device 16 can generate the speed of the vehicle 3 by integrating the front-rear, left-right, up-down acceleration detected by the acceleration sensor 28 over time. The detection control device 16 can generate the traveled distance by integrating the speed of the vehicle 3 over time. The memory 43 may record information based on the detection of these vehicle sensors acquired from the detection control device 16. The ECU 44 uses this movement history to calculate the movement distance and movement direction of the vehicle 3 from the most recent significant acquired position and generates the position of the vehicle 3 based on the GNSS. Furthermore, if a position that is significant but may not be highly accurate is acquired from the GNSS receiver 25, the ECU 44 sets the acquired position as the position of the automobile 3 based on the GNSS. Also, when a significant and highly accurate position begins to be acquired from the GNSS receiver 25, the ECU 44 sets the acquired position as the position of the automobile 3 based on the GNSS.

[0046] In step ST22, the ECU 44 generates the position of the automobile 3 based on the captured image captured by the external sensor, which is the host vehicle sensor. The ECU 44 may extract a structure, such as a utility pole 68, included in the captured image, and generate the position of the automobile 3 using the position of the extracted structure in the high-precision map data 46 as a reference, so that the angle of view position of the structure can be obtained in the virtual captured image captured by the external sensor in the high-precision map data 46. Here, the distance between the structure and the automobile 3 in the high-precision map data 46 may be the distance from the structure extracted from the captured image to the automobile 3. Details will be described later.

[0047] In step ST23, the ECU 44 determines whether the non-acquisition period during which a high-accuracy position cannot be acquired from the GNSS receiver 25 is equal to or longer than a predetermined time. The ECU 44 may, for example, instruct the timer 42 to start measurement in the processing of step ST14 in FIG. 5, and acquire the time measured by the timer 42 at the processing timing of step ST23 as the non-acquisition period. The predetermined time compared with the non-acquisition period may, for example, be the time when the error range for the position of the vehicle 3 based on the GNSS generated in step ST21 is estimated to exceed a radius of 1 meter. If the non-acquisition period is not equal to or longer than the predetermined time, the ECU 44 proceeds to step ST24. If the non-acquisition period is equal to or longer than the predetermined time, the ECU 44 proceeds to step ST25.

[0048] In step ST24, the ECU 44 sets the GNSS-based position generated in step ST21 as the current position of the automobile 3. After that, the ECU 44 advances the process to step ST26.

[0049] In step ST25, the ECU 44 sets the position based on the captured image generated in step ST22 as the current position of the automobile 3. After that, the ECU 44 advances the process to step ST26.

[0050] In step ST26, the ECU 44 determines whether the current position acquired from the GNSS receiver 25 is consistent with reality. The ECU 44 may map the current position acquired from the GNSS receiver 25 onto the high-precision map data 46 and determine whether a virtual image in the high-precision map data 46 from the mapped position matches the image captured by the exterior camera 26. The ECU 44 may determine whether the angle of view positions in the images of an upright structure such as a utility pole 68 match between the images being compared. If a match is determined, the ECU 44 determines that the current position acquired from the GNSS receiver 25 is consistent with reality, and proceeds to step ST27. Otherwise, the ECU 44 ends this control without proceeding to step ST27.

[0051] In step ST27, the ECU 44 executes a return process to use the position acquired from the GNSS receiver 25 as the current position. The ECU 44 updates the value of the position generation flag set in step ST14 of FIG. 5 to a value indicating that position generation is not executed, in order to continuously execute the current position generation control of FIG. 6. As a result, the ECU 44 determines in step ST15 that position generation for obtaining the current position is not set, and can proceed with the process to step ST16. Thereafter, the ECU 44 ends this control.

[0052] In this way, in the current position generation process of FIG. 6, the ECU 44 can generate, as the current position, the position based on the captured image generated in step ST22 or the position based on the GNSS generated in step ST21.

[0053] Then, in the control of Figures 5 and 6, if a predetermined time has elapsed since the ECU 44 determined that no location information is provided from the GNSS receiver 25 as a location information generating device, the ECU 44 can start generating the current location of the automobile 3 based on the image captured by the exterior camera 26. In addition, if a predetermined time has elapsed since the ECU 44 determined that the error range of the location information provided by the GNSS receiver 25 is greater than a threshold and is therefore not highly reliable, the ECU 44 can start generating the current position of the automobile 3 based on images captured by the exterior camera 26. In this way, the ECU 44 judges the quality of the position information from the GNSS receiver 25, and if a predetermined time has elapsed while the quality of the position information is not high, the ECU 44 starts generating the current position of the automobile 3 based on images captured by the exterior camera 26. As a result, if a dead recognition state in which high-accuracy position information cannot be obtained from the GNSS receiver 25 continues, the ECU 44 can switch the current position of the automobile 3 to one based on images captured by the exterior camera 26. If the dead recognition state continues for a long time, the accuracy of the position obtained by calculating the amount of movement from the last high-accuracy position of the GNSS receiver 25, for example, may decrease.

[0054] Furthermore, the ECU 44 can continue to generate the position based on the image captured by the exterior camera 26 as the current position of the vehicle 3, at least until the position and orientation of the vehicle 3 provided by the GNSS receiver 25 or the position and orientation based thereon return to a high degree of accuracy such that the relative positional relationship with the structure identified in the high-precision map data 46 matches the angle of view position of the identified structure in the image captured by the exterior camera 26. At least during the period when a high-precision position cannot be obtained from the GNSS receiver 25, the vehicle 3 can use a position based on the captured image or the like that is more likely to be accurate to control the driving of the vehicle 3. Furthermore, after the position of the automobile 3 obtained by the GNSS receiver 25 is restored to a high accuracy, the ECU 44 can execute high-accuracy driving control using the high-accuracy position.

[0055] FIG. 7 is a flowchart of the control of generating the current position based on the captured image of FIG. The ECU 44 of the driving control device 15 in FIG. 2 may execute current position generation control based on the captured image in FIG. 7 in step ST22 in FIG. 7, such as information on utility pole 68, which is an upright structure extracted from a past captured image, the past position of automobile 3 corresponding to the past captured image, and the amount of movement since the past position of automobile 3. Memory 43 may also record information on structures other than utility pole 68. The amount of movement may be the distance and direction of movement.

[0056] In step ST31, the ECU 44 determines the freshness of the information recorded in the memory 43. The ECU 44 may determine the freshness of the information based on, for example, the elapsed time between the recording time information attached to the past information in the memory 43 and the current time of the timer 42 . Furthermore, the ECU 44 may determine the freshness of the information based on the distance between the past position recorded in the memory 43 and the position based on the GNSS generated in step ST21 of FIG.

[0057] In step ST32, the ECU 44 determines whether or not the information recorded in the memory 43 can be used to generate the current position. For example, if the elapsed time of the information is equal to or greater than a predetermined threshold and is therefore old, the ECU 44 proceeds to step ST33 so as not to use the information in the memory 43 to generate the current position. If the elapsed time of the information is shorter than the predetermined threshold, the ECU 44 proceeds to step ST34 so as to use the information in the memory 43 to generate the current position. Alternatively, if the distance in the information is older than a predetermined threshold, the ECU 44 proceeds to step ST33 so as not to use the information in the memory 43 for generating the current position. If the elapsed time of the information is shorter than a predetermined threshold, the ECU 44 proceeds to step ST34 so as to use the information in the memory 43 for generating the current position.

[0058] In step ST33, the ECU 44 executes a single-image position generation process that does not use information from the memory 43 as a reference for generating the current position. The ECU 44 generates the position of the automobile 3 using, for example, a utility pole 68 included in the image captured by the exterior camera 26 as a position reference. Thereafter, the ECU 44 proceeds to step ST35.

[0059] In step ST34, the ECU 44 executes a multiple image position generation process using the information in the memory 43 as a reference for generating the current position. The memory 43 stores information about utility poles 68 extracted from previously captured images. The ECU 44 generates the position of the automobile 3 using, for example, one utility pole 68 included in an image captured by the exterior camera 26 and another utility pole 68 included in a previously captured image stored in the memory 43 as a position reference.

[0060] In step ST35, the ECU 44 records the information generated in the current processing of Fig. 7 in the memory 43. As a result, information based on previously captured images is accumulated and recorded in the memory 43.

[0061] In this way, depending on the freshness of the information in memory 43, ECU 44 selects either a multiple image position generation process that uses the information in memory 43 or a single image position generation process that does not use the information in memory 43, and generates the position of automobile 3 using the selected process. In this embodiment, when the accuracy of the position based on GNSS is reduced, a certain degree of accuracy can be ensured for the current position of automobile 3 by setting the position based on a single captured image or multiple captured images as the current position of automobile 3.

[0062] Next, the generation of a position based on a single captured image or multiple captured images will be described in detail. Here, the high precision map data 46 includes at least information on the location of structures installed on or near the road on which the automobile 3 travels. The high precision map data 46 includes at least information on the location of a plurality of utility poles 68, poles, and other erected structures that are lined up along the road on which the automobile 3 travels. On the road on which the automobile 3 is traveling, road surface features such as zebra zones 62, stop lines 63, traffic marks 64, etc. that can be recognized in the image captured by the exterior camera 26 of the automobile 3 are drawn. Traffic lights 61 and road signs 65 are provided at intersections and the like of roads along which automobiles 3 travel. Signs 66, electric poles 68, etc. are provided near the road on which the automobile 3 travels.

[0063] Among these structures, road surface drawings, traffic lights 61, road signs 65, and billboards 66 are difficult to capture clearly in images when there is snowfall. Road surface drawings are also difficult to capture clearly in images when the road surface is frozen. Even if the ECU 44 attempts to extract structures by analyzing the images captured by the exterior camera 26, it may not be able to extract structures whose original shapes and contours are not clearly captured in the captured images. In contrast, at least a portion of standing structures such as the poles supporting the traffic lights 61 and road signs 65, and utility poles 68 can be clearly captured in the image even if there is snowfall or ice. The ECU 44 can extract standing structures such as utility poles 68 by analyzing the image captured by the exterior camera 26. Standing structures such as utility poles 68 can be extracted with a higher probability than other structures. In this embodiment, differences in the ease of extraction due to differences in the types of such structures are taken into consideration. The high-precision map data 46 in this embodiment includes information on the positions of at least a plurality of utility poles 68, poles, and other standing structures that are lined up along the road on which the automobile 3 travels. Furthermore, when the ECU 44 extracts an upright structure such as a utility pole 68 based on a previously captured image, the memory 43 records information about the upright structure such as a utility pole 68 that has been extracted in the past. In the following, an example will be described in which a utility pole 68 captured in a captured image is used to generate the current position. The following description can be similarly applied to erected structures other than utility poles 68 and other structures.

[0064] FIG. 8 is a flowchart of the generation control of the current position based on a single image. The ECU 44 of the driving control device 15 in FIG. 2 may execute the single image position generation control in FIG. 8 in step ST33 in FIG.

[0065] In step ST41, the ECU 44 analyzes the image captured by the outside camera 26 and extracts the utility pole 68 captured in the image. Furthermore, the ECU 44 may determine that the type of the extracted structure is a utility pole 68, or may extract the characteristics of the utility pole 68 itself.

[0066] In step ST42, the ECU 44 generates vector information indicating the relative position between the extracted utility pole 68 and the automobile 3. If exterior camera 26 is a stereo camera or the like, ECU 44 calculates the relative distance from the vehicle to the standing structure based on the captured position of utility pole 68 in the captured images and the parallax between the multiple captured images. Errors in the relative distance can be reduced by, for example, the resolution of exterior camera 26. Errors are smaller when exterior camera 26 is a stereo camera than when it is a monocular camera. Furthermore, the ECU 44 calculates the relative direction of the erected structure with respect to the vehicle exterior camera 26 of the vehicle itself based on the image capturing position of the utility pole 68 in the captured image and the parallax between the multiple captured images. The relative direction of the utility pole 68 is information on the angle of view position captured from the automobile 3. This allows the ECU 44 to generate a relative position vector between the extracted utility pole 68 and the automobile 3. Furthermore, the ECU 44 may provisionally generate a relative position vector from the vehicle as an approximate position of the utility pole 68. The ECU 44 can generate information about the utility pole 68 included in the image captured by the outside camera 26. The ECU 44 may generate information about the utility pole 68 included in the image captured by the exterior camera 26 by using the image captured by the exterior camera 26 as well as spatial information about the exterior of the vehicle obtained by Lidar.

[0067] In step ST43, the ECU 44 identifies the position of the utility pole 68 extracted from the captured image in the high-precision map data 46. The ECU 44 may identify, from among the multiple utility poles 68 included in the high-precision map data 46, for example, one that is located in the direction of the relative position vector from the tentatively estimable vehicle position, as the utility pole 68 extracted in the high-precision map data 46. Alternatively, for example, the ECU 44 may identify, from among the multiple utility poles 68 included in the high precision map data 46, the one that is closest to the approximate position of the utility pole 68 as the extracted utility pole 68 in the high precision map data 46. Alternatively, for example, the ECU 44 may repeatedly execute the utility pole 68 extraction process for captured images repeatedly generated by the external camera 26 while the vehicle is traveling. In this case, the ECU 44 can continue to correctly identify utility poles 68 that can be imaged at that time for captured images during a period in which a highly accurate position based on GNSS is obtained. By executing this process while correctly identifying utility poles 68 in the past, the ECU 44 can correctly identify the utility pole 68 involved in the process even when multiple utility poles 68 that can be imaged are present in the high-precision map data 46. The ECU 44 can correctly identify the current utility pole 68 in the high-precision map data 46 based on the relative positional relationships of the extracted multiple utility poles 68. This allows the ECU 44 to identify the position of the utility pole 68 in the high-precision map data 46 as a single upright structure included in the latest image captured by the exterior camera 26.

[0068] From step ST44, the ECU 44 specifically starts the process of generating the position of the automobile 3 based on the captured image. In this embodiment, the ECU 44 generates the current position of the automobile 3 by single-image position generation process based on the image captured by the exterior camera 26 through three stages of process: first generation process in step ST44, second generation process in step ST45, and third generation process in step ST46. In the first generation process of step ST44, the ECU 44 moves the automobile 3 in the high-precision map data 46 based on the relative positional relationship (relative distance and relative direction) between the utility pole 68 and the automobile 3, and generates the position and orientation of the first automobile 94. In the second generation process of step ST45, the ECU 44 moves the automobile 3 in the high precision map data 46 based on the relative distance between the utility pole 68 and the automobile 3, and generates the position and orientation of the second automobile 95. In a third generation process in step ST46, the ECU 44 generates the position and orientation of the third vehicle 97 based on the first position in step ST44 and the second position in step ST45.

[0069] In step ST47, the ECU 44 sets the position and orientation of the automobile 3 determined in the processing up to step ST46 as the current position of the automobile 3. The ECU 44 may basically set the position and orientation of the third vehicle 97 to the current position of the vehicle 3 .

[0070] This allows the ECU 44 to generate the current position of the automobile 3 by single-image position generation processing based only on the image captured by the outside camera 26. Next, the first to third generation processes of the single image position generation process will be described in detail with reference to FIGS. The road on which the automobile 3 actually travels is in a three-dimensional space, but for ease of explanation, the following description will be given using a two-dimensional space of XY.

[0071] FIG. 9 is an explanatory diagram of main information that can be used in generating and controlling the current position by the single image position generation process. 9 shows a car 3 traveling on a suburban road 2. A utility pole 68 is erected on the left front side of the car 3.

[0072] In this case, the ECU 44 can acquire information on the position and orientation (x0, y0, θ0) of the automobile 3 based on GNSS, the position (x1, y1) of the utility pole identified in the high-precision map data 46, and the relative position vector (Ld, θd) from the automobile 3 to the utility pole 68. The position (x0, y0) of the automobile 3 based on the GNSS may be anything that can be associated with the high-precision map data 46, and may be, for example, latitude and longitude values. The orientation θ0 of the automobile 3 based on the GNSS may be anything that can be associated with the high precision map data 46, and may be, for example, an angle in a 360-degree direction with geomagnetic north as the reference. The relative position vector (Ld, θd) from the automobile 3 to the utility pole 68 may be generated based on the captured image or spatial information from the Lidar 27. Here, Ld is the relative distance, and θd is the relative angle. The relative angle may be, for example, an angle based on the front of the automobile 3.

[0073] FIG. 10 is an explanatory diagram of a method for generating the position and orientation of the automobile 3 by the first generation control of the single image position generation control in FIG. For comparison, FIG. 10 shows a car 3 traveling on a suburban road 2 and a utility pole 68 in dashed lines.

[0074] In FIG. 10, the identified utility pole 68 is located in front of the left side of the automobile 3 in the high precision map data 46. In the first generation control, the ECU 44 generates the position and orientation (x10, y10, θ10) of the first vehicle 94 as a position separated by a relative position vector (Ld, θd) based on the position (x1, y1) of the utility pole 92 identified in the high-precision map data 46. In this case, the first automobile 94 moves in parallel from the position indicated by the dashed line measured by the GNSS receiver 25, which is the host vehicle sensor. The position of the first vehicle 94 under the first generation control is a position that is separated from the identified utility pole 92 by a relative distance Ld in the direction of the relative accuracy θd. The orientation of the first vehicle 94 under the first generation control is a direction that forms a relative angle θd with respect to the direction from the vehicle 3 at that position to the utility pole 92. In this case, the ECU 44 can generate the current position of the automobile 3 by maintaining the orientation of the utility pole 92, which is the identified standing structure, at the angle of view position in the image captured by the exterior camera 26 and estimating the distance from the captured image. The current position of the automobile 3 is likely to be close to the actual position relative to the utility pole 92. However, the likelihood of the orientation of the automobile 3 cannot be increased.

[0075] FIG. 11 is an explanatory diagram of a method for generating the position and orientation of the automobile 3 according to the second generation control of FIG. For comparison, FIG. 11 shows a car 3 traveling on a suburban road 2 and a utility pole 68 in dashed lines.

[0076] In FIG. 11, the identified utility pole 92 is located in front of the left side of the automobile 3 in the high precision map data 46 . In the second generation control, the ECU 44 first executes a position generation process to move the position of the automobile 3, as shown in the upper part of Fig. 11. In the position generation process, the ECU 44 uses the position (x1, y1) of the utility pole 92 identified in the high-precision map data 46 and the position (x0, y0) of the automobile 3 based on the GNSS as references, and generates a position between them as the position of the second automobile 95. The position of the second automobile 95 may be a position separated by a relative distance Ld from the identified position of the utility pole 92. Next, the ECU 44 performs an orientation generation process to rotate the orientation of the automobile 3, and generates an orientation of the second automobile 95 such that the direction from the position of the second automobile 95 toward the identified utility pole 92 is at the relative angle θd generated based on the captured image, as shown in the lower part of Figure 11. As a result, the ECU 44 generates the position and orientation of the second vehicle 95 as (x20, y20, θ20). In this case, as a second generation process, the ECU 44 generates, as the current position of the automobile 3, a position that is between the position of the upright structure identified in the high-precision map data 46 and a tentative position based on the GNSS, which may have an error before correction, and that is at a relative distance Ld estimated from the captured image. In this case, the ECU 44 can further generate an orientation of the automobile 3 so that, at the generated current position, a viewing angle position θd of the utility pole 68, which is the identified upright structure, in the image captured by the outside camera 26 is obtained.

[0077] FIG. 12 is an explanatory diagram of a method for generating the position and orientation of the automobile 3 according to the third generation control of FIG. For comparison, the position and orientation (x10, y10, θ10) of the first vehicle 94 and the position and orientation (x20, y20, θ20) of the second vehicle 95 are shown by dashed lines in FIG.

[0078] The ECU 44 generates a position between the position (x10, y10) of the first vehicle 94 and the position (x20, y20) of the second vehicle 95 as the position of the third vehicle 97. Furthermore, the ECU 44 generates a direction between the direction θ10 of the first vehicle 94 and the direction θ20 of the second vehicle 95 as the direction of the third vehicle 97. Here, the weighting ratio between the value of the first vehicle 94 and the value of the second vehicle 95 may be, for example, 0.5:0.5. In this case, the value of the third vehicle 97 is an intermediate value between the value of the first vehicle 94 and the value of the second vehicle 95. As a result, the ECU 44 generates the position and orientation (x30, y30, θ30) of the third automobile 97. The ECU 44 can generate the position and orientation (x30, y30, θ30) of the third automobile 97 as a position and orientation between the position and orientation obtained by the first generation process and the position and orientation obtained by the second generation process.

[0079] In this way, the ECU 44 executes the series of processes shown in FIGS. 10 to 12 to generate the position and orientation (x30, y30, θ30) of the third automobile 97 as the current position and orientation of the automobile 3 based on a single image. In the single image position generation process, the ECU 44 identifies the position of a utility pole 68, which is a single upright structure included in the latest image captured by the exterior camera 26, in the high precision map data 46. The ECU 44 can then generate the current position and orientation of the automobile 3 in the high-precision map data 46 based solely on the utility pole 68 in the image captured by the exterior camera 26, without using past utility pole information recorded in the memory 43. The ECU 44 can generate the current position and orientation of the automobile 3 in the high-precision map data 46 based on the relative position of the utility pole 68, which is a single upright structure, in the image captured by the exterior camera 26. This allows the ECU 44 to generate a probable current position and orientation of the automobile 3 based on the image captured by the outside camera 26 without being affected by past information about utility poles recorded in the memory 43.

[0080] The current position and orientation of the automobile 3 generated by this single-image position generation process can be more likely to be suitable for driving control of the automobile 3. The current position and orientation of the automobile 3 generated by the single-image position generation process can be more likely to be suitable for driving control of the automobile 3 than the position and orientation based on GNSS. By controlling the driving of the automobile 3 using the current position and orientation of the automobile 3 generated by the single-image position generation process, it is expected that the automobile 3 will drive safely, for example, by maintaining its position near the center of a snowy road or lane. It is expected that the position and orientation of the automobile 3 obtained by the third generation process will be able to maintain its position near the center of a snowy road or lane more effectively than those obtained by the first generation process or the second generation process.

[0081] FIG. 13 is a flowchart of the generation control of the current position based on the multiple images of FIG. The ECU 44 of the driving control device 15 in FIG. 2 may execute the multiple image position generation control in FIG. 13 in step ST34 in FIG.

[0082] In step ST51, the ECU 44 analyzes the image captured by the outside camera 26 and extracts the utility pole 68 captured in the image. The extraction process here may be the same as that in step ST41.

[0083] In step ST52, the ECU 44 generates vector information indicating the relative position between the extracted utility pole 68 and the automobile 3. The extraction process here may be the same as that in step ST42.

[0084] In step ST53, the ECU 44 reads information about the utility pole 68 that has been previously identified from the memory 43. The memory 43 stores information about the utility pole 68 that has been previously identified, such as its position, through the processing of step ST35 in FIG.

[0085] In step ST54, the ECU 44 calculates the past relative distance of the automobile 3 from the utility pole 68. Here, the position of the automobile 3 may be based on the GNSS.

[0086] In step ST55, the ECU 44 identifies the position of the utility pole 68 extracted from the captured image and the previously identified position of the utility pole 68 in the high precision map data 46. The identification process here may be the same as that in step ST44. This allows the ECU 44 to identify, in the high-precision map data 46, the position of the utility pole 68 as a single upright structure included in the latest image captured by the exterior camera 26, and the position of the utility pole 68 based on past captured images.

[0087] From step ST56, the ECU 44 specifically starts the process of generating the position of the automobile 3 based on a plurality of captured images including the image captured by the exterior camera 26. In this embodiment, the ECU 44 generates the current position of the automobile 3 through the multiple-image position generation process by two-stage processing: the position generation process in step ST56 and the orientation generation process in step ST57. In the position generation processing of step ST56, the ECU 44 generates the position of the automobile 3 based on the position of the utility pole 68 identified based on the current captured image and the position of the utility pole 68 identified based on the previous captured image. The position of the automobile 3 may be a position where the distance from the current utility pole 68 is the relative distance Ld and the distance from the previous utility pole 68 is the relative distance calculated in step ST54. The ECU 44 may generate the position of the vertex corresponding to the automobile 3 based on a triangle in the high-precision map data 46, with the two utility poles 68 and the automobile 3 as vertices. In the orientation generation process of step ST57, the ECU 44 generates, as the orientation of the automobile 3, an orientation in which the direction from the position of the automobile 3 toward the current utility pole 68 is the relative angle θd generated based on the captured image. This allows the ECU 44 to generate the position and orientation of the vehicle 3 .

[0088] In step ST58, the ECU 44 sets the current position of the automobile 3 to the position and orientation of the automobile 3 determined in the processing up to step ST57.

[0089] This allows the ECU 44 to generate the current position of the automobile 3 through a multiple image position generation process based on the image captured by the outside camera 26 and the previously captured images. Next, the position generation process in the multiple image position generation process will be described in detail with reference to FIG. The road on which the automobile 3 actually travels is in a three-dimensional space, but for ease of explanation, the following description will be given using a two-dimensional space of XY.

[0090] FIG. 14 is an explanatory diagram of a method for generating the position of the automobile 3 by the multiple image position generation process. Figure 14 shows a car 3 traveling from the bottom to the top of the figure, as well as a first utility pole 99 identified based on an image captured by the exterior camera 26 and a second utility pole 98 identified based on a previous image captured. The first utility pole 99 is a utility pole that has been identified by the ECU 44 in the current process as the utility pole 68 included in the latest image captured by the outside camera 26 . The second utility pole 98 is a utility pole that was identified by the ECU 44 in a previous process as the utility pole 68 included in the previous captured image. In FIG. 14, the current position of the traveling automobile 3 is depicted by a solid line, and the past position is depicted by a dashed line.

[0091] In the position generation process of the multiple-image position generation process, the ECU 44 calculates the amount of movement from the past positions of the automobiles 101, 103 to obtain a tentative current position of the automobile 3. Such a tentative current position may basically be generated based on the position and orientation based on GNSS. However, if the calculation is based on the past position of the automobile 103, whose position and orientation are obtained with high accuracy based on GNSS, there is a possibility that the error in the amount of movement based on the measurement of the host vehicle sensor will be large. For this reason, it is preferable to use the previous position of the automobile 101, shown by the dashed line in the center of the figure, as the reference, rather than the position of the automobile 103, shown by the dashed line in the bottom of the figure. By reducing the amount of movement, the error in the position calculated using it can be reduced, improving the accuracy of the position. In this case, the ECU 44 can estimate the distance of the automobile 3 from the previously identified second utility pole 98 recorded in the memory 43 at the time when the exterior camera 26 captured the latest image.

[0092] Next, the ECU 44 calculates the relative distance LP from the second utility pole 98 to the automobile 102 in the high precision map data 46. The ECU 44 calculates the relative distance LP from the automobile 102, which is at its tentative current position, to the second utility pole 98 in the high precision map data 46. Furthermore, the ECU 44 calculates the pole-to-pole distance Lm from the position of the first utility pole 99 to the position of the second utility pole 98 based on the high-precision map data 46 . Furthermore, the ECU 44 generates a relative distance Ld from the tentative current position of the automobile 102 to the first utility pole 99 based on the captured image of the automobile 3 . The ECU 44 then applies the theorem of trigonometric functions to a triangle having these three sides to generate the position of the automobile 3. Next, the ECU 44 performs the same orientation generation process as that described in the lower part of Fig. 11 to generate, as the orientation of the automobile 3, an orientation in which the direction from the automobile 3 at the specified position toward the identified first utility pole 99 is the relative angle generated based on the captured image. The ECU 44 generates the orientation of the automobile 3 in the high-precision map data 46 using the angle-of-view position θd of the first utility pole 99 in the most recent captured image by the exterior camera 26. This allows the ECU 44 to generate the position and orientation of the automobile 3 based on a plurality of captured images. The position generated in this way is likely to be more accurate than the above-mentioned position based on a single utility pole or pole. Utility poles and poles are basically cylindrical in shape. Therefore, even if an image of a single utility pole or pole is captured, the relative vehicle direction from that pole may not be highly accurate. The vehicle's position may not be clearly defined in which direction on the circumference of a cylindrical utility pole or pole is the center. By determining the vehicle's position based on multiple utility poles or poles, as in this case, the accuracy of the position can be improved.

[0093] The ECU 44 may calculate the relative direction instead of the relative distance LP between the vehicle 102 at its tentative current position and the past second utility pole 98. In this case, the ECU 44 can generate the position of the vehicle 3 based on the relative direction from the second utility pole 98 and the relative angle from the first utility pole 99. The ECU 44 can generate the current position of the vehicle 3 in the high-precision map data 46 based on the distance or direction from each of the multiple utility poles 98, 99 whose positions have been identified.

[0094] As described above, in this embodiment, in the multiple image position generation process, the ECU 44 estimates the relative distance LP of the automobile 3 from the previous upstanding structure recorded in the memory 43 at the time the latest image was captured by the exterior camera 26. The ECU 44 also identifies the position of the latest upstanding structure included in the latest image captured by the exterior camera 26 and the positions of previous upstanding structures in the high-precision map data 46. The ECU 44 can then generate a reliable current position of the automobile 3 in the high-precision map data 46 based on the distance from each of the multiple upstanding structures whose positions have been identified. In this case, the ECU 44 can also generate the orientation of the automobile 3 in the high-precision map data 46 with high accuracy by using the angle of view position of the upright structure in the latest image captured by the exterior camera 26. The position and orientation of the automobile 3 obtained by such multiple image position generation processing can be probable and suitable for driving control of the automobile 3 even if the automobile remains in a dead recognition state for a long period of time, during which highly accurate position information cannot be obtained from the GNSS receiver 25. For example, even if the automobile 3 is traveling in a dead recognition state on a long, straight road or lane that is covered in snow or ice, the automobile 3 can continue to travel while maintaining a position near the center of the long, straight road or lane by continually updating its current position to a probable position.

[0095] As described above, in this embodiment, the high-precision map data 46 stored in the memory 43 includes at least position information about upstanding structures, such as utility poles 68, that are erected along the road on which the automobile 3 is traveling. Here, the information about the positions of the upstanding structures may be information about the positions of a plurality of utility poles 68 and other poles that are erected in a line along the road on which the automobile 3 is traveling. The ECU 44 acquires and processes information from the memory 43 and the exterior camera 26, generates information about the upstanding structures included in the images captured by the exterior camera 26, and identifies the generated positions of the upstanding structures in the high-precision map data 46. The ECU 44 then generates the current position of the automobile 3 on the road on which the automobile 3 is traveling in the high-precision map data 46, based on the positions of the upstanding structures identified in the high-precision map data 46 and the captured images. In particular, in this embodiment, the standing structures identified to generate the position include at least utility poles 68 and other poles arranged in a line along the road on which the automobile 3 travels. Unlike signs 66 and the like, such standing structures are widely and generally used along the road on which the automobile 3 travels. Furthermore, standing structures are less likely to be entirely covered by snow or soil, unlike zebra stripes 62, stop lines 63, traffic marks 64, and other structures painted on the road surface. Furthermore, at least a portion of the standing structures near the road surface can be captured even in the morning or evening when the sun is low in the sky. As a result, while the automobile 3 is traveling, the ECU 44 can continue to capture images of the standing structures using images captured by the exterior camera 26. Furthermore, the ECU 44 can generate information about the standing structures contained in the captured images with high reliability. Moreover, the standing structures have the characteristic of being arranged in a line along the road on which the automobile 3 travels. Therefore, the ECU 44 can continue to generate a current position of the automobile 3 with a high degree of certainty by correcting the position of each standing structure every time an image of the structure is captured. In this way, in this embodiment, the accuracy of the generated vehicle position can be improved.

[0096] In this embodiment, there is a possibility that the ECU 44 will only be able to generate information about one utility pole 68 from the image captured by the exterior camera 26. In this case, the ECU 44 will only be able to identify one utility pole 68 in the map data, and will only be able to generate the position of the automobile 3 based on that one utility pole 68. In contrast, if, for example, multiple utility poles 68 can be extracted from the image captured by the exterior camera 26, the ECU 44 will be able to improve the accuracy of the position of the automobile 3 based on these. To compensate for this lack of information, in this embodiment, information about utility poles 68 generated based on past captured images is recorded in memory 43. Even if ECU 44 can generate information about only one utility pole 68 in the image captured by exterior camera 26, it can use that information as well as the past information about utility poles 68 recorded in memory 43 to generate a probable current position based on the positions of multiple utility poles 68. ECU 44 can perform not only single-image position generation processing based only on the image captured by exterior camera 26, but also multiple-image position generation processing using past positions of utility poles 68. Then, the ECU 44 may select either the single image position generation process or the multiple image position generation process as the final current position of the automobile 3, depending on the driving conditions of the automobile 3 and past information on the utility pole 68 recorded in the memory 43. For example, if the information about the past utility pole 68 recorded in the memory 43 is not older than the threshold value, or if the position of the past utility pole 68 recorded in the memory 43 is not farther than the threshold value, the ECU 44 may select the position obtained by the multiple-image position generation process as the final current position of the automobile 3. On the other hand, if the information about the past utility pole 68 recorded in the memory 43 is older than the threshold value, or if the position of the past utility pole 68 recorded in the memory 43 is farther than the latest position of the utility pole 68 by more than the threshold value, the ECU 44 may select the position obtained by the single-image position generation process as the final current position of the automobile 3.

[0097] [Second embodiment] Next, a second embodiment of the present invention will be described. Differences from the above-described embodiment will be mainly described below. Features similar to those in the above-described embodiment will be designated by the same reference numerals as in the above-described embodiment, and a description thereof will be omitted. In this embodiment, the server device 31 controls the running of the automobile 3 .

[0098] As shown in FIG. 2 , the control system 10 of the autonomously driven automobile 3 includes an external communication device 17 that can communicate with an external server device 31 via a base station 30. In this case, the server device 31 can collect and acquire information about the driving of each of the automobiles 3 traveling on a road including a merging section, and generate driving control values ​​to be used for driving control of each automobile 3 based on the information. The control system 10 of the automobile 3 can also receive and acquire driving control values ​​from the server device 31 and use them for driving control of the automobiles 3. In this case, the server device 31 controls the driving of the automobiles 3 traveling in the merging section remotely or by remote control. In this way, the server device 31 can acquire information about the driving of the automobiles 3 traveling in the merging section from the automobiles 3 traveling in the merging section. Based on the acquired information, the server device 31 can generate driving control values ​​to control the driving of the automobiles 3 traveling in the merging lane L so that the automobile 3 to be controlled merges into the merging lane L immediately before the merge.

[0099] FIG. 15 is an explanatory diagram of a server device 31 that controls the running of an automobile 3 according to the second embodiment of the present invention. The server device 31 in FIG. 15 includes a communication device 51, a server timer 52, a server memory 53, a server CPU 54, and a server bus 55 to which these are connected.

[0100] The communication device 51 is connected to a communication network such as the Internet. The communication device 51 transmits and receives information to and from the automobile 3 traveling on a road via, for example, a base station 30 connected to the communication network. The communication device 51 is a communication unit capable of communicating with the automobile 3 to control or assist the traveling of the automobile 3. The server timer 52 measures the time or duration. The time of the server timer 52 may be calibrated, for example, based on the time of radio waves from a GNSS satellite (not shown). In this case, the time of the server timer 52 is synchronized with the time of the automobile 3. The server memory 53 stores programs and data executed by the server CPU 54. The server memory 53 may be configured, for example, with a non-volatile semiconductor memory, a HDD, a RAM, or the like. The server CPU 54 reads and executes the program recorded in the server memory 53. This realizes a server control unit. The server CPU 54 as the server control unit manages the operation of the server device 31. The server control unit can function as a vehicle driving control device that remotely controls the driving of the automobile 3. In this way, the server CPU 54 functions as a driving control unit in the server device 31 that generates driving control information for controlling the driving of the automobile 3. Furthermore, the communication device 51 transmits to the automobile 3 the driving control information generated by the driving control unit.

[0101] FIG. 16 is a flowchart of the server drive control by the server device 31 of FIG. The server CPU 54 may repeatedly execute the server travel control of FIG. 16 as the travel control unit of the server device 31.

[0102] In step ST61, the server CPU 54 receives vehicle information from the automobile 3. The automobile 3 transmits, as vehicle information, detection information from its own vehicle sensors, such as captured images from external sensors, from the external communication device 17 to the server device 31 via the base station 30. The communication device 51 of the server device 31 receives the vehicle information transmitted by the automobile 3. The communication device 51 may receive vehicle information from multiple automobiles 3.

[0103] In step ST62, the server CPU 54 executes driving control for the automatic driving of the automobile 3. The server CPU 54 acquires or generates the current position of the automobile 3 and generates a route from the current position. In this case, the server CPU 54 may map the automobiles 3 traveling on the road on the high-precision map data 46 recorded in the server memory 53, and generate a route for each automobile 3 that does not interfere with other automobiles. The server CPU 54 generates driving control values ​​for each automobile 3 that enable safe and smooth driving on each route. In addition, when acquiring or generating the current position of the automobile 3, the server CPU 54 may determine the position to be acquired from each automobile 3 using processing similar to that of the automatic driving control in Figure 5, and acquire or generate the current position for driving control.

[0104] In step ST63, the server CPU 54 transmits the generated cruise control values ​​to the automobile 3 that is the target of cruise control. The cruise control values ​​are transmitted from the communication device 51 of the server device 31 to the external communication device 17 of the automobile 3 via a base station. The ECU 44 of the cruise control device 15 of the automobile 3, for example, may output the cruise control values ​​received from the server device 31 to the drive control device 11, steering control device 12, and braking control device 13, replacing the control values ​​generated by the automobile itself.

[0105] In this way, the server CPU 54 of the server device 31 can execute the same processing as the ECU 44 of the driving control device 15 of the control system 10 of the automobile 3 in the above-described embodiment in step ST62 of FIG. In this case, the server CPU 54, as a driving control unit of the server device 31, can generate driving control values ​​for the automobile 3, for example, assuming that the automobile 3 is located at a current position generated based on the captured image.

[0106] The above-described embodiment is an example of a preferred embodiment of the present invention, but the present invention is not limited to this, and various modifications and changes are possible within the scope of the gist of the invention.

[0107] In the above-described embodiment, the ECU 44 uses the position and orientation of the automobile 3 obtained by the third generation process in the single image position generation process for controlling the running of the automobile 3. Additionally, for example, in the single image position generation process, the ECU 44 may use the position and orientation of the automobile 3 obtained by the first generation process to control the driving of the automobile 3. Also, in the single image position generation process, the ECU 44 may use the position and orientation of the automobile 3 obtained by the second generation process to control the driving of the automobile 3. Furthermore, the ECU 44 may switch between the position and orientation of the automobile 3 obtained by these multiple generation processes depending on the driving conditions of the automobile 3, and use them to control the driving of the automobile 3.

[0108] In the above-described embodiment, the ECU 44 switches between a single image position generation process based only on the image captured by the exterior camera 26 and a multiple image position generation process that also uses past information on utility poles 68 stored in the memory 43, depending on the driving conditions of the automobile 3, and uses these to control the driving of the automobile 3. Alternatively, for example, the ECU 44 may use only the single-image position generation process based only on the image captured by the exterior camera 26 for controlling the running of the automobile 3. Alternatively, the ECU 44 may use only the multiple-image position generation process for controlling the running of the automobile 3.

[0109] In the above-described embodiment, the ECU 44 generates information from the captured image only about utility poles 68 and other poles that are erected in a row along the road on which the automobile 3 is traveling, as information about erect structures that are used as the basis for driving control. In addition to this, for example, the ECU 44 may also generate information about standing structures other than utility poles 68 erected along the road, such as signboards 66 installed near the road, from the captured image as information about the standing structures to be used as the basis for driving control. Furthermore, the ECU 44 may generate information about hollow structures such as road signs 65 and traffic lights 61 that are provided in the air above the road from the captured image as information about the structures to be used as a reference for driving control. In addition, the ECU 44 may generate road surface drawings such as zebra zones 62, stop lines 63, and traffic marks 64 drawn on the road surface from the captured image as information on structures to be used as the basis for driving control. In this way, the ECU 44 may generate, from the captured image, information on structures that serve as a reference for cruise control, other than the multiple upright structures lined up along the road. However, when dealing with various driving conditions of the vehicle 3, such as snowfall and ice, it is desirable to generate, from the captured image, information on structures that serve as a reference for cruise control, at least for multiple upright structures lined up along the road on which the vehicle 3 travels, such as utility poles 68, boundary poles 67, and other poles.

[0110] FIG. 17 is a flowchart of the selection control of a structure to be used for control when a plurality of structures are extracted from a captured image. The ECU 44 of the driving control device 15 of FIG. 2 may execute the structure selection control of FIG. 17, for example, in step ST41 of FIG. 8 or step ST51 of FIG. 13, and as a result, select the utility pole extracted from the captured image. The server CPU 54 of the server device 31 may also execute the structure selection control of FIG. 17 in step ST62 of FIG. 16, and as a result, select the utility pole extracted from the captured image.

[0111] In step ST71, the ECU 44 analyzes the image captured by the exterior camera 26 and extracts structures such as utility poles included in the image. Depending on the position and direction in which the automobile 3 is traveling, various structures may be extracted from the image captured by the exterior camera 26.

[0112] In step ST72, the ECU 44 determines whether a plurality of structures have been extracted from the captured image. If a plurality of structures have been extracted, the ECU 44 proceeds to step ST73. If one structure has been extracted, the ECU 44 proceeds to step ST77.

[0113] In step ST73, the ECU 44 determines whether or not a utility pole is extracted from the extracted plurality of structures. If a utility pole is extracted, the ECU 44 proceeds to step ST77. Otherwise, that is, if a utility pole is not extracted, the ECU 44 proceeds to step ST74.

[0114] In step ST74, the ECU 44 determines whether or not any standing structures other than utility poles have been extracted from the extracted plurality of structures. Examples of standing structures other than utility poles include boundary poles 67, and poles supporting traffic lights 61 and road signs 65. If any standing structures other than utility poles have been extracted, the ECU 44 proceeds to step ST77. Otherwise, the ECU 44 proceeds to step ST75.

[0115] In step ST75, the ECU 44 determines whether or not a hollow structure is extracted from the extracted plurality of structures. Examples of hollow structures include a traffic light 61 and a road sign 65. If a hollow structure is extracted, the ECU 44 proceeds to step ST77. Otherwise, the ECU 44 proceeds to step ST76.

[0116] In step ST76, the ECU 44 determines whether or not a road surface feature has been extracted from the extracted multiple structures. Examples of road surface features include zebra zones 62, stop lines 63, and traffic marks 64. If a road surface feature has been extracted, the ECU 44 proceeds to step ST77. Otherwise, the ECU 44 ends this control without selecting the extracted feature for generating the current position. In this case, the ECU 44 may execute driving control of the vehicle 3 by setting the position of the vehicle 3 based on, for example, GNSS as the current position of the vehicle 3 without generating the position of the vehicle 3 based on the captured image.

[0117] In step ST77, the ECU 44 selects the selected structure as a structure to be identified in the high-precision map data 46 in order to generate a position based on the captured image. Thereafter, the ECU 44 ends this control. In this case, the ECU 44 identifies the position of the selected structure in the high-precision map data 46, and generates the position of the vehicle 3 based on that.

[0118] As a result, when multiple types of structures can be extracted from the captured image, the ECU 44 can preferentially select utility poles and other standing structures from among the multiple types of structures and use them to generate a position based on the captured image. By preferentially selecting utility poles and other standing structures that are less susceptible to snowfall and road ice, the ECU 44 can generate the position of the moving automobile 3 regardless of the driving environment. Moreover, since the ECU 44 preferentially selects utility poles that are more likely to be captured by the exterior camera 26 of the moving automobile 3, the processing time until the selection is completed can be shortened compared to when utility poles are selected later. As a result, the processing time for driving control, which is desirably executed repeatedly in a short period of time while the automobile 3 is traveling, can be effectively suppressed. The automobile 3 can repeat driving control at short intervals while traveling. [Explanation of symbols]

[0119] 1...Urban road, 2...Suburban road, 3...Automobile (vehicle), 10...Control system (vehicle position generation device), 11...Drive control device, 12...Steering control device, 13...Braking control device, 14...Operation detection device, 15...Driving control device, 16...Detection control device, 17...External communication device, 18...Central gateway device, 21...Steering, 22...Accelerator pedal, 23...Brake pedal, 24...Shift lever, 25...GNSS receiver, 26...External camera (imaging device), 27...Lidar, 28...Acceleration sensor, 29...Distance sensor, 30...Base station, 31...Server device (vehicle position generation device), 41...Input / output device, 42...Timer, 43...Memory, 44...ECU, 45...Internal bus, 46...High-precision map data 51...communication device, 52...server timer, 53...server memory, 54...server CPU, 55...server bus, 61...traffic light, 62...zebra zone, 63...stop line, 64...traffic mark, 65...road sign, 66...signboard, 67...boundary pole, 68...electric pole, 69...building, 70...captured image of urban area, 71...image of road surface at intersection, 72...image of zebra zone, 73...image of traffic light, 74...image of electric pole, 80...captured image of suburban area, 81...image of long road surface, 82...image of signboard, 83...image of boundary pole, 84...image of electric pole, 92...identified electric pole, 94...first vehicle, 95...second vehicle, 97...third vehicle, 98...second electric pole, 99...first electric pole, 101, 103...vehicle at past location, 102...vehicle at current location

Claims

1. a memory for recording map data including at least position information of a plurality of standing structures arranged along a road on which a vehicle travels; an imaging device that captures an image of the front, which is the traveling direction of the vehicle; a control unit that acquires and processes information from the memory and the imaging device; and the memory is capable of recording information about the standing structure generated based on a previously captured image; The control unit a single-image position generation process for generating information about the standing structure included in the image captured by the imaging device, identifying the generated position of the standing structure in the map data, and generating the position of the vehicle based on the position of the standing structure identified in the map data and the captured image, in order to generate the position of the vehicle based on information about the standing structure included in the image captured by the imaging device; a multiple image position generation process for generating a position of the vehicle based on information about an erected structure included in the image captured by the imaging device and information about the erected structure from the past recorded in the memory; generating a position of the vehicle by a process selected from the following: Vehicle position generator.

2. The map data includes, as information on the positions of the plurality of erected structures, information on the positions of at least a plurality of utility poles that are arranged side by side along a road on which the vehicle travels, The control unit generating information on the angle of view of the electric pole included in the captured image from the vehicle, information on the relative distance from the vehicle, and information on the position of the electric pole based on the captured image of the imaging device; Identifying utility poles in the map data based on the generated relative positional relationships of the plurality of utility poles; The vehicle position generating device according to claim 1 .

3. the memory is capable of recording information about the standing structure generated based on the past captured images, The control unit The position of the vehicle can be generated based on information about the standing structure included in the image captured by the imaging device and information about the standing structure from the past recorded in the memory.

3. The vehicle position generating device according to claim 1 or 2.

4. The control unit, in the single image position generation process, Identifying the position of an upright structure included in the image captured by the imaging device in the map data; generating a position of the vehicle in the map data based on a view angle position of an upright structure in an image captured by the imaging device; The device for generating a vehicle position according to any one of claims 1 to 3.

5. The control unit, in the single image position generation process, a first generation process for generating, as the position of the vehicle, a position of the identified standing structure, the position being a distance estimated from the captured image while maintaining the orientation of the identified standing structure at a position of an angle of view in the captured image of the imaging device; a second generation process for generating a position of the vehicle that is between the position of the standing structure identified in the map data and a tentative position of the vehicle before correction and that is a distance estimated from the captured image, and further generating a direction of the vehicle that obtains a field angle position of the identified standing structure in the captured image of the imaging device at the generated position; a third generation process for generating a position between the position obtained by the first generation process and the position obtained by the second generation process as a final position of the vehicle; generating a position of the vehicle in the map data by any one of the generation processes 5. The device for generating a vehicle position according to any one of claims 1 to 4.

6. The control unit, in the multiple image position generation process, estimating a distance of the vehicle from the standing structure at the time the image was captured by the imaging device, as recorded in the memory; In the map data, a position of an upstanding structure included in an image captured by the imaging device is identified, and a previous position of the upstanding structure from which a distance has been estimated is identified; generating a position of the vehicle in the map data based on a distance from each of the plurality of standing structures whose positions have been identified; generating a direction of the vehicle in the map data using a view angle position of an upright structure in the image captured by the imaging device; The device for generating a vehicle position according to any one of claims 1 to 5.

7. a location information generating device that provides information about the location of the vehicle to the control unit; The control unit generating a position of the vehicle based on an image captured by the imaging device when a predetermined time has elapsed since it was determined that the position information was not provided from the position information generating device, or when a predetermined time has elapsed since it was determined that the reliability of the position information provided by the position information generating device was not high; 7. The device for generating a vehicle position according to any one of claims 1 to 6.

8. The control unit at least until the relative positional relationship between the vehicle position provided by the position information generating device or a position based thereon and the identified standing structure in the map data becomes consistent with the angle of view position of the identified standing structure in the image captured by the imaging device, generating a position of the vehicle based on an image captured by the imaging device; The vehicle position generating device according to claim 7.

9. A vehicle comprising a driving control device capable of controlling driving using a position generated by the vehicle position generation device according to any one of claims 1 to 8.

10. a driving control unit that generates driving control information for controlling the driving of the vehicle; a transmitting device that transmits driving control information generated by the driving control unit to the vehicle, The driving control unit generates the driving control information assuming that the vehicle is located at a position generated by the vehicle position generation device according to any one of claims 1 to 8. Server device.

Images