Vehicle driving control device and vehicle driving control system

Abstract

To suppress a requirement for an imaging device while securing control performance in a travel control for a vehicle.SOLUTION: A travel control device for a vehicle 1, including a travel control unit that controls travel on the basis of positioning of the vehicle 1, acquires a timing when the travelling vehicle 1 passes through a specific position whose position has been found, acquires a position without according to the positioning of the vehicle 1 as position information on the specific position for acquiring the passage timing, and generates corrected information using the position of the specific position as a reference for the positioning of the vehicle 1 in the passage timing. The travel control unit controls travel of the vehicle 1 by using the corrected information generated for the positioning of the vehicle 1 in the passage timing of the specific position to correct the positioning of the vehicle 1 in the passage timing.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a vehicle cruise control device and a vehicle cruise control system. [Background technology]

[0002] Patent Document 1 discloses that a driving control unit provided in a vehicle controls the driving of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-010539 A Summary of the Invention [Problem to be solved by the invention]

[0004] In such vehicle travel control, control errors occur due to, for example, uneven road surfaces, etc. If control errors due to various causes overlap, the position of the controlled vehicle, for example, may end up being displaced from the actual position in the environment. For example, if a vehicle is equipped with a monocular imaging device, its position is determined based on an image captured by the monocular imaging device, and the vehicle's driving is controlled based on this positioning, the position of the vehicle, whose driving is controlled based on the positioning determined by the monocular imaging device, may deviate significantly from the actual environment due to a number of factors, including not only differences between the road surface model used with the monocular imaging device and the actual road surface on which the vehicle is driving, but also installation errors and changes in the monocular imaging device to the vehicle, and the performance of the monocular imaging device itself, such as its resolution. In order to suppress such positional deviation, it is necessary to use a high-precision, high-performance monocular imaging device, or to position the monocular imaging device with high precision on the vehicle so as to match the road surface model and to attach it firmly. If these measures are not enough to suppress the deviation, it becomes necessary to use a more high-precision, high-performance device, such as a compound-eye imaging device. In vehicle driving control, it is required to use high-precision, high-performance vehicle sensors, such as imaging devices.

[0005] In this way, in vehicle driving control, there is a need to reduce the demands on vehicle sensors such as imaging devices while ensuring control performance. [Means for solving the problem]

[0006] The vehicle driving control device according to the present invention is a vehicle driving control device having a driving control unit that controls driving of the vehicle based on vehicle positioning, An imaging device provided in the vehicle capable of travelling for positioning; a timing acquisition unit that acquires a timing when the traveling vehicle passes a specific position whose position is known; a position acquisition unit that acquires a position of the specific position not based on positioning of the vehicle as position information of the specific position of which the timing acquisition unit acquires the passing timing; and a generation unit that generates correction information for the positioning of the vehicle at the passing timing acquired by the timing acquisition unit, based on the position of the specific position acquired by the position acquisition unit; The position acquisition unit acquires, as position information of the specific object at the specific position, position information of a stop line drawn on a road surface of a road on which the vehicle is traveling, and the timing acquisition unit acquires, as the passing timing, a timing at which the stop line drawn on the road surface of the road on which the vehicle is traveling disappears downward so as to be hidden by a body of the vehicle in an image captured by the imaging device provided on the vehicle while the vehicle is traveling, and the generation unit generates information on a peripheral imaging angle of view in a vertical direction of the imaging device provided on the vehicle, a timing of the image captured by the imaging device, and a passing timing of the image captured by the imaging device, based on a position of the position information of the stop line at the specific position acquired by the position acquisition unit. a position of the vehicle at the passing timing based on the position of the stop line position information acquired by the position acquisition unit and the acquired relative position, and generates the correction information that corrects the position of the vehicle obtained by the position acquisition unit to the generated position of the vehicle, wherein a peripheral imaging angle of view in the vertical direction of the imaging device used in the calculation is a peripheral imaging depression angle obtained by excluding a range of an angle of view in the vertical direction of the imaging device that is hidden by a body of the vehicle, The traveling control unit performs positioning of the vehicle while traveling at the specific position The stop line disappears. The vehicle's travel is controlled by correcting the position of the vehicle using the correction information generated for the position of the vehicle at the timing.

[0012] Preferably, the position acquisition unit acquires, as the position information of the specific object, position information of a three-dimensional object provided on a road on which the vehicle is traveling, and the timing acquisition unit acquires, as the passing timing, the timing at which the three-dimensional object provided on the road on which the vehicle is traveling disappears upward from an image captured by the imaging device provided on the vehicle of the surroundings in front of the vehicle, and the generation unit calculates and generates the relative position of the vehicle and the three-dimensional object at the passing timing using information on the peripheral imaging field of view in the vertical direction of the imaging device provided on the vehicle, as well as information on the height of the imaging device from the road surface and information on a pitch angle that indicates a change in attitude of the vehicle in the vertical direction.

[0013] Preferably, the position acquisition unit acquires, as the position information of the specific object, position information of a three-dimensional object located adjacent to the road on which the vehicle is traveling, and the timing acquisition unit acquires, as the passing timing, the timing at which the three-dimensional object located adjacent to the road on which the vehicle is traveling disappears sideways from an image captured by the imaging device mounted on the vehicle of the surroundings in front of the vehicle, and the generation unit calculates and generates the relative position of the vehicle and the three-dimensional object at the passing timing using information on the peripheral imaging angle of view in the vehicle width direction of the imaging device mounted on the vehicle, as well as information on the distance of the imaging device from a reference position in the vehicle width direction and information on a yaw angle that indicates a change in attitude of the vehicle in the vehicle width direction.

[0014] Preferably, the driving control unit controls the driving of the vehicle using map data of a positioning system including position information of the specific position.

[0015] The vehicle driving control system according to the present invention is a vehicle having a driving control unit that controls driving of the vehicle based on vehicle positioning, and a server device that can communicate with the vehicle, wherein the vehicle has: An imaging device provided in the vehicle capable of travelling for positioning;a timing acquisition unit that acquires a timing when the traveling vehicle passes a specific position whose position is known; a position acquisition unit that acquires a position of the specific position not based on positioning of the vehicle as position information of the specific position for which the timing acquisition unit acquires passing timing; and a generation unit that generates correction information for the positioning of the vehicle at the passing timing acquired by the timing acquisition unit, based on the position of the specific position acquired by the position acquisition unit. The imaging device the server device includes at least the generating unit among the timing acquiring unit, the position acquiring unit, and the generating unit for the driving control system, which are not provided in the vehicle; The position acquisition unit acquires, as position information of the specific object at the specific position, position information of a stop line drawn on a road surface of a road on which the vehicle is traveling, and the timing acquisition unit acquires, as the passing timing, a timing at which the stop line drawn on the road surface of the road on which the vehicle is traveling disappears downward so as to be hidden by a body of the vehicle in an image captured by the imaging device provided on the vehicle while the vehicle is traveling, and the generation unit generates information on a peripheral imaging angle of view in a vertical direction of the imaging device provided on the vehicle, a timing of the image captured by the imaging device, and a passing timing of the image captured by the imaging device, based on a position of the position information of the stop line at the specific position acquired by the position acquisition unit. a position of the vehicle at the passing timing based on the position of the stop line position information acquired by the position acquisition unit and the acquired relative position, and generates the correction information that corrects the position of the vehicle obtained by the position acquisition unit to the generated position of the vehicle, wherein a peripheral imaging angle of view in the vertical direction of the imaging device used in the calculation is a peripheral imaging depression angle obtained by excluding a range of an angle of view in the vertical direction of the imaging device that is hidden by a body of the vehicle, The traveling control unit of the vehicle performs positioning of the vehicle while traveling at the specific position The stop line has disappeared The vehicle positioning device corrects the position of the vehicle using the correction information generated for the positioning of the vehicle at the timing of the vehicle positioning, and controls the traveling of the vehicle. Effect of the Invention

[0016] In the present invention, correction information for correcting the positioning of a traveling vehicle is generated based on the traveling vehicle passing through a specific position whose position is known. Specifically, in the present invention, a position that is not based on the positioning of the vehicle is acquired as the position information of the specific position whose passing timing is acquired. Also, in the present invention, the timing when the traveling vehicle passes through the specific position whose position is known is acquired, and correction information is generated for the positioning of the vehicle at the acquired passing timing based on the position of the specific position acquired by the position acquisition unit. As a result, in the present invention, the positioning of the vehicle can be corrected so as to be aligned with a positioning system including the position information of the specific position. Then, for example, the traveling control unit controls the traveling of the vehicle using map data by the control system, so that the traveling control unit can control the traveling of the vehicle in a state where it is expected that the possibility of safe traveling is high. In this way, in the present invention, when the vehicle position is measured, for example, the vehicle position can be adjusted to match the vehicle position with the positioning system of a specific position whose position is known, by using the correction information. In the present invention, a simple method based on passing through a specific position while driving can be used to reduce the requirements for the vehicle sensor such as an imaging device used for driving control, while ensuring the performance of the driving control. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is an explanatory diagram of a driving state of an automobile to which a driving control device according to an embodiment of the present invention is applied. [Diagram 2] FIG. 2 is an explanatory diagram of a state in which the car in FIG. 1 is traveling forward along a road that intersects with other roads toward an intersection. [Figure 3A] FIG. 3A is an explanatory diagram of an image capturing state by a monocular imaging device provided facing forward at the front of an automobile. [Figure 3B] FIG. 3B is an explanatory diagram of an image captured by the monocular imaging device in the imaging state of FIG. 3A. [Figure 3C] FIG. 3C is an explanatory diagram of measuring the relative position of a pedestrian included in the captured image of FIG. 3B. [Figure 4] FIG. 4 is an explanatory diagram of a control system 30 that is provided in the automobile of FIG. 1 and functions as a cruise control device. [Diagram 5] FIG. 5 is a flowchart of driving control for locating the vehicle position by the driving control unit in FIG. [Figure 6] FIG. 6 is a flowchart of outside-vehicle image capture control by the CPU of the outside-vehicle image capture device of FIG. 4 for capturing an image of the surroundings in front of the automobile and locating the positions of surrounding objects. [Figure 7] FIG. 7 is a flowchart showing control for generating position correction information for automobile positioning by the CPU of the outside-vehicle image capturing device of FIG. [Figure 8A] FIG. 8A is an explanatory diagram of an image captured by the monocular imaging device of FIG. 4, capturing an image of a stop line as a specific object located at a specific position. [Figure 8B] FIG. 8B is an explanatory diagram of an image captured by the monocular imaging device of FIG. 4 capturing an image of a traffic light as a specific object at a specific position. [Figure 8C] FIG. 8C is an explanatory diagram of an image captured by the monocular imaging device of FIG. 4, capturing an image of a roadside signboard as a specific object at a specific position. [Figure 9] FIG. 9 is an explanatory diagram of the positional relationship between the stop line and the automobile at the time when the stop line disappears from the captured image. [Figure 10] FIG. 10 is an explanatory diagram of the positional relationship between a traffic light and a car at the time when the traffic light disappears from a captured image. [Figure 11] FIG. 11 is an explanatory diagram of the positional relationship between a roadside sign and a car at the time when the roadside sign disappears from a captured image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0019] FIG. 1 is an explanatory diagram of a driving state of an automobile 1 to which a driving control unit 32 according to an embodiment of the present invention is applied. An automobile 1 is an example of a vehicle. The automobile 1 in Fig. 1 has a vehicle body 2. The vehicle body 2 is provided with a plurality of wheels 5 and a passenger compartment 3 in which passengers ride. The passenger compartment 3 is provided with a seat 4 in which the passengers sit. As described later, the vehicle body 2 of the automobile 1 is provided with a monocular imaging device 41 facing forward at the front of the passenger compartment 3. The monocular imaging device 41 captures an image of the front, which is the traveling direction of the automobile 1, at a predetermined angle of view. The front of the vehicle body 2 is reflected in the captured image of the monocular imaging device 41. For this reason, the peripheral imaging angle of view at which the monocular imaging device 41 can actually capture an image of the surroundings in the vertical direction is smaller than the original angle of view of the monocular imaging device 41.

[0020] 1 receives radio waves from a GNSS satellite 101 to determine its own position, and while checking its own position in, for example, high-precision map data 34, its driving is controlled so that it travels forward along a flat, straight road 10 toward an intersection equipped with a traffic light 12 and a stop line 11. A building 13 stands on the shoulder of the road 10 in FIG. 1. In this case, when the traffic light 12 is green, the automobile 1 may be controlled to travel so as to enter the intersection without stopping at the stop line 11. When turning right or left at an intersection, the automobile 1 may be controlled to decelerate or stop at the stop line 11 or at a position past the stop line 11, and turn right or left while checking to the left and right.

[0021] Incidentally, the road 10 on which the automobile 1 travels is not necessarily flat. For example, as shown by the dashed line in Fig. 1, the road 10 on which the automobile 1 travels may become a steep uphill slope immediately before an intersection where a traffic light 12 and a stop line 11 are provided. In this case, the positions of the traffic light 12 and the stop line 11 in the image captured by the monocular imaging device 41 provided on the automobile 1 are different from those shown by the solid line in Fig. 1. Specifically, the traffic light 12 and the stop line 11 are captured in an upper position in the captured image compared to those shown by the solid line in Fig. 1. Even in such a traveling environment, the automobile 1 needs to be able to be controlled to travel toward a predetermined destination or the like.

[0022] Fig. 2 is an explanatory diagram of a state in which the automobile 1 in Fig. 1 is traveling forward toward an intersection on a road 10 that intersects with another road 16. A signboard 14 is provided on the shoulder of the road 10 on which the automobile 1 is traveling. In this case, the automobile 1 may be controlled to travel forward toward the intersection, as shown by the solid line in FIG. 2, stop temporarily at the stop line 11, and then turn left onto another intersecting road 16 while checking to the left and right.

[0023] If the actual position of automobile 1 is at a slightly advanced position (2') as shown by the dashed line in Figure 2, the position where automobile 1 turns left at the intersection under that control may be as shown by the dashed arrow. It is important in the driving control of automobile 1 to prevent such control from occurring.

[0024] FIG. 3 is an explanatory diagram of measurement of the relative position of a surrounding object to another moving body by a monocular imaging device 41 provided facing forward at the front of the automobile 1. In FIG.

[0025] FIG. 3A is an explanatory diagram of an image capturing state by monocular imaging device 41 provided facing forward at the front of automobile 1. FIG. As shown in Fig. 3A, monocular imaging device 41 provided on automobile 1 captures an image of the surroundings of automobile 1 at a predetermined angle of view. In Fig. 3A, a pedestrian is standing on the surface of road 10. The pedestrian is within the range of angles of view 51 to 52 of monocular imaging device 41, and is therefore imaged by monocular imaging device 41.

[0026] FIG. 3B is an explanatory diagram of an image 20 captured by the monocular imaging device 41 in the imaging state of FIG. 3A. 3B includes an image of a pedestrian in a forward position to the right of the captured image 20 of the monocular imaging device 41. The image of the pedestrian is surrounded by a recognition frame 21 that is generated by analyzing the captured image 20. 3B, a road surface model 22 using concentric circles with the installation position of the monocular imaging device 41 as the imaging center can be applied to the captured image 20 of the monocular imaging device 41. Each circle connects positions on a flat road surface that are equidistant from the imaging center, which is the installation position of the monocular imaging device 41.

[0027] FIG. 3C is an explanatory diagram for determining the relative position of a pedestrian included in the captured image 20 of FIG. 3B. 3C is divided into a plurality of pixels 23, the number of which corresponds to the resolution of the monocular imaging device 41. The number of pixels 23 is basically determined by the imaging element of the monocular imaging device 41. 3B, a road surface model 22 made of concentric circles is superimposed on the captured image 20 of Fig. 3C. In this manner, a unique relative position can be defined for each pixel 23 constituting the captured image 20 of the monocular imaging device 41 according to the road surface model 22. Therefore, by using the recognition frame 21 in FIG. 3B to identify a specific pixel 24 including the center of the bottom end of the recognition frame 21, it is possible to measure the position of the captured pedestrian based on the relative distance and relative direction. The accuracy of the relative positions of surrounding objects, such as other moving bodies and objects around the automobile 1, including pedestrians, depends on the resolution of the monocular imaging device 41. To obtain the positions of surrounding objects with high accuracy, the monocular imaging device 41 needs to have high resolution and high performance.

[0028] However, as shown in Fig. 3A, it is ideal that the road surface model 22 is flush with the road surface of the actual road 10, but in reality, as shown by the example of multiple dashed lines in Fig. 3A, the road surface model 22 may deviate above or below the road surface of the actual road 10, or may be inclined relative to the actual road surface. The road surface model 22 does not necessarily match the actual road surface perfectly. These deviations may result in control errors in the driving control of the automobile 1. Furthermore, the monocular imaging device 41 is not necessarily installed in the automobile 1 at a designed position corresponding to such a road surface model 22 without error. In a traveling automobile 1, the attitude and position of the monocular imaging device 41 may change over time. A decrease in installation accuracy may result in a control error in the traveling control of the automobile 1. When control errors caused by these various factors are compounded, the position of the controlled object, for example, the automobile 1, may deviate from the actual position in the actual environment. The final control error in the position determined by the positioning of the automobile 1 may deviate significantly from the actual environment due to a number of factors, including not only differences between the road surface model 22 used with the monocular imaging device 41 and the surface of the road 10 on which the automobile 1 is actually traveling, but also installation errors of the monocular imaging device 41 to the automobile 1 and changes therein over time, and the performance of the monocular imaging device 41 itself, such as its resolution. If no measures are taken against these control errors, a large positional deviation may occur, as shown in FIG. 2 between the position of the automobile 1 indicated by the solid line and the position of the automobile 1 indicated by the dashed line. In order to suppress control errors as much as possible in processing the road surface model 22 illustrated in Fig. 3, it is necessary to use a high-precision, high-performance monocular imaging device 41, or to position the monocular imaging device 41 with high precision relative to the automobile 1 so as to match the road surface model 22 and to attach it firmly so as not to change over time. If there is still a possibility that control errors cannot be sufficiently suppressed even with these measures, it becomes necessary to use a more high-precision, high-performance imaging device such as a compound-eye imaging device, rather than the monocular imaging device 41, as the imaging device to be provided on the automobile 1. In driving control of the automobile 1, it is basically required to use high-precision, high-performance devices not only for imaging devices but also for other vehicle sensors. In this way, in driving control of the automobile 1, it is necessary to suppress the increasing sophistication of the vehicle's own sensors, such as imaging devices, while ensuring control performance that can effectively suppress position deviations.

[0029] FIG. 4 is an explanatory diagram of a control system 30 provided in the automobile 1 of FIG. 1 and functioning as a cruise control device as a whole. The control system 30 of the automobile 1 in FIG. 4 includes an exterior imaging device 31, a driving control unit 32, a GNSS receiver 33, high-precision map data 34, a sensor device 35, an exterior communication device 36, and a vehicle network 37 to which these are connected.

[0030] The vehicle network 37 may be, for example, a Controller Area Network (CAN), a Local Interconnect Network (LIN), a Local Area Network (LAN), FlexRay, or a Clock Extension Peripheral Interface (CXPI) that is used in the automobile 1. The vehicle network 37 may be configured, for example, with a plurality of bus cables connected to a plurality of control devices, and a central gateway (CGW) to which the plurality of bus cables are bus-connected. A part of the vehicle network 37 may be made wireless, for example, by using a wireless communication router. The plurality of control devices connected to the vehicle network 37 can send and receive messages to each other through the vehicle network 37. This allows the plurality of control devices to cooperate to control the running of the automobile 1, etc.

[0031] The high-precision map data 34 may be map data including information on roads 10 on which the automobile 1 can travel, and used to control travel of the automobile 1. The high-precision map data 34 may include, as information on each road 10, information on links and nodes indicating the roads 10 on which the automobile 1 travels, as well as information on intermediate locations such as buildings 13. The information on links and nodes indicating the roads 10 may include not only position information on the links and nodes based on latitude, longitude, and altitude in a predetermined positioning system, but also information useful as a road surface model, such as the slope of the road 10. Such high-precision map data 34 is three-dimensional map data. Further, in addition to the information on the road 10, the high-precision map data 34 may include information on specific objects provided at specific positions, such as the stop line 11, traffic lights 12, and signboards 14 provided around the road 10. Structures such as buildings 13 may also be included in the high-precision map data 34 as specific objects. The stop line 11 and traffic lights 12 are provided at intersections of the road 10 on which the automobile 1 travels. The signboards 14 include, for example, road guide boards and traffic sign boards. The signboards 14 are provided on the shoulders or lanes of the road 10 on which the automobile 1 travels. The information on the specific objects may include position information of latitude, longitude, and altitude in a predetermined positioning system, or position information that can be converted thereto. For the signboards 14 provided on the traffic lights 12 and lanes, position information of the installation height from the road surface may be included.

[0032] The GNSS receiver 33 receives radio waves from multiple GNSS satellites 101 and generates position information of the GNSS receiver 33. For example, the position information of the GNSS receiver 33 may be latitude, longitude, and altitude that is compatible with the positioning system of the high-precision map data 34. The GNSS receiver 33 outputs the generated position information to the vehicle network 37 as current position information of the automobile 1 in which the GNSS receiver 33 is installed.

[0033] The sensor device 35 is connected to various sensors provided in the automobile 1. In FIG. 4, an acceleration sensor 38 is connected to the sensor device 35. The acceleration sensor 38 detects acceleration acting on the automobile 1. The acceleration acting on the automobile 1 includes a pitch direction acceleration acting in the up-down direction on the automobile 1, and a yaw direction acceleration acting in the left-right direction. The sensor device 35 may obtain the pitch direction of the automobile 1 by double integrating the acceleration in the pitch direction. The sensor device 35 may obtain the yaw direction of the automobile 1 by double integrating the acceleration in the yaw direction. The sensor device 35 outputs these detection values ​​based on the detection of the sensors to the vehicle network 37.

[0034] The external vehicle communication device 36 establishes a communication path with a base station (not shown) installed around the road 10 on which the automobile 1 travels by wireless communication. The base station may be a base station of a mobile communication network or a base station of an advanced information service system (ITS) for the automobile 1 and the like. The external vehicle communication device 36 may transmit and receive various data to and from the server device 39 through the base station. The server device 39 may be connected to the base station via a communication network such as the Internet, for example, or may be provided in the base station itself. The server device 39 may be provided in a distributed manner in a plurality of base stations. The external vehicle communication device 36 transmits data acquired from the vehicle network 37 to the server device 39. The external vehicle communication device 36 outputs data received from the server device 39 to the vehicle network 37. The exterior-vehicle communication device 36 may acquire map data that is missing from the high-precision map data 34 provided in the automobile 1 from the server device 39. Examples of map data that is missing from the high-precision map data 34 provided in the automobile 1 include information on areas outside the target area of ​​the map data and update information for the map data.

[0035] The travel control unit 32 is connected to, for example, a steering device, a drive device, a braking device, and the like (not shown) of the automobile 1. The drive device may be equipped with an engine that generates driving force by burning fuel such as gasoline, diesel, or hydrogen gas, or may be equipped with a motor that generates driving force by storing electric power in a battery or generating electricity, or may be a hybrid type that combines these. The drive device uses the generated driving force to rotate and drive the wheels 5 of the automobile 1, thereby accelerating the automobile 1. The steering device changes the direction of the wheels 5 of the automobile 1 to change the traveling direction of the automobile 1. The braking device applies brakes to the rotation of the wheels 5 of the automobile 1 to slow down or stop the automobile 1. When data related to driving control, such as a course, is input from the vehicle network 37, the driving control unit 32 controls the operation of the steering device, the drive device, and the braking device accordingly, and controls the driving of the automobile 1. For example, the driving control unit 32 controls the driving of the automobile 1 based on the positioning of the vehicle's own position by the GNSS receiver 33 of the automobile 1. The driving control unit 32 controls the driving of the automobile 1 using high-precision map data 34 of the positioning system including position information of a specific position. The driving control unit 32 may control the driving of the vehicle 1 according to the operation of the occupant of the vehicle 1, may control the driving of the vehicle 1 by supporting the operation of the occupant, or may control the driving of the vehicle 1 by controlling autonomous automatic driving. The driving control unit 32 may change these driving support levels according to the driving situation.

[0036] The vehicle exterior imaging device 31 includes a monocular imaging device 41, a memory 42, a CPU 43, a timer 44, an input / output unit 45, and a system bus 46 to which these are connected.

[0037] The monocular imaging device 41 has an optical system such as a lens as well as a CMOS sensor or a CCD sensor. The light collected by the optical system is converted into an electrical signal by the CMOS sensor or the CCD sensor. The optical system may be a wide-angle optical system. As a result, the monocular imaging device 41 generates a captured image 20 at a predetermined angle of view. As shown in Figs. 1 and 2, the monocular imaging device 41 is provided facing forward at the front end center part of the roof of the passenger compartment 3 of the automobile 1. This allows the monocular imaging device 41 to capture an image of the front side, which is the traveling direction of the traveling automobile 1. When the captured image 20 of the monocular imaging device 41 is used to detect peripheral objects such as other moving bodies in front of the automobile 1 by correlating it with the road surface model 22 as shown in Fig. 3, the mounting accuracy of the monocular imaging device 41 on the automobile 1, the orientation of the monocular imaging device 41, etc. is basically required to be high reliability so that it does not change even when the automobile 1 is traveling. The position accuracy of the peripheral object that can be obtained based on the captured image 20 depends on the mounting accuracy of the monocular imaging device 41 on the automobile 1, the orientation of the monocular imaging device 41, etc.

[0038] The timer 44 measures time, clock time, etc. The time of the timer 44 may be calibrated by the time generated by the GNSS receiver 33.

[0039] The input / output unit 45 is connected to the vehicle network 37. The input / output unit 45 inputs and outputs data to and from other devices connected to the vehicle network 37.

[0040] The memory 42 may be composed of, for example, a HDD, SD, RAM, ROM, etc. The memory 42 records, for example, the programs executed by the CPU 43, data used in executing the programs, data generated during the execution of the programs, and data to be saved.

[0041] The CPU 43 reads and executes the program recorded in the memory 42. In this way, the CPU 43 functions as a control unit of the outside-of-vehicle image capture device 31. The CPU 43 as a control unit of the outside-of-vehicle image capturing device 31 controls the overall operation of the outside-of-vehicle image capturing device 31 . The CPU 43, for example, causes the monocular imaging device 41 to capture an image of the surroundings in front of the automobile 1, and analyzes the captured image 20 to detect peripheral objects present around the automobile 1. In this manner, the monocular imaging device 41 is provided in the drivable automobile 1 for measuring the relative positions of peripheral objects such as other vehicles. When a specific object is included among the surrounding objects detected based on the captured image 20, the CPU 43 generates correction information for correcting the relative positions of the vehicle position and the surrounding objects based on the timing of passing near the specific object, and records the correction information in the memory 42. When the correction information is recorded in the memory 42, the CPU 43 causes the driving control unit 32 to execute driving control of the automobile 1 using the host vehicle position and the relative positions of the peripheral objects corrected using the correction information.

[0042] FIG. 5 is a flowchart of driving control for determining the vehicle position of the automobile 1 by the driving control unit 32 of FIG. The driving control unit 32 in FIG. 4 repeatedly executes the driving control in FIG. 5 by, for example, a CPU (not shown) when the automobile 1 is driving.

[0043] In step ST1, the driving control unit 32 judges whether or not to start driving control of the automobile 1. For example, when a passenger such as a driver gets into the automobile 1 and operates an ignition switch (not shown) or sets a destination or route, the driving control unit 32 judges to start driving control of the automobile 1, and proceeds to processing in step ST2. If driving control of the automobile 1 is not to be started, the driving control unit 32 repeats this processing.

[0044] In step ST2, the driving control unit 32 acquires the vehicle position of the automobile 1 by positioning. The GNSS receiver 33 receives radio waves from the GNSS satellites 101 and generates the current position of the automobile 1 in which the GNSS receiver 33 is installed. The driving control unit 32 may acquire the current position acquired by the GNSS receiver 33 as the vehicle position of the automobile 1 that has been positioned.

[0045] In step ST3, the driving control unit 32 corrects the measured position of the automobile 1 using the correction information.

[0046] In step ST4, the driving control unit 32 acquires a route from the current position using the high-precision map data 34. The route may be used for short-term driving control. The driving control unit 32 may continue driving by repeatedly executing the driving control of FIG. 5 at short intervals. The driving control unit 32 may generate and acquire a route from the current position by itself using the high-precision map data 34. The driving control unit 32 may acquire the route from the current position from the server device 39 using the external vehicle communication device 36.

[0047] In step ST5, the driving control unit 32 checks whether there is an obstacle to the acquired route. The obstacle here is, for example, a peripheral object such as another moving body around the automobile 1. The driving environment of the automobile 1 changes from moment to moment. For example, if a peripheral object such as another moving body is on the acquired route and is estimated to be advancing toward the acquired route as if to invade the acquired route, the driving control unit 32 determines that there is an obstacle to the acquired route and proceeds to step ST7. If there is no obstacle to the acquired route, the driving control unit 32 proceeds to step ST6.

[0048] In step ST6, the driving control unit 32 executes driving control on the route that is determined to be free of obstacles. The driving control unit 32 controls the operation of the steering device, the drive device, and the braking device so that the automobile 1 travels along the acquired route. After that, the driving control unit 32 advances the process to step ST8.

[0049] In step ST7, the driving control unit 32 executes driving control on a course that avoids the obstacle in the course. The driving control unit 32 controls the operation of the steering device, the drive device, and the braking device so that the automobile 1 travels while avoiding the obstacle. The driving control unit 32 executes driving control, for example, so that the automobile 1 stops just before the obstacle. After that, the driving control unit 32 advances the process to step ST8.

[0050] In step ST8, the driving control unit 32 judges whether or not to end the driving control of the automobile 1. For example, when the automobile 1 stops and a passenger such as the driver gets off, the driving control unit 32 judges to end the driving control of the automobile 1 and ends this control. If the driving control of the automobile 1 is not to be ended, the driving control unit 32 returns the process to step ST2. The driving control unit 32 repeats the processes from step ST2 to step ST8 until it judges to end the driving control of the automobile 1.

[0051] FIG. 6 is a flowchart of outside-vehicle image capture control by the CPU 43 of the outside-vehicle image capture device 31 in FIG. 4 for capturing an image of the surroundings in front of the automobile 1 and locating the positions of surrounding objects. The CPU 43 of the vehicle outside image capturing device 31 repeatedly executes the vehicle outside image capturing control of FIG. 6 while the automobile 1 is traveling.

[0052] In step ST11, the CPU 43 judges whether or not to start driving control of the automobile 1. For example, similar to the driving control unit 32, when a passenger such as a driver gets into the automobile 1 and operates an ignition switch (not shown) or sets a destination or route, the CPU 43 judges to start driving control of the automobile 1 and proceeds to step ST12. If driving control of the automobile 1 is not to be started, the CPU 43 repeats this process.

[0053] In step ST12, the CPU 43 judges whether it is a new timing to capture an image of the periphery of the automobile 1. The CPU 43 may judge it as a new timing to capture an image for each imaging period measured by the timer 44, for example. If it is not a new timing to capture an image, the CPU 43 repeats this process. If it is a new timing to capture an image, the CPU 43 advances the process to step ST13.

[0054] In step ST13, the CPU 43 causes the monocular imaging device 41 to capture a new image. The monocular imaging device 41 captures an image of the surroundings in front of the automobile 1, which is the traveling direction of the automobile 1. The CPU 43 obtains the captured image 20 from the monocular imaging device 41.

[0055] In step ST14, the CPU 43 acquires, for example, from the sensor device 35, attitude information such as the yaw angle and pitch angle at the image capture timing, as the traveling state of the automobile 1 at the image capture timing.

[0056] In step ST15, the CPU 43 analyzes the current driving environment of the automobile 1. The CPU 43 basically analyzes the newly acquired captured image 20 to perform a recognition process of new peripheral objects such as other moving bodies around the automobile 1, the movement direction of peripheral objects already recognized in the past captured image 20, the road surface condition of the lanes of the road 10 on which the automobile 1 is traveling, and the like. The peripheral objects here may include peripheral objects of the same type as the specific object included in the captured image 20. The peripheral objects recognized here may be used by the driving control unit 32 described above to determine a path obstacle in step ST5.

[0057] In step ST16, the CPU 43 determines whether or not the analysis result of the traveling environment in step ST15 includes a peripheral object. If a peripheral object is included, the CPU 43 advances the process to step ST17. If a peripheral object is not included, the CPU 43 advances the process to step ST19.

[0058] In step ST17, the CPU 43 measures the relative position of each peripheral object included in the analysis result of the driving environment. For example, the CPU 43 uses a road surface model for the captured image 20 of the monocular imaging device 41 to identify pixels 23 capturing images of each peripheral object in the captured image 20, and obtains the relative distance and direction of the identified pixels 23 in the road surface model based on the position of the monocular imaging device 41.

[0059] In step ST18, the CPU 43 corrects the relative distance and direction acquired for each peripheral object based on the captured image 20 in step ST17 using the correction information.

[0060] In step ST19, the CPU 43 stores various information acquired by the above-mentioned processing in the memory 42 so as to be readable later by the CPU 43. The CPU 43 may also output the information on the above-mentioned peripheral objects, including position information based on the corrected relative position and direction, from the input / output unit 45 to the traveling control unit 32.

[0061] In step ST20, the CPU 43 judges whether or not to end the driving control of the automobile 1. For example, similar to the driving control unit 32, when the automobile 1 stops and passengers such as the driver get off, the CPU 43 judges to end the driving control of the automobile 1 and ends this control. If the driving control of the automobile 1 is not to be ended, the CPU 43 returns the process to step ST12. The CPU 43 repeats the processes from step ST12 to step ST20 until it judges to end the driving control of the automobile 1.

[0062] FIG. 7 is a flowchart showing a control process for generating position correction information for positioning of the automobile 1 by the CPU 43 of the outside-vehicle image capturing device 31 in FIG. When the automobile 1 is traveling, the CPU 43 of the vehicle outside image capture device 31 repeatedly executes the correction information generation control of Fig. 7. The CPU 43 may execute the correction information generation control of Fig. 7 following the vehicle outside image capture control of Fig. 6.

[0063] In step ST31, the CPU 43 judges whether or not to start driving control of the automobile 1. For example, similar to the driving control unit 32, when a passenger such as a driver gets into the automobile 1 and operates an ignition switch (not shown) or sets a destination or route, the CPU 43 judges to start driving control of the automobile 1 and proceeds to step ST32. If driving control of the automobile 1 is not to be started, the CPU 43 repeats this process.

[0064] In step ST32, the CPU 43 determines whether or not a specific object at a specific position is detected by the analysis process of the outside-of-vehicle image capture control for the captured image 20. The CPU 43 may determine whether or not a specific object at a specific position is detected by comparing with the high-precision map data 34. If a specific object at a specific position is not detected, the CPU 43 repeats this process and waits for the specific object to be detected. When a specific object is detected, the CPU 43 advances the process to step ST33 to start generating correction information based on the specific object.

[0065] In step ST33, the CPU 43 judges whether the detected specific object has disappeared and is no longer detected due to the analysis process in the outside-of-vehicle image capture control for the captured image 20. The monocular image capture device 41 captures an image in front of the traveling automobile 1. Therefore, as the automobile 1 moves forward, the detected specific object is no longer captured in the captured image 20. As a result, the specific object disappears in the analysis process in the outside-of-vehicle image capture control. The CPU 43 waits for this disappearance timing to occur. Then, when the detected specific object disappears, the CPU 43 advances the process to step ST34.

[0066] In step ST34, the CPU 43 acquires the timing when the traveling automobile 1 passes the detected specific object. At this time, the CPU 43 may acquire the timing when the detected specific object disappears as the passing timing. For example, the CPU 43 may acquire the imaging timing of the captured image 20 when the detected specific object is no longer included for the first time as the passing timing. Alternatively, the CPU 43 may estimate the movement of the specific object from multiple captured images 20 including the specific object, calculate the timing when the estimated position of the specific object becomes outside the captured image 20, and acquire the calculated disappearance timing as the passing timing. The calculated disappearance timing may be between multiple consecutive imaging timings. Thus, the CPU 43, as a timing acquisition unit, can acquire the timing when the traveling automobile 1 passes through a specific position whose position is known. The CPU 43 can acquire the timing when the traveling automobile 1 passes through the specific position based on the captured image 20 captured by the monocular imaging device 41 of a specific object at the specific position while the automobile 1 is traveling.

[0067] In step ST35, the CPU 43 acquires from the memory 42 information on the passing timing, such as attitude information, such as the yaw angle and pitch angle, of the automobile 1 at the passing timing, and information on the peripheral imaging angle of view and position of the monocular imaging device 41. The memory 42 may accumulate and record the yaw angle and pitch angle of the automobile 1 acquired from the sensor device 35. The CPU 43 may select and acquire the passing timing from among the multiple sets of yaw angles and pitch angles accumulated in the memory 42. Information such as the peripheral imaging angle of view of the monocular imaging device 41 and the installation height position on the vehicle body 2 may be recorded in advance in the memory 42 when the monocular imaging device 41 is newly installed or replaced and attached to the automobile 1. When the vehicle body 2 is not reflected in the captured image 20, the CPU 43 may acquire the angle of view of the monocular imaging device 41 itself instead of the peripheral imaging angle of view.

[0068] In step ST36, the CPU 43 acquires the position of the monocular imaging device 41 measured by the driving control unit 32 at the passing timing as the position of the vehicle itself. The driving control unit 32 measures the vehicle position in step ST2 every time the driving control of Fig. 4 is executed. The driving control unit 32 may accumulate and store the vehicle position determined by this positioning in a memory (not shown). The memory 42 of the outside-vehicle image capture device 31 may acquire the vehicle position determined by positioning from the driving control unit 32 and accumulate and record the vehicle position. The CPU 43 may select and acquire the vehicle position at the passing timing from the multiple vehicle positions determined by positioning accumulated and recorded in the memory 42.

[0069] In step ST37, the CPU 43 acquires, from the high accuracy map data 34, the position of the specific object at the detected specific position in a predetermined positioning system. This allows the CPU 43, as a position acquisition unit, to acquire a position not based on positioning by the GNSS receiver 33 of the automobile 1 as position information of a specific position for acquiring the passing timing.

[0070] In step ST38, the CPU 43 generates correction information for converting the vehicle position measured by the driving control unit 32 into the vehicle position on the high-precision map data 34. The CPU 43 uses the various acquired information described above to calculate the vehicle position on the high-precision map data 34 when passing through a specific object at the passing timing, and generates correction information as the difference between this position and the vehicle position measured by the driving control unit 32. The CPU 43 first calculates the positional relationship between the specific object and the automobile 1 at the time of passing, based on the relative positional relationship between the specific object and the automobile 1 at the time of passing. This positional relationship may be composed of, for example, the distance from the specific object to the automobile 1 and the direction from the specific object to the automobile 1. Next, the CPU 43 calculates the position of the automobile 1 at the passing timing by using the calculated positional relationship with reference to the position of the specific object in the positioning system of the high precision map data 34. The position of the automobile 1 calculated here is the position in the positioning system of the high precision map data 34. Next, the CPU 43 calculates the difference between the measured vehicle position and the vehicle position in the positioning system of the high precision map data 34. This difference becomes position correction information for converting the measured position into the position in the positioning system of the high precision map data 34. Thus, the CPU 43, as a generating unit, can generate the position of the automobile 1 based on the positional relationship between the specific object and the vehicle (monocular imaging device 41) at the time of passing, using the position of the position information of the specific object at the specific position as a reference. Furthermore, the CPU 43 can generate correction information for the vehicle position based on the position of the specific object at the specific position in the positioning system for measuring the vehicle position of the automobile 1 at the time of passing. This correction information can be used to correct the position of the vehicle position measured by the GNSS receiver 33 of the automobile 1 to the position of the positioning system.

[0071] In step ST39, the CPU 43 stores the generated position correction information in the memory 42. The CPU 43 may output the generated position correction information to the traveling control unit 32.

[0072] In step ST40, the CPU 43 judges whether or not to end the driving control of the automobile 1. For example, similar to the driving control unit 32, when the automobile 1 stops and the driver or other passengers get off, the CPU 43 judges to end the driving control of the automobile 1 and ends this control. If the driving control of the automobile 1 is not to be ended, the CPU 43 returns the process to step ST32. The CPU 43 repeats the processes from step ST32 to step ST40 until it judges to end the driving control of the automobile 1.

[0073] When the position correction information is generated in this manner, the cruise control unit 32 can convert the host vehicle position based on the positioning into the host vehicle position in the positioning system, using the position correction information in step ST3 of the cruise control in FIG. In addition, the CPU 43 of the exterior imaging device 31 can convert the relative positions of the surrounding objects detected based on the captured image 20 of the monocular imaging device 41 into relative positions in the positioning system using the position correction information in step ST18 of the exterior imaging control of Figure 6.

[0074] FIG. 8 is an explanatory diagram of an image 20 captured by the monocular imaging device 41 of FIG. 4 capturing an image of a specific object at a specific position.

[0075] FIG. 8A is an explanatory diagram of an image 20 captured by the monocular imaging device 41 of FIG. 4 capturing an image of a stop line 11 as a specific object located at a specific position. 8A, a stop line search area 61 in which the CPU 43 of the vehicle exterior image capture device 31 detects a specific object is superimposed. The stop line search area 61 is set in the lower center of the captured image 20 so as to be in contact with the lower edge of the captured image 20. The CPU 43 of the vehicle exterior image capture device 31 may detect a road surface object, such as the stop line 11, drawn on the road surface of the road (lane) 10 on which the automobile 1 travels as a specific object in the entire captured image 20, or may detect the road surface object as a specific object only in the stop line search area 61. When the automobile 1 travels forward, the stop line 11 appears in the center of the captured image 20, moves downward, and disappears from the lower edge of the captured image 20 to the lower side of the captured image 20. In addition to the stop line 11, road surface objects drawn on the road surface include road 10 signs on the road surface. The high-precision map data 34 may include information on the positions at which these road surface objects are drawn.

[0076] FIG. 8B is an explanatory diagram of the captured image 20 captured by the monocular imaging device 41 of FIG. 4, capturing an image of a traffic light 12 as a specific object located at a specific position. 8B, a traffic light search area 62 in which the CPU 43 of the vehicle exterior image capture device 31 detects a specific object is superimposed. The traffic light search area 62 is set in the upper center of the captured image 20 so as to be in contact with the upper edge of the captured image 20. The CPU 43 of the vehicle exterior image capture device 31 may detect a three-dimensional object, such as a traffic light 12, provided on the road (lane) 10 on which the automobile 1 travels as a specific object in the entire captured image 20, or may detect a three-dimensional object only in the traffic light search area 62 as a specific object. When the automobile 1 travels forward, the traffic light 12 appears in the center of the captured image 20, moves upward, and disappears from the upper edge of the captured image 20 to the upper side of the captured image 20. In addition to the traffic light 12, the three-dimensional objects provided on the road (lane) 10 include a signboard 14 for road guidance, road 10 sign, etc. The high-precision map data 34 may include information on the positions at which these three-dimensional objects are provided. The position information may include information on the installation height from the road surface.

[0077] FIG. 8C is an explanatory diagram of the captured image 20 captured by the monocular imaging device 41 of FIG. 4 while capturing an image of the signboard 14 as a specific object located at a specific position. 8C, a signboard search area 63 in which the CPU 43 of the vehicle exterior image capture device 31 detects a specific object is superimposed. The signboard search area 63 is set in the left part of the captured image 20 so as to be in contact with the side edge of the road shoulder side of the road 10 in the captured image 20. The CPU 43 of the vehicle exterior image capture device 31 may detect a three-dimensional object such as a signboard 14 provided on the road shoulder of the road (lane) 10 on which the automobile 1 travels as a specific object in the entire captured image 20, or may detect a three-dimensional object only in the signboard search area 63 as a specific object. When the automobile 1 travels forward, the signboard 14 on the road shoulder appears in the center part of the captured image 20, moves laterally to the left, and disappears from the left side edge of the captured image 20 to the left side of the captured image 20. The high-precision map data 34 may include information on the positions where these three-dimensional objects are provided.

[0078] FIG. 9 is an explanatory diagram of the positional relationship between the stop line 11 and the automobile 1 at the time when the stop line 11 disappears from the captured image 20. As shown in FIG. The positional relationship between the stop line 11 and the automobile 1 in FIG. 9 is at the time when the stop line 11 disappears, which is after the time when the captured image 20 in FIG. 8A is captured.

[0079] The CPU 43 of the outside-vehicle image capture device 31 generates position correction information based on the positional relationship between the stop line 11, whose position is known in the positioning system, and the automobile 1, which is performing positioning by its own vehicle, using the disappearance timing in Figure 9 as the passing timing. 9, the stop line 11 is drawn on the surface of the road 10, and its position in the positioning system is included in the high-precision map data 34. The CPU 43 of the outside-vehicle image capture device 31 serves as a position acquisition unit and acquires position information of a specific object such as the stop line 11 drawn on the surface of the road (lane) 10 on which the automobile 1 is traveling. Although the stop line 11 disappearing from the captured image 20 is within the angle of view of the monocular imaging device 41, it is hidden by the vehicle body 2 and is therefore below the peripheral imaging angle of view in the vertical direction. As a result, it disappears from the captured image 20 of the monocular imaging device 41. The CPU 43 of the outside-vehicle imaging device 31, as a timing acquisition unit, acquires, as the passing timing, the timing at which a specific object such as the stop line 11 drawn on the surface of the road (lane) 10 on which the automobile 1 is traveling disappears downward from the captured image 20 captured by the monocular imaging device 41 provided on the automobile 1 of the surroundings in front of the automobile 1.

[0080] The CPU 43 of the outside-vehicle image capture device 31 can calculate the distance D (Stop Line) from the monocular imaging device 41 provided on the automobile 1 to the stop line 11 by substituting the depression angle θu of the peripheral imaging field angle of the monocular imaging device 41 from the horizontal direction parallel to the surface of the road 10, the height Lc of the monocular imaging device 41 from the surface of the road 10 (the installation height on the automobile 1), and the pitch angle θp of the automobile 1 with respect to the road surface into the following equation 1. In each of the following equations, ABS(X) means the absolute value of X. TAN(θ) means tangent calculation with respect to θ. " / " means division. In this way, the CPU 43 of the exterior imaging device 31 can use information on the peripheral imaging angle of view of the monocular imaging device 41 provided on the automobile 1, information on the position of the monocular imaging device 41 (installation height on the automobile 1, amount of snow accumulation such as snow that changes the height of the road surface), and information on the attitude of the automobile 1 (pitch angle in the vertical direction) to calculate the relative position (distance, direction, spacing) between the automobile 1 (monocular imaging device 41) and the stop line 11 as a specific object at the time of passing. The CPU 43 of the vehicle outside image capture device 31 can then calculate the position of the automobile 1 according to the positioning system, using the calculated relative position based on the position of the stop line 11 as a specific object according to the positioning system. Furthermore, the CPU 43 of the outside-vehicle image capture device 31 can calculate the difference between the position of the automobile 1 determined by the positioning system and the vehicle's own position determined by the GNSS receiver 33 of the vehicle as correction information for the positioning of the vehicle. This enables the CPU 43 of the outside-vehicle image capture device 31 to apply the correction information to the vehicle position measured by the GNSS receiver 33 of the vehicle, for example, and obtain the vehicle position in the positioning system with a high degree of accuracy. In addition, the CPU 43 of the outside-vehicle imaging device 31 applies correction information to the positions of surrounding objects detected based on the captured image 20 of the monocular imaging device 41, thereby making it possible to obtain the positions of the surrounding objects in the positioning system with a high degree of accuracy.

[0081] D(Stop Line)=Lc / TAN(ABS(θu)+ABS(θp))...Equation 1

[0082] FIG. 10 is an explanatory diagram of the positional relationship between the traffic light 12 and the automobile 1 at the time when the traffic light 12 disappears from the captured image 20. As shown in FIG. The positional relationship between the traffic light 12 and the automobile 1 in FIG. 10 is at the time of disappearance, which is after the time when the captured image 20 in FIG. 8B is captured.

[0083] The CPU 43 of the outside-vehicle image capture device 31 generates position correction information based on the positional relationship between the traffic light 12, whose position in the positioning system is known, and the automobile 1, which is performing positioning by its own vehicle, using the disappearance timing in Figure 10 as the passing timing. In FIG. Traffic light 12 is provided on the road 10, and its position in the positioning system is included in the high-precision map data 34. The CPU 43 of the outside-vehicle image capture device 31 serves as a position acquisition unit and acquires position information of a specific object such as a traffic light 12 provided on the road (lane) 10 on which the automobile 1 is traveling. The traffic light 12 that disappears from the captured image 20 is outside the range of the angle of view of the monocular imaging device 41, and is therefore above the peripheral imaging angle of view in the vertical direction. As a result, it disappears from the captured image 20 of the monocular imaging device 41. The CPU 43 of the outside-vehicle imaging device 31, as a timing acquisition unit, acquires, as the passing timing, the timing at which a specific object such as the traffic light 12 provided on the road (lane) 10 on which the automobile 1 is traveling disappears upward from the captured image 20 captured by the monocular imaging device 41 provided on the automobile 1 of the surroundings in front of the automobile 1.

[0084] The CPU 43 of the exterior imaging device 31 can calculate the distance D (Signal) from the monocular imaging device 41 installed on the automobile 1 to the traffic light 12 by substituting the elevation angle θo of the peripheral imaging field angle of the monocular imaging device 41 from the horizontal direction parallel to the surface of the road 10, the height of the monocular imaging device 41 from the surface of the road 10 (installation height on the automobile 1) Lc, the pitch angle θp of the automobile 1 relative to the road surface, the amount of change Lr in the road surface height of the road 10 due to snow accumulation, etc., and the installation height Mo of the traffic light 12 from the road surface into the following equation 2. In this way, the CPU 43 of the exterior imaging device 31 can use information on the peripheral imaging angle of view of the monocular imaging device 41 provided on the automobile 1, information on the position of the monocular imaging device 41 (installation height on the automobile 1, amount of snow accumulation such as snow that changes the height of the road surface), and information on the attitude of the automobile 1 (pitch angle in the vertical direction) to calculate the relative position (distance, direction, interval) between the automobile 1 (monocular imaging device 41) and the traffic light 12 as a specific object at the time of passing. The CPU 43 of the outside-vehicle image capture device 31 can then calculate the position of the automobile 1 according to the positioning system, using the calculated relative position based on the position of the traffic light 12 as a specific object according to the positioning system. Furthermore, the CPU 43 of the outside-vehicle image capture device 31 can calculate the difference between the position of the automobile 1 determined by the positioning system and the vehicle's own position determined by the GNSS receiver 33 of the vehicle as correction information for the positioning of the vehicle. This enables the CPU 43 of the outside-vehicle image capture device 31 to apply the correction information to the vehicle position measured by the GNSS receiver 33 of the vehicle, for example, and obtain the vehicle position in the positioning system with a high degree of accuracy. In addition, the CPU 43 of the outside-vehicle imaging device 31 applies correction information to the positions of surrounding objects detected based on the captured image 20 of the monocular imaging device 41, thereby making it possible to obtain the positions of the surrounding objects in the positioning system with a high degree of accuracy.

[0085] D(Signal)=(Mo-Lc+Lr) / TAN(ABS(θo)+ABS(θp))...Equation 2

[0086] FIG. 11 is an explanatory diagram of the positional relationship between the roadside signboard 14 and the automobile 1 at the time when the roadside signboard 14 disappears from the captured image 20. As shown in FIG. The positional relationship between the roadside signboard 14 and the automobile 1 in FIG. 11 is at the time when the signboard 14 disappears, which is after the time when the captured image 20 in FIG. 8C is captured.

[0087] The CPU 43 of the outside-vehicle image capturing device 31 generates position correction information based on the positional relationship between the roadside sign 14, whose position in the positioning system is known, and the automobile 1, which is performing positioning, using the disappearance timing in Figure 11 as the passing timing. 11, the roadside signboard 14 is erected on the roadside to be next to the road 10, and its position in the positioning system is included in the high-precision map data 34. The CPU 43 of the outside-vehicle image capture device 31 serves as a position acquisition unit and acquires position information of a specific object such as the signboard 14 provided on the roadside of the road (lane) 10 on which the automobile 1 is traveling. The roadside signboard 14 that disappears from the captured image 20 is outside the range of the angle of view of the monocular imaging device 41, and therefore is laterally outside the peripheral imaging angle of view in the vehicle width direction. As a result, it disappears from the captured image 20 of the monocular imaging device 41. The CPU 43 of the outside-vehicle imaging device 31, as a timing acquisition unit, acquires, as the passing timing, the timing at which a specific object such as the signboard 14 provided on the roadside of the road (lane) 10 on which the automobile 1 is traveling disappears laterally from the captured image 20 in which the monocular imaging device 41 provided on the automobile 1 captures the surroundings in front of the automobile 1.

[0088] The CPU 43 of the exterior imaging device 31 can calculate the distance D(Side) from the monocular imaging device 41 installed on the automobile 1 to the traffic light 12 by substituting the distance Ms from the center of the lane of the road 10 on which the automobile 1 is traveling to the signboard 14 on the roadside, the distance Ld from the center of the lane of the road 10 on which the automobile 1 is traveling to the center position of the automobile 1 in the vehicle width direction, the peripheral imaging field of view (field of view) θs in the vehicle width direction of the monocular imaging device 41 from the front of the automobile 1, and the yaw angle θc of the automobile 1 toward the roadside side relative to the lane direction into the following equation 3. In this way, the CPU 43 of the exterior imaging device 31 can use information on the peripheral imaging angle of view of the monocular imaging device 41 provided on the automobile 1, information on the position of the monocular imaging device 41 (installation height on the automobile 1, amount of snow accumulation such as snow that changes the height of the road surface), and information on the attitude of the automobile 1 (yaw angle in the vehicle width direction) to calculate the relative position (distance, direction, spacing) between the automobile 1 (monocular imaging device 41) and the roadside signboard 14 as a specific object at the time of passing. The CPU 43 of the vehicle outside image capture device 31 can then calculate the position of the automobile 1 according to the positioning system by using the calculated relative position based on the position of the roadside signboard 14 as a specific object according to the positioning system. Furthermore, the CPU 43 of the outside-vehicle image capture device 31 can calculate the difference between the position of the automobile 1 determined by the positioning system and the vehicle's own position determined by the GNSS receiver 33 of the vehicle as correction information for the positioning of the vehicle. This enables the CPU 43 of the outside-vehicle image capture device 31 to apply the correction information to the vehicle position measured by the GNSS receiver 33 of the vehicle, for example, and obtain the vehicle position in the positioning system with a high degree of accuracy. In addition, the CPU 43 of the outside-vehicle imaging device 31 applies correction information to the positions of surrounding objects detected based on the captured image 20 of the monocular imaging device 41, thereby making it possible to obtain the positions of the surrounding objects in the positioning system with a high degree of accuracy.

[0089] D(Side)=(Ms-Ld) / TAN(ABS(θs)-ABS(θc))...Equation 3

[0090] In the above description, when a specific object such as a stop line 11, a traffic light 12, or a roadside sign 14 is passed, the CPU 43 of the exterior image capture device 31 updates the position correction information based on the passing of the specific object, and uses the latest correction information for the correction process. In addition, for example, the CPU 43 of the exterior image capture device 31 may accumulate and store in the memory 42 multiple pieces of position correction information generated based on the passing timing of multiple specific objects, and calculate their representative value, average value, average value within a 3σ or 2σ variance range, value obtained by deep learning, etc., to use as the position correction information. Furthermore, the CPU 43 of the outside-vehicle image capture device 31 may use, for the position correction process, position correction information that has been generated in a state that is closest to the current driving environment during driving.

[0091] As described above, in this embodiment, correction information for correcting the positioning of the vehicle 1 while it is moving is generated based on the vehicle 1 passing through a specific position whose position is known. Specifically, in this embodiment, the position of the positioning system of the high-precision map data 34, which is not based on the positioning of the vehicle 1, is acquired as the position information of the specific position whose passing timing is acquired. Also, in this embodiment, the timing when the vehicle 1 passes through a specific position whose position is known is acquired, and position correction information is generated based on the position of the specific position in the positioning system for the positioning of the vehicle 1 at the acquired passing timing. As a result, in this embodiment, the positioning of the vehicle can be corrected so as to be aligned with the positioning system of the high-precision map data 34 including the position information of the specific position. Then, by controlling the running of the vehicle 1 using the map data, in this embodiment, the running of the vehicle 1 can be controlled in a state where it is expected that the possibility of safe running is high. In this embodiment, when the vehicle position is measured as the positioning of the automobile 1, the vehicle position can be corrected by the correction information to align with the positioning system of a specific position whose position is known. In this embodiment, the correction information is generated by a simple method based on the automobile 1 passing through a specific position while traveling, so that it is possible to reduce the requirements for the vehicle sensor such as the monocular imaging device 41 used for traveling control while ensuring the performance of the traveling control. The monocular imaging device 41 does not necessarily need to be a high-performance one.

[0092] Moreover, in this embodiment, the position of the automobile 1 is generated based on the positional relationship between the specific object and the monocular imaging device 41 at the time of passing, using the position of the position information of the specific object at the specific position as a reference, and correction information is generated to correct the vehicle position measured by the GNSS receiver 33 of the automobile 1 to the position in the generated positioning system. As a result, in this embodiment, not only the vehicle position of the automobile 1 but also the position of the monocular imaging device 41 provided in the automobile 1 can be aligned to the positioning system of the specific position whose position is known. In this embodiment, by a simple method, for example, not only the vehicle position of the automobile 1 but also the position of the monocular imaging device 41 provided in the automobile 1 can be aligned in a composite manner to the positioning system of the specific position whose position is known in the high-precision map data 34. In this embodiment, even if the accuracy of the mounting position of the monocular imaging device 41 in the automobile 1 is not high, it is possible to obtain a highly accurate position aligned with the positioning system of the specific position as the relative position of a surrounding object such as another vehicle that can be obtained based on the captured image 20 of the monocular imaging device 41 provided in the automobile 1.

[0093] In particular, as in the present embodiment, by acquiring the timing at which a specific object at a specific position disappears from the captured image 20 of the monocular imaging device 41 as the passing timing, the angle of view and the imaging range of the monocular imaging device 41 can also be adjusted to the positioning system of the specific position whose position is known. As a result, the position of a peripheral object such as another vehicle captured by the monocular imaging device 41 is adjusted to the positioning system of the specific position whose position is known. Even if the positional accuracy within the angle of view of the monocular imaging device 41 itself is not high, the position of the peripheral object such as another vehicle can be well aligned with the positioning system of the specific position whose position is known, at least when the peripheral object is approaching the automobile 1 so as to approach the outer edge of the captured image 20. For example, even if the positional accuracy of a peripheral object such as another vehicle that appears near the center of the captured image 20 is initially low based on the imaging part in the captured image 20 of the monocular imaging device 41, the positional accuracy is improved as the peripheral object approaches the automobile 1, i.e., as the peripheral object is captured so as to approach the outer edge of the captured image 20 of the monocular imaging device 41. The driving control unit 32 controls driving in a positioning system of a specific position whose location is known, and thereby becomes able to perform driving control so as to avoid contact with an approaching surrounding object, such as another vehicle, even if the surrounding object approaches the vehicle.

[0094] The above-described embodiment is an example of a preferred embodiment of the present invention, but the present invention is not limited to this, and various modifications and changes are possible without departing from the gist of the invention.

[0095] For example, in the embodiment described above, the monocular imaging device 41 provided in the automobile 1 captures an image of the surroundings in front of the automobile 1. Alternatively, for example, the monocular imaging device 41 provided on the automobile 1 may capture an image of the surroundings on one side of the automobile 1 in the vehicle width direction, or the surroundings on the rear side of the automobile 1. Even in these cases, the CPU 43 of the exterior imaging device 31 can generate correction information for the position of the positioning system based on the specific object at the specific position by acquiring the passing timing of the specific object at the specific position while the automobile 1 is traveling, the timing at which the specific object disappears from the captured image 20 of the monocular imaging device 41, the timing at which it appears in the captured image 20, or the timing at which it overlaps with the outer edge of the captured image 20.

[0096] In the embodiment described above, the automobile 1 is provided with a monocular imaging device 41 that captures an image of a part of the surroundings of the automobile 1. In addition, the automobile 1 may be provided with an imaging device such as a so-called 360-degree camera that captures images of the surroundings of the automobile 1 in a 360-degree manner. Even in this case, the CPU 43 of the exterior imaging device 31 can generate correction information for the position of the positioning system based on the specific object at the specific position by obtaining the timing when the specific object at the specific position passes a position directly to the side of the imaging device in the vehicle width direction.

[0097] In the embodiment described above, the driving control in FIG. 5 is executed by a CPU (not shown) of the driving control unit 32, and the outside-vehicle imaging control in FIG. 6 and the correction information generation control in FIG. 7 are executed by the CPU 43 of the outside-vehicle imaging device 31. 5 to 7 may be executed by one CPU 43 of the control system 30 of the automobile 1. Also, CPUs (not shown) of multiple other devices of the control system 30 of the automobile 1 may execute some or all of the controls of FIGS. 4, the control system 30 of the automobile 1 can communicate with a server device 39 outside the vehicle using an external communication device 36. In this case, a CPU (not shown) of the server device 39 capable of communicating with the automobile 1 may execute some or all of the controls shown in FIGS. 5 to 7 in the server device 39, the monocular imaging device 41, the process for detecting the passing timing, the sensor device 35 for detecting the attitude of the automobile 1 at the passing timing, etc. must be provided in the automobile 1. A CPU (not shown) of the server device 39 may obtain this information from the automobile 1 to execute some or all of the control shown in FIGS. Specifically, for example, the CPU (not shown) of the server device 39 may execute at least a generation process among a timing acquisition process for acquiring the timing when the traveling automobile 1 passes a specific position whose position is known, a position acquisition process for acquiring the position of a positioning system not based on the positioning of the GNSS receiver 33 of the automobile 1 as position information of the specific position for acquiring the passing timing, and a generation process for generating correction information of the vehicle position based on the position of the specific position for the positioning of the vehicle position of the automobile 1 at the passing timing. Also, the control system 30 of the automobile 1 may execute at least a timing acquisition process among the timing acquisition process, the position acquisition process, and the generation process. The position acquisition process may be executed by the server device 39 or the control system 30 of the automobile 1.

[0098] In the above-described embodiment, the stop line 11, the traffic light 12, the roadside sign 14, and the like are exemplified as specific objects at specific positions in the high precision map data 34 for which the position of the positioning system is known. Other objects whose positioning system locations are known in the high-precision map data 34 include the building 13, towers, statues, and overpasses. In addition, it is possible that the positions of the positioning system for police stations, fire stations, police boxes, telephone booths, station entrances, manholes, and the like will also become known in the high-precision map data 34. The CPU 43 of the outside-vehicle image capture device 31 may determine these objects as specific objects located at specific positions. [Explanation of symbols]

[0099] 1...Automobile (vehicle), 2...Vehicle body, 3...Vehicle interior, 4...Seat, 5...Wheel, 10...Road, 11...Stop line (specific object), 12...Traffic signal (specific object), 13...Building, 14...Signboard (specific object), 16...Other roads, 20...Captured image, 21...Recognition frame, 22...Road surface model, 23...Pixel, 24...Specific pixel, 30...Control system (driving control device), 31...External imaging device, 32...Driving control unit, 33...GNSS receiver, 34...High-precision map data, 35...Sensor device, 36...External communication device, 37...Vehicle network, 38...Acceleration sensor, 39...Server device, 41...Monocular imaging device, 42...Memory, 43...CPU, 44...Timer, 45...Input / output unit, 46...System bus, 51-52...Angle of view, 61...Stop line search area, 62...Traffic signal search area, 63...Signboard search area, 101...GNSS satellite

Claims

1. A vehicle driving control device having a driving control unit that controls driving of the vehicle based on vehicle positioning, An imaging device provided in the vehicle capable of travelling for positioning; a timing acquisition unit that acquires a timing when the traveling vehicle passes a specific position whose position is known; a position acquisition unit that acquires a position of the vehicle that is not based on positioning as position information of the specific position from which the timing acquisition unit acquires passing timing; a generation unit that generates correction information for the positioning of the vehicle at the passing timing acquired by the timing acquisition unit, based on the position of the specific position acquired by the position acquisition unit; having The position acquisition unit is acquiring position information of a stop line drawn on a road surface on which the vehicle is traveling as position information of the specific object at the specific position; The timing acquisition unit is while the vehicle is traveling, a timing at which the stop line drawn on a road surface on which the vehicle is traveling disappears downward so as to be hidden by a body of the vehicle in an image captured by the imaging device provided on the vehicle of the surroundings in front of the vehicle is acquired as the passing timing; The generation unit is using a position of position information of the stop line at the specific position acquired by the position acquisition unit as a reference, generating a relative position between the vehicle and the stop line at the passing timing by calculation using information on a peripheral imaging field angle in a vertical direction of the imaging device provided on the vehicle, information on a height of the imaging device from the road surface, and information on a pitch angle that indicates a change in attitude of the vehicle in a vertical direction; generating a position of the vehicle at the passing timing based on the position of the stop line position information acquired by the position acquisition unit and the acquired relative position; generating the correction information for correcting the position of the vehicle based on the positioning so as to match the generated position of the vehicle; The peripheral imaging angle of view in the vertical direction of the imaging device used by the generation unit for calculation is a peripheral imaging depression angle obtained by excluding a range of the angle of view in the vertical direction of the imaging device that is hidden by a body of the vehicle, The traveling control unit is correcting a position of the vehicle while the vehicle is traveling using the correction information generated for the position of the vehicle at a timing when the stop line at the specific position disappears, thereby controlling the traveling of the vehicle. Vehicle driving control device.

2. The position acquisition unit, acquiring, as the position information of the specific object, position information of a three-dimensional object provided on a road on which the vehicle is traveling; The timing acquisition unit is acquiring, as the passing timing, a timing at which the three-dimensional object provided on a road on which the vehicle is traveling disappears upward from an image captured by the imaging device provided on the vehicle of an area in front of the vehicle; The generation unit is a relative position between the vehicle and the three-dimensional object at the passing timing is calculated using information on a peripheral imaging field angle in a vertical direction of the imaging device provided on the vehicle, information on a height of the imaging device from the road surface, and information on a pitch angle indicating a change in attitude of the vehicle in a vertical direction; 2. A vehicle cruise control device according to claim 1.

3. The position acquisition unit, acquiring, as the position information of the specific object, position information of a three-dimensional object provided adjacent to a road on which the vehicle is traveling; The timing acquisition unit is acquiring, as the passing timing, a timing at which the three-dimensional object provided adjacent to a road on which the vehicle is traveling disappears laterally from an image captured by the imaging device provided on the vehicle of an area in front of the vehicle; The generation unit is a relative position between the vehicle and the three-dimensional object at the passing timing is calculated using information on a peripheral imaging angle of the imaging device in the vehicle width direction, information on a distance from a reference position in the vehicle width direction of the imaging device, and information on a yaw angle indicating a change in attitude of the vehicle in the vehicle width direction; The vehicle cruise control device according to claim 1 or 2.

4. The driving control unit: controlling the travel of the vehicle using map data of a positioning system including the position information of the specific position; The vehicle cruise control device according to any one of claims 1 to 3.

5. A vehicle driving control system including a vehicle having a driving control unit that controls driving of the vehicle based on vehicle positioning, and a server device capable of communicating with the vehicle, The vehicle includes: An imaging device provided in the vehicle capable of travelling for positioning; a timing acquisition unit that acquires a timing when the traveling vehicle passes a specific position whose position is known; a position acquisition unit that acquires a position of the vehicle that is not based on positioning as position information of the specific position from which the timing acquisition unit acquires passing timing; a generation unit that generates correction information for the positioning of the vehicle at the passing timing acquired by the timing acquisition unit, based on the position of the specific position acquired by the position acquisition unit; at least the imaging device and the timing acquisition unit; The server device includes: The vehicle driving control system includes at least the timing acquisition unit, the position acquisition unit, and the generation unit that are not provided in the vehicle, The position acquisition unit is acquiring position information of a stop line drawn on a road surface on which the vehicle is traveling as position information of the specific object at the specific position; The timing acquisition unit is while the vehicle is traveling, a timing at which the stop line drawn on a road surface on which the vehicle is traveling disappears downward so as to be hidden by a body of the vehicle in an image captured by the imaging device provided on the vehicle of the surroundings in front of the vehicle is acquired as the passing timing; The generation unit is using a position of position information of the stop line at the specific position acquired by the position acquisition unit as a reference, generating a relative position between the vehicle and the stop line at the passing timing by calculation using information on a peripheral imaging field angle in a vertical direction of the imaging device provided on the vehicle, information on a height of the imaging device from the road surface, and information on a pitch angle that indicates a change in attitude of the vehicle in a vertical direction; generating a position of the vehicle at the passing timing based on the position of the stop line position information acquired by the position acquisition unit and the acquired relative position; generating the correction information for correcting the position of the vehicle based on the positioning so as to match the generated position of the vehicle; The peripheral imaging angle of view in the vertical direction of the imaging device used by the generation unit for calculation is a peripheral imaging depression angle obtained by excluding a range of the angle of view in the vertical direction of the imaging device that is hidden by a body of the vehicle, The traveling control unit of the vehicle correcting a position of the vehicle while the vehicle is traveling using the correction information generated for the position of the vehicle at a timing when the stop line at the specific position disappears, thereby controlling the traveling of the vehicle. Vehicle driving control system.

Images