Vehicle control device, vehicle control method, and vehicle control program

Abstract

To provide a vehicle control device that can more accurately measure the current position of a vehicle.SOLUTION: A vehicle control device 1 controls the travel of a vehicle provided with an imaging apparatus, a radar, and a remote sensor. The vehicle control device 1 includes processing of calculating the absolute position of the vehicle with reference to the position of an artificial satellite, processing of calculating the relative position of the vehicle indicating a position relative to a reference station located at a known point, processing of controlling the travel of the vehicle by using position information of the vehicle based on the absolute position or the relative position, and processing of acquiring information on an external environment of the vehicle from the imaging apparatus, radar, and remote sensor. The processing of controlling the travel of the vehicle controls the travel of the vehicle by using the position information based on the absolute position or the position information based on the relative position according to an external video, a detected object, and the distance to the object.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a vehicle control program, and particularly to a vehicle control device, a vehicle control method, and a vehicle control program for controlling the running of a vehicle based on information on the current position of the running vehicle.

Background Art

[0002] In recent years, in order to realize the safety and comfort of drivers, vehicles equipped with an ADAS (Advanced Driver Assistance System) that enables the vehicle itself to grasp information on the surrounding external environment, control the running of the vehicle on behalf of the driver, and perform autonomous driving are known. In addition, when a driving obstacle occurs due to a malfunction of the vehicle or the like, a remote driving technology for operating the running of the vehicle by communication via a network is known in order to safely stop the vehicle or safely continue running (see, for example, Patent Document 1).

[0003] In the vehicle control device described in Patent Document 1, any one of a remote driving mode, an autonomous driving mode, and a driving support mode can be selected, and the vehicle can be controlled in the selected operation mode. When the remote driving mode is selected, an operator remotely operates the vehicle using a remote driving device connected to the control device through a network.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, in a vehicle remote operation system such as Patent Document 1, a technology for accurately measuring the current position of a traveling vehicle is required, and this technology is used for automatic driving control or for a navigation service that guides a driver or operator to a destination along a route. For example, in the automatic operation method described in Patent Document 2, a GPS signal is received during the travel of the vehicle to acquire the position (absolute position) of the vehicle in real time. When the reliability of the position accuracy of the vehicle decreases, the coordinates and azimuth based on GPS (Global Positioning System) and the coordinates and azimuth based on an inertial measurement unit (IMU) are matched to correct the absolute position. Under such circumstances, a technology for more accurately measuring the current position of a traveling vehicle has been required.

[0006] An object of the present invention is to provide a vehicle control device, a vehicle control method, and a vehicle control program that can more accurately measure the current position of a traveling vehicle and can measure the position information of the vehicle in response to the external environment of the vehicle.

Means for Solving the Problems

[0007] According to the vehicle control device of the present invention, the above problems are solved by a vehicle control device that controls the travel of a vehicle including an imaging device, a radar that detects an object, and a remote sensor that measures the distance to the object, and acquires GNSS information using a GNSS receiver mounted on the vehicle from an artificial satellite, calculates the absolute position of the vehicle based on the position of the artificial satellite, corrects the GNSS information for correcting the error of the GNSS information using a reference station located at a known point, calculates the relative position of the vehicle indicating the relative position with respect to the reference station located at the known point, controls the travel of the vehicle using the position information of the vehicle based on the relative position, and acquires information on the external environment of the vehicle from the imaging device, the radar, and the remote sensor. before The GNSS correction information for correcting the error of the GNSS information acquired Corrects the absolute position and calculates the relative position of the vehicle indicating the relative position with respect to the reference station located at the known point. before The recorded absolute position or The relative position position Based on the position information of the vehicle, a process of controlling the travel of the vehicle, and a process of acquiring information on the external environment of the vehicle from the imaging device, the radar, and the remote sensor are provided. acquire information on the external environmentThe process includes capturing an external image around the vehicle using the imaging device, detecting an object around the vehicle using the radar, measuring the distance to the object using the remote sensor, and controlling the running of the vehicle. around the vehicle, which is the information on the external environment External image while in a state where the area around the vehicle is imaged , around the vehicle, which is the information on the external environment Object while in a state where the area around the vehicle is detected , the external environment information that is the Distance to the object while in a state where it is measured , when the GNSS information can be received to , the position information based on the absolute position, or higher position accuracy than the position information based on the absolute position Either the position information based on the relative position select according to the information on the external environment Control the running of the vehicle perform This solves the problem.

[0008] Also, for a computer that controls the running of a vehicle equipped with an imaging device, a radar for detecting an object, and a remote sensor for measuring the distance to the object, the computer acquires GNSS information from an artificial satellite using a GNSS receiver mounted on the vehicle, calculates the absolute position of the vehicle based on the position of the artificial satellite, and corrects the absolute position with GNSS correction information for correcting the error of the GNSS information recorded by a reference station located at a known point or the GNSS correction information for correcting the error of the GNSS information recorded by the reference station located at a known point, calculates the relative position of the vehicle indicating the relative position to the reference station located at the known point, controls the running of the vehicle using the position information of the vehicle based on the absolute position or the relative position, and acquires information on the external environment of the vehicle from the imaging device, the radar, and the remote sensor. In this process, an external image around the vehicle is captured using the imaging device, an object around the vehicle is detected using the radar, the distance to the object is measured using the remote sensor, before GNSS correction information for correcting the error of the GNSS information recorded by the above-mentioned reference station acquired Correct the absolute position, and calculate the relative position of the vehicle indicating the relative position to the reference station located at the known point. before The above-mentioned absolute position or The above-mentioned relative position position Based on the position information of the vehicle, control the running of the vehicle, and acquire information on the external environment of the vehicle from the imaging device, the radar, and the remote sensor. In the process of the vehicle, an external image around the vehicle is captured using the imaging device, an object around the vehicle is detected using the radar, the distance to the object is measured using the remote sensor, acquire information on the external environment In this process, an external image around the vehicle is captured using the imaging device, an object around the vehicle is detected using the radar, the distance to the object is measured using the remote sensor, In the process of controlling the running of the vehicle, the area around the vehicle, which is the information on the external environment External image while in a state where the area around the vehicle is imaged , around the vehicle, which is the information on the external environment Object while in a state where the area around the vehicle is detected , the external environment information that is the Distance to the object while in a state where it is measured , when the GNSS information can be received to , the position information based on the absolute position, orhigher position accuracy than the position information based on the absolute position Any of the position information based on the relative position select according to the information on the external environment Control the running of the vehicle perform It is also solved by including this. Further, the problem is that a computer that controls the running of a vehicle equipped with an imaging device, a radar that detects an object, and a remote sensor that measures the distance to the object obtains GNSS information using a GNSS receiver mounted on the vehicle from an artificial satellite, and calculates the absolute position of the vehicle based on the position of the artificial satellite, and a reference station located at a known point or before GNSS correction information for correcting the error of the GNSS information acquired Correct the absolute position and calculate the relative position of the vehicle indicating the relative position to the reference station located at the known point before The absolute position or The relative position position Control the running of the vehicle using the position information of the vehicle based on the relative position, and execute the process of obtaining information on the external environment of the vehicle from the imaging device, the radar, and the remote sensor, and in the process of the vehicle acquire information on the external environment In the process, an external image around the vehicle is captured using the imaging device, an object around the vehicle is detected using the radar, and the distance to the object is measured using the remote sensor In the process of controlling the running of the vehicle, the area around the vehicle, which is the information on the external environment External image while in a state where the area around the vehicle is imaged , around the vehicle, which is the information on the external environment Object while in a state where the area around the vehicle is detected , the external environment information that is the Distance to the object while in a state where it is measured When the GNSS information can be received to , either the position information based on the absolute position, or higher position accuracy than the position information based on the absolute position Any of the position information based on the relative position select according to the information on the external environment Control the running of the vehicle perform It is also solved by a vehicle control program including this.

Effect of the Invention

[0009] According to the vehicle control device, vehicle control method, and vehicle control program of the present invention, it is possible to more accurately measure the current position of a running vehicle.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 - 6. As shown in FIG. 1, the vehicle remote operation system S according to this embodiment grasps the external environment of the traveling vehicle V, plans the traveling route of the vehicle V on behalf of the driver, and controls the vehicle V based on the traveling route to make it travel, which is "automatic driving", and realizes "remote driving" in which an operator outside the vehicle V remotely operates (externally operates) the vehicle V to make it travel. It is a system that can perform "mode switching processing" for switching between the automatic driving mode and the remote driving mode. In addition, there is a manual driving mode in which the driver rides in the vehicle V and performs driving operations (details will be described later). In the above mode switching processing, in addition to switching between this manual driving mode and the automatic driving mode, it is also possible to switch between the manual driving mode and the remote driving mode. Note that the operator does not have to be a human, and may be, for example, an AI (artificial intelligence).

[0012] <Hardware Configuration of Vehicle Remote Operation System> As shown in FIGS. 1 - 3, the vehicle remote operation system S is mounted on the vehicle V and includes a vehicle control device 1 that comprehensively controls the running of the vehicle V, an in - vehicle sensor 10 that detects the external environment around the vehicle V, an in - vehicle locator 20 that receives GNSS signals from the artificial satellite SA and the reference station ST and measures the current position of the vehicle V, an in - vehicle ECU 30 that controls the steering, acceleration, and deceleration of the vehicle V, and an in - vehicle communication device 40 that communicates with an operation device 50 and external devices installed outside the vehicle V. In addition, the vehicle remote operation system S includes an operation device 50 that is connected to the vehicle control device 1 via a network (digital communication path) and operates the running of the vehicle V through communication via the network. Of course, the vehicle control device 1 and the operation device 50 may directly communicate with each other.

[0013] As shown in FIG. 2, the vehicle control device 1 is a computer connected to the in - vehicle sensor 10, the in - vehicle locator 20, the in - vehicle ECU 30, and the in - vehicle communication device 40 through an in - vehicle network (CAN). Specifically, it is a computer including a CPU as a data calculation and control processing device, a ROM, a RAM, and an HDD (SSD) as storage devices, and a communication interface for transmitting and receiving information data through the in - vehicle network. In the storage device of the vehicle control device 1, in addition to the main program that performs the necessary functions as a computer, a vehicle control program and a vehicle remote operation program are stored. When these programs are executed by the CPU, the functions of the vehicle control device 1 are exerted. Note that the in - vehicle ECU 30 (integrated ECU 31) and the operation device 50 are also computers having a similar hardware configuration.

[0014] The vehicle control device 1 controls the in - vehicle ECU 30 (integrated ECU 31) based on the information on the external environment obtained from the in - vehicle sensor 10, the information on the current position obtained from the in - vehicle locator 20, and the vehicle information obtained from the in - vehicle ECU 30 in order to execute "automatic driving", thereby controlling the running of the vehicle V. In addition, the vehicle control device 1 wirelessly communicates with the operation device 50 through the in-vehicle communication device 40 to execute "remote driving", and transmits information on the external environment, the current position, and the vehicle information to the operation device 50. The operation device 50 receives these pieces of information, and in addition to displaying on the monitor 51 (navigation monitor 52) the content based on the information on the external environment and the current position, can also notify the operator. More specifically, the vehicle control device 1 is newly installed in the vehicle V equipped in advance with the "automatic driving function (in-vehicle sensors 10, in-vehicle locator 20, in-vehicle ECU 30)" to enhance the performance of the existing "automatic driving function" and newly impart the "remote driving function".

[0015] The in-vehicle sensors 10 detect moving objects (other vehicles, pedestrians, etc.), various structures, road shapes, etc. around the vehicle V as the external environment around the vehicle V. Specifically, the in-vehicle sensors 10 are mainly composed of a plurality of imaging devices 11, a plurality of radars 12, and a plurality of lidars 13. Note that the in-vehicle sensors 10 may further include detection sensors other than those described above.

[0016] The imaging device 11, also referred to as an imaging apparatus, is a small imaging camera (wide-angle camera) that captures (images) the external video around the vehicle V. To execute the "sensing function" for the driving control of the vehicle V and the "monitoring function" for the driver (operator), the imaging device 11 creates external video data and transmits the external video data to the vehicle control device 1. A plurality of imaging devices 11 are installed in the vehicle V. The first imaging device 11a, the second imaging device 11b, and the third imaging device 11c are attached to the front glass of the vehicle V and capture the front, the right side, and the left side of the vehicle V. The fourth imaging device 11d is attached to the rear bumper of the vehicle V and captures the rear of the vehicle V. The fifth imaging device 11e and the sixth imaging device 11f are attached to the left and right mirrors of the vehicle V and capture the right rear diagonal and the left rear diagonal of the vehicle V. These are provided as main cameras. In addition, as sub-cameras, the imaging device 11 includes a seventh imaging device 11g attached to the front bumper of the vehicle V to image the front of the vehicle V, and an eighth imaging device 11h and a ninth imaging device 11i attached to the peripheries of the left and right backlights of the vehicle V to image the right rear diagonal and the left rear diagonal of the vehicle V. In this embodiment, a total of nine imaging devices 11 are attached to predetermined positions on the vehicle V. However, the number and attachment positions of the imaging devices 11 can be changed according to the vehicle type and shape of the vehicle V. The same applies to the radar 12 and the lidar 13. As another example of the sub-camera, the seventh imaging device 11g may be attached to the upper part of the back glass (rear glass) of the vehicle V to image the rear of the vehicle V from that position. In that case, it is preferable that the eighth imaging device 11h is attached to the right A-pillar at the front of the vehicle V, and the ninth imaging device 11i is attached to the left A-pillar at the front.

[0017] The radar 12 is a millimeter-wave radar that transmits radio waves while continuously changing the irradiation direction, receives reflected waves from the target object to detect the target object (measures the position and speed of the target object), and performs three-dimensional spatial imaging. Compared with the imaging device 11 and the lidar 13, it can detect accurately even in environmental situations such as at night with poor visibility or in bad weather. The radar 12 acquires the detection result data (detection signal) of the above target object and transmits the detection result data toward the vehicle control device 1. A plurality of radars 12 are mounted on the vehicle V, and include a first radar 12a and a second radar 12b attached to the peripheries of the left and right front lights of the vehicle V, and a third radar 12c and a fourth radar 12d attached to the peripheries of the left and right backlights of the vehicle V. Note that the radar 12 is not particularly limited to a millimeter-wave radar, and may be a radar such as a laser radar or an ultrasonic sensor.

[0018] The lidar 13 is a remote sensor that measures the distance to a target object by irradiating laser light and receiving the reflected light from the target object, and performs three-dimensional spatial imaging. Compared with the imaging device 11 and the radar 12, the distance to surrounding target objects can be measured in units of several centimeters. The lidar 13 acquires distance measurement data obtained by measuring the distance to the target object, and transmits the distance measurement data toward the vehicle control device 1. A plurality of lidars 13 are mounted on the vehicle V, including a first lidar 13a and a second lidar 13b attached to the periphery of the left and right front lights of the vehicle V, a third lidar 13c attached to the back bumper of the vehicle V, and a fourth lidar 13d and a fifth lidar 13e attached to the periphery of the left and right back lights of the vehicle V.

[0019] The in-vehicle locator 20 measures the current position of the vehicle V using a satellite positioning system that uses artificial satellites SA and a reference station ST, and measures the acceleration and angular velocity of the vehicle V in order to improve the measurement accuracy of the current position. Specifically, the in-vehicle locator 20 includes a GNSS receiver 21 that receives GNSS radio waves (GPS radio waves) from a plurality of artificial satellites SA, and an inertial measurement device 22 that measures the acceleration and angular velocity of the vehicle V.

[0020] Specifically, the GNSS receiver 21 is an RTK-GNSS receiver that receives GNSS radio waves from a plurality (specifically, four) of artificial satellites SA and generates "GNSS information" necessary for single-point positioning. In addition, it receives "GNSS correction information" necessary for relative positioning from an external reference station ST. Note that the reference station ST is a fixed reference station set at a known point, which receives GNSS radio waves from a plurality of artificial satellites SA, generates "GNSS correction information", and transmits it toward the GNSS receiver 21. The "GNSS information" is distance information between a plurality of artificial satellites SA and the GNSS receiver 21. The "GNSS correction information" is distance information obtained by correcting the measurement error of the "GNSS information" by the reference station ST located at a known point receiving GNSS radio waves and the reference station ST and the GNSS receiver 21 communicating with each other.

[0021] The inertial measurement device 22, also called an IMU, includes a three-axis gyro sensor (angular velocity meter) and a three-axis acceleration sensor (accelerometer), measures the three-dimensional angular velocity and acceleration of the vehicle V, and transmits information on the acceleration and angular velocity of the vehicle V to the vehicle control device 1. The vehicle control device 1 can measure the current position of the vehicle V within a smaller error range by combining the GNSS information (GNSS correction information) received from the GNSS receiver 21 and the information on the angular velocity and acceleration of the vehicle V received from the inertial measurement device 22 for positioning.

[0022] The in-vehicle ECU 30 is, for example, an ECU for ADAS, is connected to the vehicle control device 1, includes a higher-level integrated ECU 31 that performs transmission and reception of various data, and a steering wheel ECU 32, an accelerator ECU 33, and a brake ECU 34 that are respectively connected to the integrated ECU 31 as this higher level and control the steering, acceleration, and deceleration of the vehicle V in a subdivided manner, forming a hierarchical structure. Note that the steering wheel ECU 32 is also called a driving support computer, and the accelerator ECU 33 and the brake ECU 34 are also called power management control units. Note that the number and functions of the individual ECUs connected to the integrated ECU 31 are not particularly limited to the above three ECUs 32 - 34, and other ECUs may be further provided at the same level as these ECUs.

[0023] The steering wheel ECU 32 controls the electric power steering V1 of the vehicle V in response to an instruction from the integrated ECU 31 and mainly controls the traveling direction of the vehicle V. The electric power steering V1 includes a steering mechanism that steers the front wheels of the vehicle V. For example, in the manual driving mode, the front wheels of the vehicle V are steered by the steering operation of the steering wheel V1a by the driver. Note that the steering wheel V1a is equipped with a torque sensor and an angle sensor, and "mode switching processing" of the driving mode can be performed based on the detection results of these sensors.

[0024] The accelerator ECU 33 controls the electric throttle V2 of the vehicle V in response to an instruction from the integrated ECU 31, and mainly controls the acceleration and deceleration of the vehicle V. The electric throttle V2 is provided with a drive mechanism that outputs a driving force for rotating the drive wheels of the vehicle V. For example, in the manual driving mode, the output of the engine is adjusted in response to the accelerator operation of the accelerator pedal V2a by the driver.

[0025] The brake ECU 34 controls the electromagnetic brake device V3 of the vehicle V in response to an instruction from the integrated ECU 31, and mainly controls the deceleration and stop of the vehicle V. The electromagnetic brake device V3 is attached to each wheel of the vehicle V, and is provided with a mechanism for decelerating or stopping the vehicle V by applying resistance to the rotation of the wheels. For example, in the manual driving mode, the operation of the electromagnetic brake device V3 is adjusted in response to the brake operation of the brake pedal V3a by the driver.

[0026] The in-vehicle communication device 40 is a device that communicates information via a network with an operation device 50 installed outside the vehicle V and an external server (not shown). For example, it transmits the information of the external video acquired by the vehicle control device 1 and the information of the current position as the information necessary for "remote driving" to the operation device 50. Also, it receives the driving operation information of the vehicle V from the operation device 50 that has received the user input by the operator, and transmits it to the vehicle control device 1. The in-vehicle communication device 40 communicates information with an external server (not shown), and can also receive, for example, the latest traffic information, weather information, etc. from the external server.

[0027] As shown in FIG. 3, the operation device 50 is a computer operated by an operator for performing "remote driving" of the vehicle V. As a specific hardware configuration of the operation device 50, it includes a plurality of monitors 51, a navigation monitor 52, a steering wheel 53, an accelerator pedal 54, a brake pedal 55, and a plurality of operation switches 56. Note that the operation device 50 may further include components such as a speaker, a microphone, and a shift lever.

[0028] The monitor 51 and the navigation monitor 52 are display units that output visual information for performing "remote driving". On the monitor 51, a composite video (composite image) obtained by synthesizing external images of the vehicle V captured by a plurality of imaging devices 11a - 11i based on predetermined layout information is displayed. The predetermined layout information is, for example, a display mode with a layout that does not create blind spots for the operator and is easy for the operator to operate. At this time, it is also possible to store a plurality of layout information in a predetermined storage unit of the vehicle V in association with a layout ID (layout identification information). In this case, after switching the layout display using an operation switch or the like described later and then performing an operation to change the layout information, the changed layout ID is transmitted from the operation device 50 (operation switch 56) to the vehicle V. Although details will be described later, at this time, in the vehicle V, a composite video obtained by synthesizing external images based on the layout information corresponding to the changed layout ID is generated, and the composite video is displayed on the monitor 51.

[0029] The steering wheel 53 is an operation unit that is operated by an operator and is used to adjust the steering angle (steering amount) of the vehicle V. The accelerator pedal 54 and the brake pedal 55 are operation units that are respectively operated by an operator and are used to adjust the drive of the electric throttle V2 and the operation of the electromagnetic brake device V3 of the vehicle V. The plurality of operation switches 56 are used, for example, to input setting information for performing "remote driving" by the user. For example, by appropriately operating the operation switch 56 by the operator, it is possible to switch the external image (composite image) of the vehicle V to a predetermined layout display or to switch the driving mode between the automatic driving mode and the remote driving mode.

[0030] <Functions of the Vehicle Remote Operation System> As shown in Fig. 4, from a functional perspective, the vehicle control device 1 mainly includes a storage unit 100 that stores various programs and various data, an external information acquisition unit 101, an absolute position calculation unit 102, a relative position calculation unit 103, a corrected position calculation unit 104, a reception determination unit 105, a position identification unit 106, a video processing unit 107, a communication unit 108, a vehicle control unit 109, a speed calculation unit 110, and a position information selection unit 111. These are composed of a CPU, ROM, RAM, HDD, a communication interface, and various programs, etc.

[0031] Regarding the operation device 50, from a functional perspective, it mainly includes a storage unit 500 that stores various programs and various data, a communication unit 501 that transmits and receives various data to and from the vehicle control device 1, a screen display unit 502 that displays the external video of the vehicle V and vehicle information on the monitor 51, and also displays content based on the information of the current position of the vehicle V (for example, vehicle navigation) on the navigation monitor 52, an operation data creation unit 503 that receives user operations and creates operation data, and a user notification unit 504 that notifies the operator.

[0032] Hereinafter, the functions of the vehicle control device 1 will be described in detail. <<Main Functions>> The external information acquisition unit 101 acquires "detection information of the external environment" around the vehicle V from the in-vehicle sensor 10, and also acquires "measurement information of the current position" of the vehicle V from the in-vehicle locator 20. Specifically, as the "detection information of the external environment", it acquires external video data around the vehicle V from the imaging device 11, acquires detection result data of target objects around the vehicle V from the radar 12, and acquires distance measurement data obtained by measuring the distance to the target object of the vehicle V from the lidar 13. Also, as the "measurement information of the current position", it acquires GNSS information (GNSS correction information) from the GNSS receiver 21, and acquires information on the angular velocity and acceleration of the vehicle V from the inertial measurement unit 22. Note that the external information acquisition unit 101 may further acquire "vehicle information" of the vehicle V from the in-vehicle ECU 30. Examples of the "vehicle information" include "steering angle information" obtained from the steering wheel ECU 32, "throttle opening information" obtained from the accelerator ECU 33, "brake depression amount information" obtained from the brake ECU 34, and the like.

[0033] The absolute position calculation unit 102 acquires the above-mentioned "GNSS information" necessary for independent positioning and calculates the "absolute position" of the vehicle V by independent positioning. The "absolute position" of the vehicle V is a three-dimensional position of the vehicle V obtained by receiving GNSS radio waves from a plurality of artificial satellites SA, measuring the distances between the artificial satellites SA located at known points and the vehicle V, and solving a three-dimensional equation for obtaining an unknown point from the respective measured distances (corresponding to GNSS information). As shown in FIG. 5, the positional accuracy of the "absolute position" is about ±10 m.

[0034] The relative position calculation unit 103 acquires the above-mentioned "GNSS correction information" necessary for relative positioning, corrects the "absolute position" by relative positioning, and calculates the "relative position" of the vehicle V. The "relative position" of the vehicle V is a three-dimensional position of the vehicle V obtained by receiving GNSS radio waves also at a reference station ST located at a known point, acquiring distances with smaller measurement errors from the reference station ST (the distances between the respective artificial satellites SA and the vehicle V), and obtaining from the respective measured distances (corresponding to GNSS correction information). As shown in FIG. 5, the positional accuracy of the "relative position" is about ±40 cm, and the positional accuracy is higher than that of the absolute position. In FIG. 5, the "GNSS correction information" is expressed as "GNSS information + RTK information". As methods for calculating the "relative position", there are a calculation method of the RTK positioning method (interferometric positioning method) and a calculation method of the DGPS positioning method (relative positioning method). Any calculation method may be used. Note that the reference station ST is, in principle, the reference station installed at the position closest to the vehicle V among a plurality of reference stations ST located at known points.

[0035] The correction position calculation unit 104 acquires the above "angular velocity and acceleration information" of the vehicle V, and calculates a "corrected absolute position" obtained by correcting the absolute position of the vehicle V based on the "GNSS information" and the "acceleration and angular velocity information". The "corrected absolute position" of the vehicle V is a three-dimensional position of the vehicle V obtained by combining the GNSS information and the information on the angular velocity and acceleration of the vehicle V (also called IMU information) for positioning. The position accuracy of the "corrected absolute position" is higher than that of the absolute position.

[0036] In addition, the correction position calculation unit 104 calculates a "corrected relative position" obtained by correcting the relative position of the vehicle V based on the "GNSS correction information" and the "acceleration and angular velocity information". As shown in FIG. 5, the position accuracy of the "corrected relative position" is about ±5 cm, which is higher than the absolute position and the relative position. In FIG. 5, the "acceleration and angular velocity information" is expressed as "IMU information", and it is described as "GNSS information + RTK information + IMU information".

[0037] The reception determination unit 105 determines whether the GNSS information can be received in real time. If it is determined that the GNSS information can be received in real time, it then determines whether the GNSS correction information can be received in real time. Specifically, the reception determination unit 105 assumes a case where there are obstacles around the vehicle V and radio waves cannot be received from the artificial satellite SA, or a case where data cannot be transmitted and received to and from the reference station ST, and determines whether radio waves can be received from the artificial satellite SA and whether data can be transmitted and received to and from the reference station ST.

[0038] The position identification unit 106 identifies the current position of the vehicle based on the determination result by the reception determination unit 105. Specifically, as shown in FIG. 6, when it is determined that the GNSS information and the GNSS information can be received in real time (S3 in FIG. 6), the position identification unit 106 identifies the current position of the vehicle V using the "corrected relative position" with the highest position accuracy (S4 in FIG. 6). Also, when it is determined that the GNSS information can be received in real time but the GNSS correction information cannot be received in real time (S3 in FIG. 6), the position identification unit 106 identifies the current position of the vehicle V using the "corrected absolute position" with high position accuracy (S5 in FIG. 6). Furthermore, when it is determined that the GNSS information and the GNSS correction information cannot be received in real time (S2 in FIG. 6), the position identification unit 106 identifies the current position of the vehicle V using the "estimated position" calculated based on the "GNSS information" received immediately before and the "information on acceleration and angular velocity" (S6 in FIG. 6). Note that the "estimated position" is the three-dimensional position of the vehicle V calculated based on the "GNSS information" received immediately before and the "information on acceleration and angular velocity" from the past time point when the "GNSS information" was received to the current time point. The position accuracy of the "estimated position" is equivalent to the position accuracy of the "absolute position".

[0039] The video processing unit 107 acquires external video data of the vehicle V from a plurality of imaging devices 11a - 11i, and creates a composite video (composite video data) by synthesizing each external video based on predetermined layout information. By generating the above composite video and transmitting the generated composite video data to the operating device 50, the amount of data to be transmitted can be reduced (the number of communication lines can be reduced) compared to the case of transmitting a plurality of external video data, and the cost associated with data communication can be reduced.

[0040] The communication unit 108 executes data transmission and reception between the vehicle control device 1 and the operating device 50 using the in-vehicle communication device 40. Specifically, the communication unit 108 transmits, to the operation device 50, the "detection information of the external environment" obtained by the external information acquisition unit 101 and the "information of the current position" specified by the position specification unit 106 as information necessary for "remote driving" of the vehicle V. Note that the communication unit 108 may also transmit the "vehicle information" obtained by the external information acquisition unit 101 to the operation device 50. In addition, the communication unit 108 receives the driving operation information of the vehicle V from the operation device 50 that has received user input by the operator.

[0041] The vehicle control unit 109 controls the integrated ECU 31 based on the "detection information of the external environment" obtained by the external information acquisition unit 101 and the "information of the current position" specified by the position specification unit 106, and executes "automatic driving" of the vehicle V. In addition, the vehicle control unit 109 controls the integrated ECU 31 based on the "driving operation information" of the vehicle V acquired from the operation device 50, and executes "remote driving" of the vehicle V. Note that when executing "automatic driving" of the vehicle V, the vehicle control unit 109 may acquire the "vehicle information" of the vehicle V from the in-vehicle ECU 30, and further combine the "vehicle information" to control the integrated ECU 31.

[0042] <<Sub-function>> When specifying the current position of the vehicle V, in order to further improve the position accuracy, it may be possible to further acquire the "speed information" of the vehicle V. Specifically, the speed calculation unit 110 acquires the "angular velocity and acceleration information" of the vehicle V, and calculates the "speed" of the vehicle V by integrating the acceleration and the angular velocity. Then, the corrected position calculation unit 104 can calculate a "corrected absolute position" with higher position accuracy by combining the "GNSS information", the "angular velocity and acceleration information", and the newly acquired "speed information" for positioning. Alternatively, a "corrected relative position" can be calculated. In addition, when calculating the "speed" of the vehicle V, the speed calculation unit 110 may calculate the "speed" of the vehicle V by processing the "GNSS information (GNSS correction information)" and the "acceleration and angular velocity information" with a Kalman filter. In this way, the "speed" can be calculated with higher accuracy. In addition, when identifying the current position of the vehicle V, the "steering angle information" of the vehicle V may be acquired and further combined and calculated. For example, by newly installing a steering angle sensor on the vehicle V, the "steering angle information" can be acquired through the steering angle sensor. In addition, when acquiring the "speed information" of the vehicle V, a wheel speed sensor may be newly installed on the vehicle V, and the "speed information" may be acquired through the wheel speed sensor.

[0043] Also, when executing the "automatic driving" or "remote driving" of the vehicle V in an environment where there are no obstacles around the vehicle V, it may be preferable to prioritize reducing the data communication volume and the cost associated with data communication over the position accuracy of the current position of the vehicle V. Specifically, the position information selection unit 111 selects whether to use the "relative position" or the "corrected relative position" based on the detection information of the external environment obtained by the external information acquisition unit 101 before S7 in FIG. 6. More specifically, when there are no moving objects or buildings (structures) around the vehicle V in the detection information of the external environment (for example, when driving in the wilderness or grassland), it is advisable to select the "relative position" in order to suppress the data communication volume of the vehicle control device 1. That is, it is acceptable not to acquire the angular velocity and acceleration information of the vehicle V (the measurement of the angular velocity and acceleration may be temporarily interrupted). Alternatively, when there are moving objects or buildings around the vehicle V in the detection information of the external environment (for example, when driving in an urban area), it is advisable to prioritize the position accuracy of the current position of the vehicle V and select the "corrected relative position". In this way, the method for identifying the current position of the vehicle V can be appropriately selected according to the external environment around the vehicle V. The selection by the position information selection unit 111 is preferably made according to preset selection conditions or user-set selection conditions.

[0044] With the above configuration, it is possible to realize a vehicle remote operation system that can flexibly measure the position information of the vehicle V according to the external environment (radio wave situation and data communication situation) around the vehicle V and specify the current position with higher position accuracy.

[0045] <Vehicle Remote Operation Method> Next, the processing of the vehicle remote operation program (vehicle remote operation method) executed by the vehicle remote operation system S will be described based on FIG. 6. The above program according to the present embodiment is a program for realizing the above-described external information acquisition unit 101, absolute position calculation unit 102, relative position calculation unit 103, corrected position calculation unit 104, reception determination unit 105, position specification unit 106, video processing unit 107, communication unit 108, and vehicle control unit 109 as functional components of the vehicle control device 1 provided with the storage unit 100, and the CPU of the vehicle control device 1 executes this vehicle maintenance support program. The above program is executed by receiving an operation instruction from a user (specifically, a driver or an operator).

[0046] In the vehicle remote operation flow shown in FIG. 6, first, the external information acquisition unit 101 starts acquiring "GNSS information" through the GNSS receiver 21 from step S1. Actually, the external information acquisition unit 101 starts acquiring "GNSS information" and "GNSS correction information" through the GNSS receiver 21, and also starts acquiring "angular velocity and acceleration information" of the vehicle V through the inertial measurement device 22. In addition, the external information acquisition unit 101 also starts acquiring "detection information of the external environment" around the vehicle V from the in-vehicle sensor 10.

[0047] Next, in step S2, the reception determination unit 105 determines whether GNSS information can be received in real time. When it is determined that "GNSS information" can be received (step S2: Yes), the process proceeds to step S3, and then the reception determination unit 105 determines whether "GNSS correction information" can be received in real time. When the reception determination unit 105 determines that "GNSS correction information" can also be received (step S3: Yes), the process proceeds to step S4.

[0048] In step S4, the absolute position calculation unit 102, the relative position calculation unit 103, and the correction position calculation unit 104 calculate the "corrected relative position (relative position)" of the vehicle V. Specifically, first, the absolute position calculation unit 102 acquires "GNSS information" and calculates the "absolute position" of the vehicle V by single positioning. Then, the relative position calculation unit 103 acquires "GNSS correction information" and calculates the "relative position" of the vehicle V by relative positioning. Then, the correction position calculation unit 104 acquires "angular velocity and acceleration information" and calculates the "corrected relative position" obtained by correcting the relative position of the vehicle V. Then, the process proceeds to step S7. The "corrected relative position" is the position information with the highest position accuracy.

[0049] In step S3 above, when the reception determination unit 105 determines that "GNSS correction information" cannot be received (step S3: No), the process proceeds to step S5. In step S5, the absolute position calculation unit 102 and the correction position calculation unit 104 calculate the "absolute correction position (absolute position)" of the vehicle V. Specifically, first, the absolute position calculation unit 102 acquires "GNSS information" and calculates the "absolute position" of the vehicle V by single positioning. Then, the correction position calculation unit 104 acquires "angular velocity and acceleration information" and calculates the "corrected absolute position" obtained by correcting the absolute position of the vehicle V. Then, the process proceeds to step S7.

[0050] In step S2 above, when the reception determination unit 105 determines that "GNSS information" cannot be received (step S2: No), the process proceeds to step S6. In step S6, the position identification unit 106 calculates the "estimated position" of the vehicle V. Specifically, the position identification unit 106 calculates an "estimated position" based on the "GNSS information" received immediately before and the "acceleration and angular velocity information" (corresponding to an estimated position calculation unit). Then, the process proceeds to step S7.

[0051] Next, in step S7, the position identification unit 106 identifies the current position of the vehicle V using the calculated position information. When proceeding from step S4 above, the position identification unit 106 identifies the current position of the vehicle V using the "corrected relative position" with the highest position accuracy. When proceeding from step S5 above, the position identification unit 106 identifies the current position of the vehicle V using the "corrected absolute position". When proceeding from step S6 above, the position identification unit 106 identifies the current position of the vehicle V using the "estimated position".

[0052] Next, in step S8, the communication unit 108 transmits, as information necessary for "remote operation" of the vehicle V, the "current position information" identified by the position identification unit 106 and the "detection information of the external environment" obtained by the external information acquisition unit 101 to the operation device 50. Note that the operation device 50 receives these pieces of information, displays the content based on the current position information and the detection information of the external environment on the monitor 51 and the navigation monitor 52, and performs user notification to the operator as necessary.

[0053] If it finally receives the stop of the remote operation from the user while going through steps S1 to S8 above (step S9: Yes), the process in FIG. 6 ends. On the other hand, if it has not received the stop of the remote operation (step S9: No), it returns to step S2. With the configuration of the above vehicle remote operation program, it becomes possible to more accurately measure the current position of the vehicle V in response to the external environment around the vehicle V.

[0054] <Other Embodiments> In the above embodiment, as shown in FIG. 2, the vehicle remote operation system S includes a vehicle control device 1 and an in-vehicle ECU 30, and a "remote operation function (vehicle control device 1)" is newly added to the vehicle V equipped with an "automatic driving function (in-vehicle ECU 30)", but it can be changed without particular limitation. For example, the vehicle control device 1 may also have the functions of the in-vehicle ECU 30. That is, the vehicle remote operation system S may mainly be composed of a vehicle control device 1 (including the functions of the in-vehicle ECU 30), an in-vehicle sensor 10, an in-vehicle locator 20, an in-vehicle communication device 40, and an operation device 50 (the in-vehicle ECU 30 may be removed from the configuration).

[0055] In the above embodiment, as shown in FIG. 2, the vehicle remote operation system S includes a vehicle control device 1 and an in-vehicle locator 20, but without particular limitation, the vehicle control device 1 may have the in-vehicle locator 20. In that case, it is preferable that the in-vehicle locator 20 has an absolute position calculation unit 102 that calculates the absolute position of the vehicle V, a relative position calculation unit 103 that calculates the relative position of the vehicle V, and a correction position calculation unit 104 that calculates the corrected relative position of the vehicle V. And it is preferable that the vehicle control device 1 acquires the information of the absolute position, relative position, and corrected relative position of the vehicle V respectively.

[0056] In the above embodiment, as shown in FIGS. 2 and 4, the correction position calculation unit 104 acquires the information of the acceleration and angular velocity of the vehicle V from the inertial measurement device 22 and calculates the corrected relative position of the vehicle V, but it can be changed without particular limitation. For example, when the vehicle control device 1 identifies the current position of the vehicle V, it may not acquire (use) the information of the acceleration and angular velocity of the vehicle V. In that case, the current position of the vehicle V will be identified using the "absolute position" and "relative position" of the vehicle V. Also, for example, when the vehicle control device 1 identifies the current position of the vehicle V, it may not acquire (use) the information of the acceleration and angular velocity of the vehicle V and may acquire (use) the information of the speed of the vehicle V.

[0057] In the above embodiment, as shown in FIG. 2, the imaging device 11 detects information on the external environment around the vehicle V as part of the in-vehicle sensor 10. However, without being particularly limited, the imaging device 11 may only acquire an external image around the vehicle V. That is, the radar 12 and the lidar 13 may function as the in-vehicle sensor 10 to detect information on the external environment around the vehicle V.

[0058] In the above embodiment, a vehicle remote operation program is stored in a recording medium readable by the vehicle control device 1, and the process is executed by the vehicle control device 1 reading and executing the program. Here, the recording medium readable by the vehicle control device 1 refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, and the like. In addition, a dedicated web app may be launched using a terminal (portable terminal) that serves as the vehicle control device 1, and the vehicle remote operation program may be executed on a web browser.

[0059] In the above embodiment, the vehicle remote operation system and the vehicle remote operation method according to the present invention have been mainly described. However, the above embodiment is merely an example for facilitating the understanding of the present invention and does not limit the present invention. The present invention can be changed and improved without departing from its gist, and it goes without saying that equivalents of the present invention are included therein.

Explanation of Reference Numerals

[0060] S Vehicle remote operation system V Vehicle V1 Electric power steering V1a Steering wheel V2 Electric throttle V2a Accelerator pedal V3 Electromagnetic brake device V3a Brake pedal 1 Vehicle control device 10 In-vehicle sensor 11 Imaging device 11a - 11i First imaging device - Ninth imaging device 12 Radar (millimeter - wave radar) 12a - 12d First radar - Fourth radar 13 Lidar 13a - 13e First lidar - Fifth lidar 20 Vehicle - mounted locator 21 GNSS receiver (RTK - GNSS receiver) 22 Inertial measurement unit (IMU) 30 Vehicle - mounted ECU 31 Integrated ECU 32 Steering wheel ECU 33 Accelerator ECU 34 Brake ECU 40 Vehicle - mounted communication device 50 Operating device 51 Display monitor 52 Navigation display monitor 53 Steering wheel 54 Accelerator pedal 55 Brake pedal 56 Operation switch 100 Memory unit 101 External information acquisition unit 102 Absolute position calculation unit 103 Relative position calculation unit 104 Corrected position calculation unit 105 Reception determination unit 106 Position identification unit 107 Image processing unit 108 Communication unit (First communication unit) 109 Vehicle control unit 110 Speed calculation unit 111 Position information selection unit 500 Memory unit 501 Communication unit (Second communication unit) 502 Screen display unit 503 Operation data creation unit 504 User notification unit SA Artificial satellite ST Reference station

Claims

1. An imaging device, a radar for detecting an object, a vehicle control device for controlling the running of a vehicle including a remote sensor for measuring the distance to the object, a process of acquiring GNSS information using a GNSS receiver mounted on the vehicle from a satellite and calculating an absolute position of the vehicle based on the position of the satellite, a process of acquiring GNSS correction information for correcting an error of the GNSS information from a reference station located at a known point, correcting the absolute position, and calculating a relative position of the vehicle indicating a relative position to the reference station located at the known point, a process of controlling the running of the vehicle using the position information of the vehicle based on the absolute position or the relative position, a process of acquiring information on the external environment of the vehicle from the imaging device, the radar, and the remote sensor, The process of acquiring information on the external environment of the vehicle is imaging an external video around the vehicle using the imaging device, detecting an object around the vehicle using the radar, measuring the distance to the object using the remote sensor, The process of controlling the running of the vehicle is in a state where an external video around the vehicle which is the information on the external environment is imaged, in a state where an object around the vehicle which is the information on the external environment is detected, and in a state where the distance to the object which is the information on the external environment is measured, When the GNSS information can be received, the running of the vehicle is controlled by selecting either the position information based on the absolute position or the position information based on the relative position having higher position accuracy than the position information based on the absolute position according to the information on the external environment. A vehicle control device.

2. A computer for controlling the running of a vehicle including an imaging device, a radar for detecting an object, and a remote sensor for measuring the distance to the object, a process of acquiring GNSS information using a GNSS receiver mounted on the vehicle from a satellite and calculating an absolute position of the vehicle based on the position of the satellite, a process of acquiring GNSS correction information for correcting an error of the GNSS information from a reference station located at a known point, correcting the absolute position, and calculating a relative position of the vehicle indicating a relative position to the reference station located at the known point, a process of controlling the running of the vehicle using the position information of the vehicle based on the absolute position or the relative position, Execute the process of acquiring information on the external environment of the vehicle from the imaging device, the radar, and the remote sensor. In the process of acquiring information on the external environment of the vehicle, an external video around the vehicle is captured using the imaging device, an object around the vehicle is detected using the radar, and the distance to the object is measured using the remote sensor. In the process of controlling the running of the vehicle, when the GNSS information can be received in a state where an external video around the vehicle, which is the information on the external environment, is captured, in a state where an object around the vehicle, which is the information on the external environment, is detected, and in a state where the distance to the object, which is the information on the external environment, is measured, select either the position information based on the absolute position or the position information based on the relative position with higher position accuracy than the position information based on the absolute position according to the information on the external environment, and control the running of the vehicle. A vehicle control method.

3. On a computer that controls the running of a vehicle equipped with an imaging device, a radar for detecting an object, and a remote sensor for measuring the distance to the object, Acquire GNSS information using a GNSS receiver mounted on the vehicle from an artificial satellite, and calculate the absolute position of the vehicle with reference to the position of the artificial satellite. Acquire GNSS correction information for correcting the error of the GNSS information from a reference station located at a known point, correct the absolute position, and calculate the relative position of the vehicle indicating the relative position with the reference station located at the known point. Control the running of the vehicle using the position information of the vehicle based on the absolute position or the relative position. Cause to execute the process of acquiring information on the external environment of the vehicle from the imaging device, the radar, and the remote sensor. In the process of acquiring information on the external environment of the vehicle, an external video around the vehicle is captured using the imaging device, an object around the vehicle is detected using the radar, and the distance to the object is measured using the remote sensor. In the process of controlling the running of the vehicle, when the external image around the vehicle, which is the information of the external environment, is being imaged, when the object around the vehicle, which is the information of the external environment, is being detected, when the distance to the object, which is the information of the external environment, is being measured, and when the GNSS information can be received, the vehicle control program includes selecting either the position information based on the absolute position or the position information based on the relative position with higher position accuracy than the position information based on the absolute position according to the information of the external environment and controlling the running of the vehicle.

Images