Information processing apparatus, information processing method, and information processing program

Abstract

To provide an information processor capable of measuring the present position of a traveling vehicle.SOLUTION: The processor executes a process of acquiring GNSS information necessary for point positioning, a process of calculating an absolute position of the vehicle by point positioning, a process of determining whether or not GNSS correction information necessary for relative positioning can be received, a process of calculating a relative position of the vehicle by relative positioning when the GNSS correction information can be received, a process of calculating a speed by integrating information on an acceleration and an angular velocity of the vehicle, and a process of calculating a corrected relative position of the vehicle based on the GNSS correction information and the information on the acceleration and the angular velocity. The processor executes a process of selecting the relative position when there is no moving object around the vehicle and selecting the corrected relative position when there is a moving object, based on the detection information of the external environment acquired from the in-vehicle sensor, and a process of specifying the current position of the vehicle using the selected position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] In recent years, vehicles equipped with ADAS (Advanced Driver Assistance Systems) have become known that enable autonomous driving by grasping information about the external environment around the vehicle itself and controlling the vehicle's driving on behalf of the driver in order to ensure safety and comfort for the driver. In addition, when a driving obstacle occurs due to a malfunction of the vehicle, etc., a remote driving technology is known that controls the driving of the vehicle through communication via a network in order to safely stop the vehicle or continue driving safely (see, for example, Patent Document 1).

[0003] The vehicle control device described in Patent Document 1 allows a user to select one of a remote driving mode, an automatic driving mode, and a driving assistance mode, and controls the vehicle in the selected operating mode. When the remote driving mode is selected, an operator remotely controls the vehicle using a remote driving device connected to the control device via a network. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-164056 [Patent Document 2] Japanese Patent Publication No. 2020-32873 Summary of the Invention [Problem to be solved by the invention]

[0005] Meanwhile, a vehicle remote control system such as that disclosed in Patent Document 1 requires a technology for measuring the current position of a traveling vehicle with high accuracy, and this technology is used to control automatic driving and for navigation services that guide drivers or operators to routes to their destinations. For example, in the automatic operation method described in Patent Document 2, a GPS signal is received while the vehicle is traveling to acquire the vehicle's position (absolute position) in real time, and if the reliability of the vehicle's position accuracy decreases, the coordinates and azimuth angles based on the GPS (Satellite Positioning System) are aligned with the coordinates and azimuth angles based on an inertial measurement unit (IMU) to correct the absolute position.

[0006] An object of the present invention is to provide an information processing device, an information processing method, and an information processing program that are capable of accurately measuring the current position of a traveling vehicle. [Means for solving the problem]

[0007] The above-mentioned problem is solved by an information processing device of the present invention, which includes a memory and a processor, and the processor executes a program stored in the memory to perform an acquisition process of acquiring GNSS information required for point positioning via a GNSS receiver mounted on a vehicle, an absolute position calculation process of calculating the absolute position of the vehicle by the point positioning, a reception determination process of determining whether GNSS correction information required for relative positioning can be received from an external reference station, and when it is determined that the GNSS correction information can be received, a relative position calculation process of correcting the absolute position by the relative positioning and calculating the relative position of the vehicle, and a reception determination process of determining whether GNSS correction information can be received from an external reference station. a corrected position calculation process that calculates a corrected relative position by correcting the relative position of the vehicle based on the GNSS correction information and the acceleration and angular velocity information; a position information selection process that selects the relative position when no moving object is present around the vehicle based on detection information of the external environment acquired from an on-board sensor mounted on the vehicle, and selects the corrected relative position when a moving object is present around the vehicle; and a position identification process that identifies the current position of the vehicle using the relative position or corrected relative position selected by the position information selection process.

[0008] The above problem can also be solved by an information processing method in which a computer executes the following processes: a process of acquiring GNSS information required for stand-alone positioning via a GNSS receiver mounted on a vehicle; a process of calculating the absolute position of the vehicle by the stand-alone positioning; a reception determination process of determining whether GNSS correction information required for relative positioning can be received from an external reference station; a process of correcting the absolute position by the relative positioning and calculating the relative position of the vehicle if it is determined that the GNSS correction information can be received; a process of calculating the speed of the vehicle by integrating acceleration and angular velocity information acquired from an inertial measurement unit mounted on the vehicle; a process of calculating a corrected relative position in which the relative position of the vehicle is corrected based on the GNSS correction information and the acceleration and angular velocity information; a process of selecting the relative position if no moving objects are present around the vehicle and selecting the corrected relative position if a moving object is present around the vehicle, based on detection information of the external environment acquired from an on-board sensor mounted on the vehicle; The above problem can also be solved by an information processing program that causes a computer to execute the following processes: a process of acquiring GNSS information required for stand-alone positioning via a GNSS receiver mounted on the vehicle; a process of calculating the absolute position of the vehicle by the stand-alone positioning; a reception determination process of determining whether GNSS correction information required for relative positioning can be received from an external reference station; a process of correcting the absolute position by the relative positioning and calculating the relative position of the vehicle if it is determined that the GNSS correction information can be received; a process of calculating the speed of the vehicle by integrating acceleration and angular velocity information acquired from an inertial measurement unit mounted on the vehicle; a process of calculating a corrected relative position by correcting the relative position of the vehicle based on the GNSS correction information and the acceleration and angular velocity information; a process of selecting the relative position if no moving objects are present around the vehicle based on detection information of the external environment acquired from an on-board sensor mounted on the vehicle, and selecting the corrected relative position if a moving object is present around the vehicle; and a process of identifying the current position of the vehicle using the relative position selected by the position information selection process or the corrected relative position. [Effects of the Invention]

[0009] According to the information processing device, the information processing method, and the information processing program of the present invention, it is possible to accurately measure the current position of a traveling vehicle. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating the overall configuration of a vehicle remote control system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating the hardware configuration of the vehicle remote control system (excluding the control device). [Figure 3] FIG. 2 is a diagram illustrating a hardware configuration of the operation device. [Figure 4] FIG. 2 is a diagram illustrating functions of a vehicle control device and an operation device. [Figure 5] 10A and 10B are diagrams for explaining processing by a position specifying unit, and are diagrams comparing the position accuracy of absolute positions, relative positions, and corrected relative positions. [Figure 6] FIG. 2 is a process flow diagram showing a vehicle remote control method according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to FIGS. 1, the vehicle remote operation system S of this embodiment is a system that realizes "automatic driving," in which the external environment of a traveling vehicle V is grasped, a driving route for the vehicle V is planned on behalf of the driver, and the vehicle V is driven by controlling the vehicle V based on the driving route, and "remote driving," in which an operator outside the vehicle V remotely controls (externally controls) the vehicle V to drive the vehicle, and is capable of performing a "mode switching process" that switches between the automatic driving mode and the remote driving mode. In addition, there is a manual driving mode (details of which will be described later) in which the driver gets into the vehicle V and drives it, and the mode switching process not only switches between the manual driving mode and the automatic driving mode, but also switches between the manual driving mode and the remote driving mode. The operator does not have to be a human, but may be, for example, an AI (artificial intelligence).

[0012] <Hardware configuration of vehicle remote control system> As shown in Figures 1 to 3, the vehicle remote control system S comprises a vehicle control device 1 mounted on a vehicle V and providing comprehensive control of the driving of the vehicle V, an on-board sensor 10 for detecting the external environment surrounding the vehicle V, an on-board locator 20 for receiving GNSS signals from an artificial satellite SA and a reference station ST and measuring the current position of the vehicle V, an on-board ECU 30 for controlling the steering, acceleration / deceleration, etc. of the vehicle V, and an on-board communication device 40 for communicating with an operating device 50 installed outside the vehicle V and external devices. The vehicle remote control system S is also connected to the vehicle control device 1 via a network (digital communication path) and includes an operation device 50 for operating the traveling of the vehicle V through communication via the network. Of course, the vehicle control device 1 and the operation device 50 may communicate directly with each other.

[0013] As shown in FIG. 2, the vehicle control device 1 is a computer connected to an on-vehicle sensor 10, an on-vehicle locator 20, an on-vehicle ECU 30, and an on-vehicle communication device 40 via an on-vehicle network (CAN). Specifically, it is a computer equipped with a CPU as a data calculation and control processing device, ROM, RAM and HDD (SSD) as storage devices, and a communication interface for sending and receiving information data via the in-vehicle network. In addition to a main program that performs the functions necessary for a computer, the memory device of the vehicle control device 1 also stores a vehicle control program and a vehicle remote operation program, and the functions of the vehicle control device 1 are performed by executing these programs by the CPU. The in-vehicle ECU 30 (integrated ECU 31) and the operation device 50 are also computers having similar hardware configurations.

[0014] In order to perform "autonomous driving," the vehicle control device 1 controls the driving of the vehicle V by controlling the on-board ECU 30 (overall ECU 31) based on information on the external environment obtained from the on-board sensor 10, information on the current location obtained from the on-board locator 20, and vehicle information obtained from the on-board ECU 30. Furthermore, in order to execute "remote driving," the vehicle control device 1 communicates wirelessly with the operation device 50 via the in-vehicle communication device 40, and transmits information on the external environment, information on the current location, and vehicle information to the operation device 50. The operation device 50 receives this information, and displays content based on the information on the external environment and the information on the current location on the monitor 51 (navigation monitor 52), and can also notify the user to the operator. More specifically, the vehicle control device 1 is newly installed on a vehicle V that is already equipped with an "autonomous driving function (on-board sensor 10, on-board locator 20, on-board ECU 30)," thereby improving the performance of the existing "autonomous driving function" and adding a new "remote driving function."

[0015] The on-board sensor 10 detects the external environment around the vehicle V, such as moving objects (other vehicles, pedestrians, etc.), various structures, road shapes, etc. around the vehicle V, and specifically, is mainly composed of multiple imaging devices 11, multiple radars 12, and multiple lidars 13. The on-vehicle sensor 10 may further include detection sensors other than those described above.

[0016] The photographing device 11, also referred to as an imaging device, is a small photographing camera (wide-angle camera) that photographs (captures) external images around the vehicle V, and creates external image data and transmits the external image data to the vehicle control device 1 in order to perform a "sensing function" for driving control of the vehicle V and a "monitoring function" for the driver (operator). Multiple photographing devices 11 are mounted on the vehicle V, and include a first photographing device 11a, a second photographing device 11b, and a third photographing device 11c that are attached to the windshield of the vehicle V and photograph the front, right side, and left side of the vehicle V, a fourth photographing device 11d that is attached to the back bumper of the vehicle V and photographs the rear of the vehicle V, and a fifth photographing device 11e and a sixth photographing device 11f that are attached to the left and right mirrors of the vehicle V and photograph the right rear and left rear diagonal directions of the vehicle V, as main cameras. The photographing device 11 also includes sub-cameras, namely a seventh photographing device 11g attached to the front bumper of the vehicle V and photographing the area in front of the vehicle V, and an eighth photographing device 11h and a ninth photographing device 11i attached around the left and right backlights of the vehicle V and photographing the area diagonally rear to the right and left of the vehicle V. In this embodiment, a total of nine imaging devices 11 are attached to predetermined positions of the vehicle V, but the number and attachment positions of the imaging devices 11 can be changed depending on the type and shape of the vehicle V. The same applies to the radar 12 and the lidar 13. As another example of the sub-camera, the seventh image capturing device 11g may be attached to the top of the back window (rear window) of the vehicle V and capture images of the rear of the vehicle V from that position. In that case, the eighth image capturing device 11h may be attached to the front right A-pillar of the vehicle V, and the ninth image capturing device 11i may be attached to the front left A-pillar.

[0017] The radar 12 is a millimeter-wave radar that detects a target object by transmitting radio waves while continuously changing the irradiation direction and receiving reflected waves from the target object (measuring the position and speed of the target object), thereby performing three-dimensional spatial imaging. Compared to the imaging device 11 and the lidar 13, it can perform detection with high accuracy even in environmental conditions with poor visibility, such as at night or in bad weather. The radar 12 acquires the detection result data (detection signal) of the target object, and transmits the detection result data to the vehicle control device 1. Multiple radars 12 are mounted on the vehicle V, including a first radar 12a and a second radar 12b mounted around the left and right front lights of the vehicle V, and a third radar 12c and a fourth radar 12d mounted around the left and right back lights of the vehicle V. The radar 12 is not particularly limited to a millimeter wave radar, but may be a laser radar, an ultrasonic sensor, or other radar.

[0018] The lidar 13 is a remote sensor that measures the distance to a target object by emitting laser light and receiving the light reflected from the target object, and performs three-dimensional spatial imaging. Compared to the imaging device 11 and radar 12, the lidar 13 can measure the distance to surrounding target objects in units of a few centimeters. The lidar 13 acquires distance measurement data that measures the distance to the target object, and transmits the distance measurement data to the vehicle control device 1. Multiple riders 13 are mounted on the vehicle V, and include a first rider 13a and a second rider 13b attached around the left and right front lights of the vehicle V, a third rider 13c attached to the back bumper of the vehicle V, and a fourth rider 13d and a fifth rider 13e attached around the left and right back lights of the vehicle V.

[0019] The vehicle-mounted locator 20 measures the current position of the vehicle V using a satellite positioning system that uses artificial satellites SA and reference stations ST, and also measures the acceleration and angular velocity of the vehicle V to improve the accuracy of measuring the current position. Specifically, the on-board locator 20 includes a GNSS receiver 21 that receives GNSS radio waves (GPS radio waves) from a plurality of artificial satellites SA, and an inertial measurement unit 22 that measures the acceleration and angular velocity of the vehicle V.

[0020] The GNSS receiver 21 is specifically an RTK-GNSS receiver that receives GNSS radio waves from multiple (specifically, four) satellites SA, generates "GNSS information" necessary for point positioning, and also receives "GNSS correction information" necessary for relative positioning from an external reference station ST. The reference station ST is a fixed reference station set at a known point, receives GNSS radio waves from a plurality of artificial satellites SA, generates “GNSS correction information”, and transmits it to the GNSS receiver 21. The "GNSS information" is information about the distances between the multiple artificial satellites SA and the GNSS receiver 21. "GNSS correction information" is distance information in which the measurement error of the "GNSS information" is corrected by the reference station ST located at a known point receiving GNSS radio waves and communication between the reference station ST and the GNSS receiver 21.

[0021] The inertial measurement unit 22, also known as an IMU, is equipped with a three-axis gyro sensor (angular velocity sensor) and a three-axis acceleration sensor (accelerometer), measures the three-dimensional angular velocity and acceleration of the vehicle V, and transmits information on the acceleration and angular velocity of the vehicle V to the vehicle control device 1. The vehicle control device 1 can measure the current position of the vehicle V with a smaller error range by combining the GNSS information (GNSS correction information) received from the GNSS receiver 21 with the angular velocity and acceleration information of the vehicle V received from the inertial measurement unit 22.

[0022] The in-vehicle ECU 30 is, for example, an ECU for ADAS, and is connected to the vehicle control device 1 and includes an upper-level comprehensive ECU 31 that transmits and receives various data, and lower-level steering ECU 32, accelerator ECU 33, and brake ECU 34 that are each connected to this upper-level comprehensive ECU 31 and perform detailed control of the steering, acceleration, deceleration, etc. of the vehicle V, forming a hierarchical structure. The steering wheel ECU 32 is also called a driving support computer, and the accelerator ECU 33 and the brake ECU 34 are also called power management control units. The number and functions of the individual ECUs connected to the integrated ECU 31 are not particularly limited to the three ECUs 32-34, and other ECUs may be provided at the same level as these ECUs.

[0023] The steering ECU 32 controls the electric power steering V1 of the vehicle V in response to instructions from the integrated ECU 31, and mainly controls the direction in which the vehicle V travels. The electric power steering V1 includes a steering mechanism for steering the front wheels of the vehicle V. For example, in a manual driving mode, the front wheels of the vehicle V are steered by the driver operating the steering wheel V1a. The steering wheel V1a is equipped with a torque sensor and an angle sensor, and the "mode switching process" of the driving mode can be performed based on the detection results of these sensors.

[0024] The accelerator ECU 33 controls the electric throttle V2 of the vehicle V in response to instructions from the integrated ECU 31, and mainly controls the acceleration and deceleration of the vehicle V. The electric throttle V2 includes a drive mechanism that outputs a driving force to rotate the drive wheels of the vehicle V. For example, in manual driving mode, the electric throttle V2 adjusts the engine output in response to the driver's operation of the accelerator pedal V2a.

[0025] The brake ECU 34 controls the electromagnetic brake device V3 of the vehicle V in response to instructions from the integrated ECU 31, and mainly controls the deceleration and stopping of the vehicle V. The electromagnetic brake device V3 is attached to each wheel of the vehicle V and has a mechanism that applies resistance to the rotation of the wheels to slow down or stop the vehicle V. For example, in manual driving mode, the operation of the electromagnetic brake device V3 is adjusted in response to the braking operation of the brake pedal V3a by the driver.

[0026] The in-vehicle communication device 40 is a device that communicates information via a network with an operation device 50 installed outside the vehicle V and an external server (not shown). For example, it transmits to the operation device 50 information on external images acquired by the vehicle control device 1 as information necessary for "remote driving" and information on the current location. It also receives driving operation information for the vehicle V from the operation device 50 that has accepted user input by the operator, and transmits it to the vehicle control device 1. The in-vehicle communication device 40 communicates with an external server (not shown), and can receive, for example, the latest traffic information, weather information, and the like from the external server.

[0027] As shown in Figure 3, the operating device 50 is a computer that is operated by an operator and performs "remote driving" of the vehicle V, and the specific hardware configuration of the operating device 50 includes multiple monitors 51, a navigation monitor 52, a steering wheel 53, an accelerator pedal 54, a brake pedal 55, and multiple operating switches 56. The operating device 50 may further include components such as a speaker, a microphone, and a shift lever.

[0028] The monitor 51 and the navigation monitor 52 are display units that output visual information for "remote driving," and the monitor 51 displays a composite video (composite image) that is created by combining external images of the vehicle V captured by multiple image capturing devices 11a-11i based on predetermined layout information. The predetermined layout information is, for example, a display mode of a layout that does not create blind spots for the operator and is easy for the operator to operate. In this case, it is possible to store a plurality of pieces of layout information in a predetermined storage unit of the vehicle V, each associated with a layout ID (layout identification information). In this case, after switching to layout display using an operation switch or the like described below, by further operating to change the layout information, the changed layout ID is transmitted from the operation device 50 (operation switch 56) to the vehicle V. At this time, as will be described in detail later, a composite image is generated in the vehicle V by combining an external image based on the layout information for the changed layout ID, and the composite image is displayed on the monitor 51.

[0029] The handle 53 is an operating part that is operated by an operator and is used to adjust the steering angle (steering amount) of the vehicle V. The accelerator pedal 54 and the brake pedal 55 are operating parts that are operated by an operator and are used to adjust the drive of the electric throttle V2 of the vehicle V and the operation of the electromagnetic brake device V3, respectively. The multiple operation switches 56 are used to allow the user to input setting information for performing, for example, "remote driving." For example, by appropriately operating the operation switches 56, the operator can switch the external image (composite image) of the vehicle V to a predetermined layout display, or switch the driving mode between an automatic driving mode and a remote driving mode.

[0030] <Vehicle remote control system functions> As shown in FIG. 4, from a functional standpoint, the vehicle control device 1 mainly comprises a memory unit 100 that stores various programs and various data, an external information acquisition unit 101, an absolute position calculation unit 102, a relative position calculation unit 103, a corrected position calculation unit 104, a reception determination unit 105, a position identification unit 106, an image processing unit 107, a communication unit 108, a vehicle control unit 109, a speed calculation unit 110, and a position information selection unit 111. These are composed of a CPU, ROM, RAM, HDD, communication interface, various programs, etc.

[0031] Explaining the operation device 50 from a functional perspective, its main components are a memory unit 500 that stores various programs and various data, a communication unit 501 that sends and receives various data to and from the vehicle control device 1, a screen display unit 502 that displays external images and vehicle information of the vehicle V on the monitor 51 and also displays content based on information about the current location of the vehicle V (e.g., vehicle navigation) on the navigation monitor 52, an operation data creation unit 503 that accepts user operation input and creates operation data, and a user notification unit 504 that notifies the user to the operator.

[0032] The functions of the vehicle control device 1 will be described in detail below. <<Main Features>> The external information acquisition unit 101 acquires “detection information of the external environment” around the vehicle V from the on-board sensor 10 , and also acquires “measurement information of the current position” of the vehicle V from the on-board locator 20 . In detail, as "detection information of the external environment," external image data of the surroundings of the vehicle V is obtained from the imaging device 11, detection result data of target objects around the vehicle V is obtained from the radar 12, and distance measurement data measuring the distance between the vehicle V and the target object is obtained from the lidar 13. In addition, as the "measurement information of the current position", GNSS information (GNSS correction information) is acquired from the GNSS receiver 21, and information on the angular velocity and acceleration of the vehicle V is acquired from the inertial measurement unit 22. The external information acquisition unit 101 may further acquire "vehicle information" of the vehicle V from the in-vehicle ECU 30. Examples of the "vehicle information" include "steering angle information" obtained from the steering wheel ECU 32, "throttle opening information" obtained from the accelerator ECU 33, and "brake depression amount information" obtained from the brake ECU 34.

[0033] The absolute position calculation unit 102 acquires the above-mentioned "GNSS information" necessary for point positioning, and calculates the "absolute position" of the vehicle V by point positioning. The "absolute position" of vehicle V is the three-dimensional position of vehicle V obtained by receiving GNSS radio waves from multiple satellites SA, measuring the distance between vehicle V and satellites SA located at known points, and solving a three-dimensional equation to determine unknown points from each measured distance (corresponding to GNSS information). As shown in Figure 5, the position accuracy of the "absolute position" is approximately ±10 m.

[0034] The relative position calculation unit 103 acquires the above-mentioned "GNSS correction information" necessary for relative positioning, corrects the "absolute position" by relative positioning, and calculates the "relative position" of the vehicle V. The "relative position" of vehicle V is the three-dimensional position of vehicle V that is determined by receiving GNSS radio waves at a reference station ST located at a known point, obtaining the distance with the smaller measurement error from reference station ST (the distance between each satellite SA and vehicle V), and then using each measured distance (corresponding to GNSS correction information). As shown in Figure 5, the position accuracy of the "relative position" is about ±40 cm, which is higher than the position accuracy of the absolute position. In Figure 5, "GNSS correction information" is expressed as "GNSS information + RTK information." The "relative position" can be calculated using either the RTK positioning method (interferometric positioning method) or the DGPS positioning method (relative positioning method). The reference station ST is, in principle, the reference station that is installed at the position closest to the vehicle V among a plurality of reference stations ST located at known points.

[0035] The corrected position calculation unit 104 acquires the above-mentioned "angular velocity and acceleration information" of the vehicle V, and calculates a "corrected absolute position" by correcting the absolute position of the vehicle V based on the "GNSS information" and the "acceleration and angular velocity information." The "corrected absolute position" of vehicle V is the three-dimensional position of vehicle V obtained by combining GNSS information with angular velocity and acceleration information of vehicle V (also called IMU information) to determine its position. The position accuracy of the "corrected absolute position" is higher than that of the absolute position.

[0036] In addition, the corrected position calculation unit 104 calculates a "corrected relative position" by correcting the relative position of the vehicle V based on the "GNSS correction information" and the "acceleration and angular velocity information." As shown in FIG. 5, the position accuracy of the "corrected relative position" is about ±5 cm, which is higher than the position accuracy of the absolute position and the relative position. In Figure 5, "acceleration and angular velocity information" is expressed as "IMU information" and described as "GNSS information + RTK information + IMU information."

[0037] The reception determination unit 105 determines whether or not GNSS information can be received in real time, and if it determines that GNSS information can be received in real time, it subsequently determines whether or not GNSS correction information can be received in real time. Specifically, the reception determination unit 105 assumes a case where there is an obstacle around the vehicle V and radio waves cannot be received from the artificial satellite SA, or where data cannot be sent or received with the reference station ST, and determines whether radio waves can be received from the artificial satellite SA and whether data can be sent or received with the reference station ST.

[0038] The position specifying unit 106 specifies the current position of the vehicle based on the determination result by the reception determining unit 105 . In detail, as shown in FIG. 6, when it is determined that GNSS information and GNSS correction information can be received in real time (S3 in FIG. 6), the position determination unit 106 determines the current position of the vehicle V using the "corrected relative position" with the highest position accuracy (S4 in FIG. 6). Furthermore, if the position determination unit 106 determines that it can receive GNSS information in real time but cannot receive GNSS correction information in real time (S3 in FIG. 6), it determines the current position of the vehicle V using a "corrected absolute position" with high position accuracy (S5 in FIG. 6). Furthermore, if the position determination unit 106 determines that it is unable to receive GNSS information and GNSS correction information in real time (S2 in Figure 6), it determines the current position of the vehicle V using an "estimated position" calculated based on the "GNSS information" and "acceleration and angular velocity information" received immediately before (S6 in Figure 6). The "estimated position" is the three-dimensional position of the vehicle V calculated based on the "GNSS information" received just before and the "acceleration and angular velocity information" from the past time when the "GNSS information" was received to the present time. The position accuracy of the "estimated position" is the same as that of the "absolute position."

[0039] The image processing unit 107 acquires external image data of the vehicle V from the plurality of image capturing devices 11a-11i, and creates a composite image (composite image data) by combining the external images based on predetermined layout information. By generating the above composite video and transmitting the generated composite video data to the operation device 50, the amount of data to be transmitted can be reduced (the number of communication lines can be reduced) compared to when multiple external video data are transmitted, thereby reducing the cost of data communication.

[0040] The communication unit 108 transmits and receives data between the vehicle control device 1 and the operation device 50 using the in-vehicle communication device 40 . Specifically, the communication unit 108 transmits to the operation device 50 the "detection information of the external environment" obtained by the external information acquisition unit 101 and the "current location information" identified by the location identification unit 106 as information necessary for "remote driving" of the vehicle V. The communication unit 108 may also transmit the “vehicle information” obtained by the external information acquisition unit 101 to the operation device 50. The communication unit 108 also receives driving operation information for the vehicle V from the operation device 50 that has accepted a user input from an operator.

[0041] The vehicle control unit 109 controls the integrated ECU 31 based on the "detection information of the external environment" obtained by the external information acquisition unit 101 and the "current location information" identified by the location identification unit 106, and performs "automatic driving" of the vehicle V. In addition, the vehicle control unit 109 controls the integrated ECU 31 based on the "driving operation information" of the vehicle V acquired from the operation device 50, and executes "remote driving" of the vehicle V. In addition, when performing "automatic driving" of the vehicle V, the vehicle control unit 109 may acquire "vehicle information" of the vehicle V from the on-board ECU 30 and further combine the "vehicle information" to control the integrated ECU 31.

[0042] <<Subfunction>> When identifying the current position of the vehicle V, "speed information" of the vehicle V may be further acquired in order to further improve the position accuracy. Specifically, the speed calculation unit 110 acquires the above-mentioned "information on angular velocity and acceleration" of the vehicle V, and calculates the "speed" of the vehicle V by integrating the acceleration and angular velocity. The corrected position calculation unit 104 then performs positioning by combining the "GNSS information," the "angular velocity and acceleration information," and the new "velocity information," thereby being able to calculate a "corrected absolute position" or a "corrected relative position" with higher positional accuracy. Note that the speed calculation unit 110 may calculate the "speed" of the vehicle V by processing the "GNSS information (GNSS correction information)" and the "acceleration and angular velocity information" using a Kalman filter. In this way, the "speed" can be calculated with higher accuracy. In addition, when determining the current position of the vehicle V, "steering angle information" of the vehicle V may be acquired and further combined for calculation. For example, by newly installing a steering angle sensor in the vehicle V, "steering angle information" can be acquired through the steering angle sensor. In addition, when obtaining "speed information" of the vehicle V, a wheel speed sensor may be newly installed in the vehicle V, and the "speed information" may be obtained through the wheel speed sensor.

[0043] Furthermore, when performing "automatic driving" or "remote driving" of vehicle V in an environment where there are no obstacles around vehicle V, it may be possible to prioritize reducing the amount of data communication and the costs associated with data communication over the positional accuracy of vehicle V's current position. Specifically, before S7 in FIG. 6, the position information selection unit 111 selects whether to use the "relative position" or the "corrected relative position" based on the detection information of the external environment obtained by the external information acquisition unit 101. More specifically, when the detected information of the external environment indicates that there are no moving objects or buildings (structures) around the vehicle V (for example, when traveling through wilderness or grassland), it is advisable to select "relative position" in order to reduce the amount of data communication by the vehicle control device 1. In other words, it is advisable not to acquire information on the angular velocity and acceleration of the vehicle V (measurement of the angular velocity and acceleration may be temporarily suspended). Alternatively, if the detected information of the external environment indicates that there are moving objects or buildings around the vehicle V (for example, when driving in an urban area), it is advisable to prioritize the position accuracy of the current position of the vehicle V and select "corrected relative position." In this way, the method for identifying the current position of the vehicle V can be appropriately selected in accordance with the external environment around the vehicle V. The selection by the location information selection unit 111 may be performed according to a selection condition set in advance or a selection condition set by the user.

[0044] With the above configuration, a vehicle remote control system can be realized that can flexibly measure the location information of vehicle V in response to the external environment around vehicle V (radio wave conditions and data communication conditions) and determine the current location with higher location accuracy.

[0045] <Vehicle remote control method> Next, the processing of the vehicle remote control program (vehicle remote control method) executed by the vehicle remote control system S will be described with reference to FIG. The above program in this embodiment is a program for realizing the above-mentioned external information acquisition unit 101, absolute position calculation unit 102, relative position calculation unit 103, corrected position calculation unit 104, reception judgment unit 105, position identification unit 106, video processing unit 107, communication unit 108, and vehicle control unit 109 as functional components of a vehicle control device 1 equipped with a memory unit 100, and the CPU of the vehicle control device 1 executes this vehicle maintenance assistance program. The above program is executed upon receiving an operation instruction from a user (specifically, a driver or an operator).

[0046] The vehicle remote control flow shown in FIG. 6 begins with step S1 in which the external information acquisition unit 101 starts acquiring "GNSS information" through the GNSS receiver 21. In practice, the external information acquisition unit 101 starts acquiring “GNSS information” and “GNSS correction information” through the GNSS receiver 21, and also starts acquiring “angular velocity and acceleration information” of the vehicle V through the inertial measurement unit 22. The external information acquisition unit 101 also starts acquiring “detection information of the external environment” around the vehicle V from the on-board sensor 10.

[0047] Next, in step S2, the reception determination unit 105 determines whether or not GNSS information can be received in real time. If it is determined that the "GNSS information" can be received (step S2: Yes), the process proceeds to step S3, where the reception determination unit 105 subsequently determines whether or not the "GNSS correction information" can be received in real time. If the reception determination unit 105 determines that the "GNSS correction information" can also be received (step S3: Yes), the process proceeds to step S4.

[0048] In step S4, the absolute position calculation unit 102, the relative position calculation unit 103, and the corrected position calculation unit 104 calculate the "corrected relative position (relative position)" of the vehicle V. Specifically, first, the absolute position calculation unit 102 acquires "GNSS information" and calculates the "absolute position" of the vehicle V by single point positioning. Then, the relative position calculation unit 103 acquires "GNSS correction information" and calculates the "relative position" of the vehicle V by relative positioning. Then, the corrected position calculation unit 104 acquires "angular velocity and acceleration information" and calculates a "corrected relative position" by correcting the relative position of the vehicle V. Then, the process proceeds to step S7. The "corrected relative position" is the position information with the highest positional accuracy.

[0049] In step S3 above, if the reception determination unit 105 determines that the "GNSS correction information" cannot be received either (step S3: No), the process proceeds to step S5. In step S5, the absolute position calculation unit 102 and the corrected position calculation unit 104 calculate the "absolute corrected position (absolute position)" of the vehicle V. Specifically, first, the absolute position calculation unit 102 acquires "GNSS information" and calculates the "absolute position" of the vehicle V by single point positioning. Then, the corrected position calculation unit 104 acquires "angular velocity and acceleration information" and calculates a "corrected absolute position" by correcting the absolute position of the vehicle V. Then, proceed to step S7.

[0050] In step S2, if the reception determination unit 105 determines that the "GNSS information" cannot be received (step S2: No), the process proceeds to step S6. In step S6, the position identification unit 106 calculates the "estimated position" of the vehicle V. Specifically, the position specifying unit 106 calculates an "estimated position" based on the "GNSS information" and "acceleration and angular velocity information" received immediately before (corresponding to an estimated position calculation unit). Then, proceed to step S7.

[0051] Next, in step S7, the position identifying unit 106 identifies the current position of the vehicle V using the calculated position information. If the process proceeds from step S4, the position specifying unit 106 specifies the current position of the vehicle V using the "corrected relative position" with the highest position accuracy. If the process proceeds from step S5, the position specifying unit 106 specifies the current position of the vehicle V using the "corrected absolute position." If the process proceeds from step S6, the position identifying unit 106 identifies the current position of the vehicle V using the "estimated position."

[0052] Next, in step S8, the communication unit 108 transmits to the operation device 50 the "current location information" identified by the location identification unit 106 and the "external environment detection information" obtained by the external information acquisition unit 101 as information necessary for "remote driving" of the vehicle V. The operation device 50 receives this information, displays the content based on the current position information and the detected information of the external environment on the monitor 51 and the navigation monitor 52, and notifies the operator as needed.

[0053] After going through steps S1 to S8, if a request to stop remote operation is finally received from the user (step S9: Yes), the process of FIG. 6 ends. On the other hand, if the request to stop the remote operation has not been received (step S9: No), the process returns to step S2. The above-described configuration of the vehicle remote control program makes it possible to measure the current position of the vehicle V more accurately in accordance with the external environment surrounding the vehicle V.

[0054] <Other embodiments> In the above embodiment, as shown in FIG. 2, the vehicle remote control system S includes a vehicle control device 1 and an on-board ECU 30, and newly adds a "remote driving function (vehicle control device 1)" to a vehicle V equipped with an "automatic driving function (on-board ECU 30)", but this can be modified without any particular limitations. For example, the vehicle control device 1 may also have the functions of the on-board ECU 30. That is, the vehicle remote control system S may be mainly composed of the vehicle control device 1 (including the functions of the on-board ECU 30), the on-board sensor 10, the on-board locator 20, the on-board communication device 40, and the operation device 50 (the on-board ECU 30 may be excluded from the configuration).

[0055] In the above embodiment, as shown in FIG. 2, the vehicle remote control system S includes a vehicle control device 1 and an on-board locator 20, but this is not particularly limited, and the vehicle control device 1 may also include the on-board locator 20. In this case, it is preferable that the in-vehicle locator 20 has an absolute position calculation unit 102 that calculates the absolute position of the vehicle V, a relative position calculation unit 103 that calculates the relative position of the vehicle V, and a corrected position calculation unit 104 that calculates the corrected relative position of the vehicle V. Then, it is preferable that the vehicle control device 1 acquires information on the absolute position, relative position, and corrected relative position of the vehicle V, respectively.

[0056] In the above embodiment, as shown in Figures 2 and 4, the corrected position calculation unit 104 acquires information on the acceleration and angular velocity of the vehicle V from the inertial measurement unit 22 and calculates the corrected relative position of the vehicle V, but this can be changed without any particular limitation. For example, when the vehicle control device 1 determines the current position of the vehicle V, it may not acquire (use) information on the acceleration and angular velocity of the vehicle V. In that case, the current position of the vehicle V will be determined using the "absolute position" and "relative position" of the vehicle V. Furthermore, for example, when the vehicle control device 1 identifies the current position of the vehicle V, it may not acquire (use) information on the acceleration and angular velocity of the vehicle V, but may acquire (use) information on the speed of the vehicle V.

[0057] In the above embodiment, as shown in FIG. 2, the photographing device 11 detects information about the external environment around the vehicle V as part of the on-board sensor 10, but is not particularly limited thereto, and the photographing device 11 may simply acquire external images around the vehicle V. That is, the radar 12 and the lidar 13 may function as the on-board sensor 10 to detect information about the external environment around the vehicle V.

[0058] In the above embodiment, a vehicle remote control program is stored in a recording medium readable by the vehicle control device 1, and processing is performed by the vehicle control device 1 reading and executing the program. Here, the recording medium readable by the vehicle control device 1 refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, a dedicated web application may be started using a terminal (mobile terminal) that serves as the vehicle control device 1, and the vehicle remote control program may be executed on a web browser.

[0059] In the above embodiment, the vehicle remote control system and the vehicle remote control method according to the present invention have been mainly described. However, the above embodiment is merely an example for facilitating understanding of the present invention, and does not limit the present invention. The present invention can be modified or improved without departing from the spirit thereof, and the present invention naturally includes equivalents thereof. [Explanation of symbols]

[0060] S Vehicle remote control system V vehicle V1 Electric Power Steering V1a Handle V2 Electric Throttle V2a accelerator pedal V3 Electromagnetic Brake Device V3a brake pedal 1 Vehicle control device 10. In-vehicle sensors 11 Imaging equipment 11a-11i 1st imaging device - 9th imaging device 12 Radar (millimeter wave radar) 12a-12d 1st radar - 4th radar 13 Rider 13a-13e 1st Rider-5th Rider 20 In-vehicle locator 21 GNSS receiver (RTK-GNSS receiver) 22 Inertial measurement unit (IMU) 30 Automotive ECU 31 Integrated ECU 32 Steering ECU 33 Accelerator ECU 34 Brake ECU 40 In-vehicle communication device 50 Operating device 51 Display monitor 52 Display Navigation Monitor 53 Handle 54 Accelerator pedal 55 Brake pedal 56 Operation switch 100 Storage section 101 External information acquisition department 102 Absolute position calculation unit 103 Relative position calculation unit 104 Correction position calculation section 105 Reception determination unit 106 Location identification part 107 Video Processing Unit 108 Communications Department (First Communications Department) 109 Vehicle control unit 110 Speed ​​calculation section 111 Location information selection section 500 storage section 501 Communications Department (Second Communications Department) 502 Screen display section 503 Operation Data Creation Department 504 User Notification Unit SA satellite ST reference station

Claims

1. An information processing device comprising a memory and a processor, The processor: an acquisition process for acquiring GNSS information required for independent positioning through a GNSS receiver mounted on the vehicle by executing a program stored in the memory; an absolute position calculation process for calculating an absolute position of the vehicle by the point positioning; a reception determination process for determining whether GNSS correction information required for relative positioning can be received from an external reference station; a relative position calculation process for correcting the absolute position by the relative positioning and calculating the relative position of the vehicle when it is determined that the GNSS correction information can be received; a speed calculation process for calculating the speed of the vehicle by integrating information on acceleration and angular velocity acquired from an inertial measurement unit mounted on the vehicle; a corrected position calculation process for calculating a corrected relative position by correcting the relative position of the vehicle based on the GNSS correction information and the information on the acceleration and angular velocity; a position information selection process that selects the relative position when no moving object is present around the vehicle, and selects the corrected relative position when a moving object is present around the vehicle, based on detection information of the external environment acquired from an on-board sensor mounted on the vehicle; and a position specifying process for specifying a current position of the vehicle using the relative position or the corrected relative position selected by the position information selection process.

2. The computer A process of acquiring GNSS information necessary for independent positioning through a GNSS receiver mounted on a vehicle; a process of calculating an absolute position of the vehicle by the point positioning; a reception determination process for determining whether GNSS correction information required for relative positioning can be received from an external reference station; If it is determined that the GNSS correction information can be received, a process of correcting the absolute position by the relative positioning and calculating a relative position of the vehicle; calculating a speed of the vehicle by integrating information on acceleration and angular velocity acquired from an inertial measurement unit mounted on the vehicle; A process of calculating a corrected relative position by correcting the relative position of the vehicle based on the GNSS correction information and the information on the acceleration and angular velocity; a process of selecting the relative position when no moving object is present around the vehicle, and selecting the corrected relative position when a moving object is present around the vehicle, based on detection information of an external environment acquired from an on-board sensor mounted on the vehicle; and specifying a current position of the vehicle using the relative position or the corrected relative position selected by the selecting process.

3. On the computer, A process of acquiring GNSS information necessary for independent positioning through a GNSS receiver mounted on a vehicle; a process of calculating an absolute position of the vehicle by the point positioning; a reception determination process for determining whether GNSS correction information required for relative positioning can be received from an external reference station; If it is determined that the GNSS correction information can be received, a process of correcting the absolute position by the relative positioning and calculating a relative position of the vehicle; calculating a speed of the vehicle by integrating information on acceleration and angular velocity acquired from an inertial measurement unit mounted on the vehicle; A process of calculating a corrected relative position by correcting the relative position of the vehicle based on the GNSS correction information and the information on the acceleration and angular velocity; a process of selecting the relative position when no moving object is present around the vehicle, and selecting the corrected relative position when a moving object is present around the vehicle, based on detection information of an external environment acquired from an on-board sensor mounted on the vehicle; and specifying a current position of the vehicle using the relative position or the corrected relative position selected by the selecting process.

Images