Vehicle control systems, remote control systems

Abstract

This technology provides capabilities to improve the safety of remotely controlled vehicles or other road users in their vicinity. [Solution] The on-board control device 30 comprises a processor 31, a memory 32, and an input / output circuit 33. The on-board control device 30 is connected to a wireless communication device 14 configured to communicate wirelessly with a remote control device, and the processor 31 acquires data indicating the communication status with the remote control device (e.g., round-trip delay time). The processor 31 dynamically adjusts the speed limit, which is the upper limit of the driving speed applicable to the remote control of the vehicle, according to the communication status with the remote control device. For example, if the round-trip delay time is greater than or equal to a predetermined safety threshold, the speed limit is determined based on the speed of the preceding vehicle.

Description

Technical Field

[0001] The present disclosure relates to a technology for remotely controlling the driving of a vehicle.

Background Art

[0002] Patent Document 1 discloses an in-vehicle device that controls the driving of a vehicle according to an instruction signal transmitted from a remote control device. The remote control device predicts the traffic situation around the vehicle based on the information received from the vehicle. Then, when the actual traffic situation received from the in-vehicle device does not match the predicted traffic situation, the remote control device instructs the vehicle to stop or slow down instead of performing the planned control (e.g., turning right at an intersection).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a system for remotely controlling a vehicle, communication delay between the vehicle and the remote control device can be a problem. Patent Document 1 only discloses a configuration for instructing the vehicle to slow down or stop when the predicted traffic situation does not match the actual traffic situation. In the technology for remotely controlling a vehicle, further measures against deterioration of the communication state between the vehicle and the remote control device are required.

[0005] One of the objects of the present disclosure is to provide a technology capable of improving the safety of a remotely controlled vehicle or other road users existing around it.

Means for Solving the Problems

[0006] The vehicle control device disclosed herein is a vehicle control device used in a vehicle configured to be remotely controllable, comprising: a communication unit (33) for communicating with one or more other devices mounted on the vehicle; and a processing unit (31) that performs processing related to vehicle driving control based on data received by the communication unit, wherein one or more other devices include a wireless communication device (14) configured to be wirelessly connected to a remote control device, which is an external device used to remotely control the vehicle, and the processing unit is configured to acquire the communication state between the remote control device and the wireless communication device based on data received by the communication unit from the wireless communication device, and to dynamically adjust a speed limit, which is an upper limit of the driving speed in remote control of the vehicle, according to the communication state.

[0007] Furthermore, the remote control system included in this disclosure is a remote control system that includes a vehicle control device (30) used in a vehicle configured to be remotely controllable, and an external system (5) located outside the vehicle for remotely controlling the vehicle, wherein the vehicle control device includes a communication unit (33) for communicating with a wireless communication device (14) configured to be wirelessly controllable with the external system, and a processing unit (31) that performs processing related to vehicle driving control based on data received by the communication unit, wherein the processing unit is configured to acquire the communication state between the external system and the wireless communication device based on data received by the communication unit from the wireless communication device, and to dynamically adjust the speed limit value, which is the upper limit value of the driving speed in remote control of the vehicle, according to the communication state, wherein the external system includes a communication device (51) for communicating with the vehicle control device, and a remote control device (53) that performs processing for remotely controlling the vehicle based on data received by the communication device, wherein the remote control device is configured to acquire the communication state in association with the vehicle's location information using the communication device, store the communication state data for each location in a predetermined memory (532), and determine the vehicle's driving path based on the communication state data for each location stored in the memory.

[0008] According to the technology described above, the upper limit of the vehicle's speed during remote control is determined according to the communication status between the remote control device and the wireless communication device installed in the vehicle. In other words, the vehicle's speed is limited by the communication status. By limiting the vehicle's speed, the risk of the vehicle traveling at excessively high speeds when the communication status deteriorates can be reduced. Therefore, the safety of the remotely controlled vehicle and other road users in its vicinity can be improved.

[0009] The reference numerals in parentheses in the claims indicate the correspondence with the specific means described later in the embodiments, and do not limit the technical scope of this disclosure. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram illustrating the vehicle management system. [Figure 2] This is a block diagram showing the configuration of an in-vehicle system. [Figure 3] This is a diagram showing an example of an environmental sensor. [Figure 4] This is a functional block diagram of the in-vehicle control system. [Figure 5] This block shows the configuration of a remote monitoring system. [Figure 6] This flowchart shows an example of the operation of an in-vehicle control device. [Figure 7] This is a flowchart of the AEB (Autonomous Emergency Braking) detection process. [Figure 8] This is a flowchart illustrating an example of the operation of a vehicle management system. [Figure 9] This is a control flowchart that shows how data items transmitted from the vehicle to the remote monitoring system are changed depending on the communication status. [Figure 10] This is a flowchart illustrating how alert control is executed depending on the communication status. [Figure 11] This is a flowchart of the route selection process in a remote control device. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described below with reference to the drawings. This disclosure is not limited to the embodiments described below. The configurations disclosed below may be implemented with various modifications without departing from the gist of the invention. Various modifications may be combined as appropriate, without causing any technical inconsistencies. This disclosure also includes configurations that are not explicitly stated, which are combinations of multiple modifications. In the following description, components having the same function may be denoted by the same reference numeral and their specific description may be omitted. Also, components having the same function may be denoted by the same or similar names and their specific description may be omitted. If only a part of a configuration is referred to, the description of the other parts may be applied elsewhere.

[0012] <Vehicle Management System> Figure 1 is a diagram showing an example of a schematic configuration of the vehicle management system Sys according to this disclosure. The vehicle management system Sys according to this disclosure includes an in-vehicle system 1 and a remote monitoring system 5, as shown in Figure 1. The vehicle management system Sys corresponds to the remote control system, and the remote monitoring system 5 corresponds to the external system.

[0013] The in-vehicle system 1 is installed in the host vehicle Hv. The remote monitoring system 5 recognizes the driving environment of the host vehicle Hv by communicating wirelessly with the in-vehicle system 1 and remotely controls the host vehicle Hv as needed. The remote monitoring system 5 and the in-vehicle system 1 are configured to communicate data with each other via a wide-area communication network. The wide-area communication network may include wireless sections provided by wireless base stations, wireless LAN routers (in other words, access points), or communication satellites. The host vehicle Hv is configured to be remotely controllable by the remote monitoring system 5, as described separately later. For example, the host vehicle Hv may be configured to enable remote substitute driving.

[0014] The host vehicle Hv is a vehicle equipped with the in-vehicle system 1. The host vehicle Hv is, for example, a privately-owned vehicle (so-called owner's car). As an example, the host vehicle Hv of the present embodiment has a plurality of driving operation members such as an accelerator pedal, a brake pedal, a steering wheel, and a shift lever. The driving operation members may also include operation members for operating secondary actuators such as a direction indicator switch (e.g., a lever), a light switch, a wiper switch, and a hazard lamp switch. The secondary actuator means an actuator that does not directly affect the movement of the host vehicle Hv but enables safe and legal driving. In contrast, an actuator that directly affects the movement of the host vehicle Hv, such as the motion actuator 20 described later, may also be referred to as a primary actuator in the present disclosure. Note that the host vehicle Hv may be configured to be able to receive acceleration and deceleration operations by operation members of types other than pedals, such as buttons and levers.

[0015] In the following description, the driver's seat in the host vehicle Hv is a seat suitable for operating the above-described driving operation members. In the following, the driver means a person sitting in the driver's seat. Also, the operator means a person having the authority to control the host vehicle Hv by remote operation from outside the host vehicle Hv. The operator can also be a person who operates the host vehicle Hv. Therefore, in a broad sense, the operator may also be regarded as a form of the driver. In the following, the term "operator" may be replaced with the term "off-vehicle operator", and the term "driver" may be replaced with the term "in-vehicle operator". Also, the term "mere operator" may be replaced with, for example, "off-vehicle driver" or "remote driver".

[0016] The host vehicle Hv may be a service vehicle such as a bus or a taxi. The host vehicle Hv may be a shuttle bus, a school bus, or a truck, etc. The host vehicle Hv may be a service vehicle without a driver in the vehicle, such as a driverless bus, an autonomous bus, or a robot bus. In other embodiments, the host vehicle Hv may be a vehicle without a driver's seat. Although only one host vehicle Hv is shown in FIG. 1, there may be multiple vehicles that can be remotely controlled by the remote monitoring system 5.

[0017] Also, hereinafter, the case where the host vehicle Hv is an electric vehicle will be exemplified, but the type of the host vehicle Hv is not limited to an electric vehicle. The host vehicle Hv may be an engine vehicle or a hybrid vehicle. Electric vehicles include fuel cell vehicles (FCV: Fuel Cell Vehicle). Diesel vehicles may be included in engine vehicles.

[0018] The host vehicle Hv may be configured such that the vehicle power is switched from off to on based on receiving a specific activation command from the remote monitoring system 5 in addition to the pressing operation of the power switch by the driver or administrator. The power switch is a switch for switching on and off the vehicle power, which is arranged inside the vehicle (for example, on the instrument panel).

[0019] <In-vehicle system> As shown in FIG. 2, the in-vehicle system 1 includes an environmental sensor 11, a vehicle state sensor 12, a locator 13, a wireless communication device 14, a driving operation sensor 15, a speaker 16, a display 17, an input device 18, and an outer notification device 19. The in-vehicle system 1 also includes a motion actuator 20 and an in-vehicle control device 30. Note that the in-vehicle system 1 may include various devices other than the configured shown in the figure, such as an audio device, an air conditioning device, a wiper system, a headlight system, etc. The term of device may include a system, a subsystem, a module, a sensor, etc.

[0020] The in-vehicle control device 30 is connected to the environmental sensor 11, vehicle status sensor 12, locator 13, wireless communication device 14, driving operation sensor 15, speaker 16, display 17, and external notification device 19 via the in-vehicle network 101, enabling mutual communication. In addition, the in-vehicle control device 30 is connected to the motion actuator 20 using a dedicated cable, without going through the in-vehicle network 101.

[0021] The in-vehicle network 101 is a communication network established within the vehicle. The in-vehicle network 101 may be a communication network conforming to any standard, such as Controller Area Network (CAN: registered trademark), Ethernet, or FLEXRAY (registered trademark). The network topology shown in Figure 2 is an example. The connection relationships between devices may be changed as appropriate. The on-board control device 30 may be connected to the motion actuator 20 via the in-vehicle network 101. The on-board control device 30 corresponds to the vehicle control device.

[0022] The environmental sensor 11 is a sensor that recognizes the surrounding environment of the host vehicle Hv. The environmental sensor 11 includes, for example, one or more of the following: a camera unit, millimeter-wave radar, LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging), and sonar. The detection range of the environmental sensor 11 is set to include the entire surroundings of the host vehicle Hv. The environmental sensor 11 detects moving and stationary objects within the detection range of the host vehicle Hv. Since the environmental sensor 11 is a sensor that monitors the area around the host vehicle Hv, it may also be called a surrounding monitoring sensor.

[0023] In this embodiment, the environmental sensor 11 includes a plurality of in-vehicle cameras, namely a front camera 111, a rear camera 112, a right camera 113, and a left camera 114, as shown in Figure 3. The front camera 111 is a camera that captures the area in front of the host vehicle Hv. The rear camera 112 is a camera that captures the area behind the host vehicle Hv. The right camera 113 and the left camera 114 are cameras that capture the areas to the sides of the host vehicle Hv, respectively. The environmental sensor 11 also includes one or more millimeter-wave radars 115. One or more millimeter-wave radars 115 may include a front radar that is a millimeter-wave radar that forms a detection range in front of the host vehicle Hv. Furthermore, the environmental sensor 11 may include one or more LiDARs 116. The environmental sensor 11 may also include one or more microphones 117 that capture sounds from outside the vehicle. The microphones 117 may be integrated into the in-vehicle cameras. The microphones 117 may also be called external microphones. Furthermore, the environmental sensor 11 may include an acoustic sensor that detects the external environment or a predetermined event (e.g., the approach of an emergency vehicle) based on sound.

[0024] The environmental sensor 11, such as a camera, may be configured to detect and identify objects registered as detection targets from captured video footage, for example, using a classifier that applies deep learning. Detection targets include moving objects such as pedestrians, cyclists, and other vehicles. Moving objects may be rephrased as road users (RUs) or traffic participants. The environmental sensor 11 may also be configured to detect specific features. Features that the environmental sensor 11 detects may include road markings, traffic signs, traffic lights, guardrails, road edges, and median strips. Road markings include lane markings, pedestrian crossings, stop lines, or traffic guides.

[0025] The environmental sensor 11 inputs data indicating the detection result to the on-board control unit 30 via the in-vehicle network 101. On-board cameras, such as the forward camera 111, provide captured video data to the wireless communication device 14, either through the on-board control unit 30 or without passing through the on-board control unit 30. Note that the function of recognizing objects based on the observation data generated by the environmental sensor 11 may be provided by other devices connected to the sensor body, such as the on-board control unit 30. Here, observation data refers to video data generated by cameras, 3D point cloud data generated by LiDAR, and reception result data of probe waves acquired by millimeter-wave radar / sonar.

[0026] The vehicle status sensor 12 is a sensor that detects information regarding the status of the host vehicle Hv. The vehicle status sensor 12 includes a vehicle speed sensor, steering angle sensor, acceleration sensor, yaw rate sensor, etc. The vehicle status sensor 12 may include a cabin sensor that generates information indicating the presence or absence of occupants or the number of occupants. The cabin sensor may include at least one of the following: DSM (Driver Status Monitor), seat belt sensor, seat occupancy sensor, in-cabin camera, and in-cabin microphone. The in-cabin camera is a camera that photographs the interior of the vehicle and may be an optical camera or an infrared camera. The in-cabin microphone is a microphone placed inside the vehicle to collect sounds inside the vehicle. The vehicle status sensor 12 inputs a signal indicating the detection result to the in-vehicle control device 30 via the in-vehicle network 101.

[0027] The locator 13 includes a GNSS (Global Navigation Satellite System) receiver and an inertial sensor, etc. The locator 13 periodically calculates the position of the host vehicle Hv by combining positioning signals received from multiple positioning satellites by the GNSS receiver, measurement results from the inertial sensor, and vehicle speed information output to the communication bus. The position information of the host vehicle Hv may be represented by three-dimensional coordinates of latitude, longitude, and altitude. The locator 13 may be configured to calculate the direction and speed of movement of the host vehicle Hv using some or all of the above information. The locator 13 provides the in-vehicle control device 30 with a dataset containing the position information of the host vehicle Hv based on the calculation results as locator information.

[0028] The locator 13 may further have a map database (hereinafter referred to as the map DB) that stores map data. The map DB mainly consists of a large-capacity storage medium that stores a large amount of 3D map data and 2D map data. The 3D map data is a so-called HD (High Definition) map and contains road information necessary for autonomous driving. The 2D map data may be a navigation map that shows the connections between roads. The locator 13 may be configured to read 3D map data of the area around its current location from the map DB and transmit it to the in-vehicle control device 30. The 2D map data may be used for route searching from the current location to the destination.

[0029] The map data stored locally in the host vehicle Hv may be updated by data received by the wireless communication device 14 from a map server or the like. The map DB may be a storage device for temporarily holding the map data received by the wireless communication device 14 from the map server until the data expires. The map DB may be provided separately from the locator 13.

[0030] The wireless communication device 14 is a device for the in-vehicle system 1 (mainly the in-vehicle control device 30) to perform wireless communication with an external device. The external device may include one or more of the following: the remote monitoring system 5, other vehicles, servers, traffic information centers, roadside units, and mobile devices (e.g., smartphones). The wireless communication device 14 is configured to perform cellular communication. Cellular communication refers to wireless communication compliant with LTE (Long Term Evolution), 4G, or 5G, etc. The wireless communication device 14 may also be configured to perform cellular V2X (PC5 / Uu) or DSRC (Dedicated Short Range Communications).

[0031] The driving operation sensor 15 is a sensor that detects the driver's driving operations. The driving operation sensor 15 includes at least one of an accelerator sensor, a brake sensor, and a steering angle sensor. The accelerator sensor may be a sensor that detects the amount (angle) of depression of the accelerator pedal as the accelerator operation amount (a so-called accelerator position sensor). The brake sensor may be a sensor that detects the amount (angle) of depression of the brake pedal as the brake operation amount (a so-called brake pedal position sensor). The steering angle sensor is a sensor that detects the rotation angle of the steering wheel (i.e., the steering angle). The steering angle corresponds to the steering operation amount. The driving operation sensor 15 provides driver operation information, including the accelerator operation amount, brake operation amount, and steering operation amount, to the on-board control device 30.

[0032] The manipulated quantity may be rephrased as a driving instruction value. Driver operation information may be understood as information including multiple driving instruction values ​​corresponding to accelerator operation, brake operation, and steering operation. In addition, driver operation information may also include information related to the operation of secondary actuators such as turn signals and headlights. The driving operation sensor 15 may include a switch / sensor for detecting the driver's operation on the secondary actuator.

[0033] The speaker 16 is a device that outputs sound. The term "sound" in this disclosure may include notification sounds, alarm sounds, buzzers, as well as voices and music. The speaker 16 outputs sound corresponding to the voice signal input from the in-vehicle control device 30, etc.

[0034] Display 17 is a display located inside the vehicle. Display 17 displays an image corresponding to a video signal input from an in-vehicle control device 30 or the like. If the host vehicle Hv is an owner's car, display 17 may be located on the instrument panel or the like. In other embodiments, if the host vehicle Hv is a robot bus or robot taxi, display 17 may be located in any position visible to passengers.

[0035] The input device 18 is a device for receiving operations from the occupants on in-vehicle devices such as the in-vehicle control device 30, air conditioning system, and audio system. The input device 18 may include steering wheel switches, switches located on the instrument panel, touch panels, etc. The input device 18 may also include switches for inputting start / end requests for remote control. Driving control components such as the accelerator pedal, brake pedal, and steering wheel may also be considered a type of input device 18.

[0036] The external notification device 19 is a device for displaying information to people present around the host vehicle Hv. People present around the host vehicle Hv include pedestrians, cyclists, kick boat users, and drivers of other vehicles. The external notification device 19 includes at least one of a brake light, a turn signal, and a hazard light. The external notification device 19 may include a display device (hereinafter also referred to as an outward-facing display) configured to present information to the outside of the vehicle. The outward-facing display may be a liquid crystal display or an organic EL display mounted on the exterior surface of the vehicle or inside the window frame with the screen facing outwards. The outward-facing display may also be a projector that projects an image onto the rear window. The outward-facing display may also be an LED array. The external notification device 19 operates according to instruction signals input from the in-vehicle control device 30. The external notification device 19 may be understood as a type of secondary actuator.

[0037] The motion actuator 20 is an actuator that generates power corresponding to acceleration, deceleration, and steering of the host vehicle Hv. The motion actuator 20 includes a powertrain that includes at least one of an engine and a drive motor. The motion actuator 20 related to vehicle propulsion may also be called a drive unit. The drive unit may be an engine, an EV system, or a hybrid system. The motion actuator 20 also includes a brake actuator and a steering actuator. The brake actuator may be a braking unit. The brake actuator may include at least one of a hydraulic brake and an electric regenerative brake. The steering actuator may be an EPS (Electric Power Steering) motor.

[0038] The motion actuator 20 operates based on control signals input from the on-board control device 30 and controls the motion of the host vehicle Hv. Other ECUs, such as a steering ECU that controls steering, a power unit control ECU that controls the drive source, and a brake ECU, may be interposed between the on-board control device 30 and the motion actuator 20.

[0039] The on-board control device 30 controls the motion actuator 20 based on the detection results of the environmental sensor 11 and driver operation information input from the driving operation sensor 15, thereby controlling the motion of the host vehicle Hv. The on-board control device 30 may be implemented using one or more computers.

[0040] The in-vehicle control device 30 includes a processor 31, memory 32, input / output circuits 33, and a bus connecting them. The processor 31 may be a CPU or the like. The processor 31 corresponds to the control unit. The memory 32 includes a rewritable volatile storage medium coupled with the processor 31. The memory 32 includes, for example, RAM (Random Access Memory). The memory 32 may include multiple types of non-transitional storage media. The memory 32 may include, for example, rewritable non-volatile memory such as flash memory as storage. The memory 32 stores a vehicle control program, which is a program executed by the processor 31. The execution of the vehicle control program by the processor 31 corresponds to the execution of part or all of the vehicle control method.

[0041] The input / output circuit 33 is hardware for the processor 31 to communicate with other devices constituting the in-vehicle system 1, such as the environmental sensor 11. The input / output circuit 33 may include a circuit that is compatible with the communication method with the other devices. The input / output circuit 33 may be a communication interface or an input / output port. The input / output circuit 33 corresponds to a communication unit or a communication circuit. The input / output circuit 33 may support any type of wired or wireless communication. Part or all of the wireless communication device 14 may be included in the input / output circuit 33. Digital data corresponding to signals received by the input / output circuit 33 may be temporarily stored in the memory 32.

[0042] The input / output circuit 33 receives information from multiple devices connected to the in-vehicle control device 30. Reception can be rephrased as acquisition. The input / output circuit 33 acquires sensor data (i.e., detection results) from the environmental sensor 11. The sensor data includes data about objects present around the vehicle, such as other moving objects, features, and obstacles. Other moving objects are moving objects other than the host vehicle Hv. The data for detected objects may include the position, speed (relative speed), and type or size of the detected object. Sensor data related to features may include lane mark data and road edge data.

[0043] Furthermore, the input / output circuit 33 acquires sensor data related to the motion state of the host vehicle Hv, such as the driving speed, acceleration, and yaw rate, from the vehicle state sensor 12. The input / output circuit 33 also acquires data indicating the conditions inside the vehicle, such as the presence or absence of a driver and the driver's level of awareness, from cabin sensors such as the DSM. In addition, the input / output circuit 33 acquires its own vehicle position data from the locator 13. The input / output circuit 33 may also acquire map data of the area around the host vehicle Hv by referring to a map database.

[0044] The input / output circuit 33 may, in cooperation with the wireless communication device 14, acquire data transmitted from an external device. For example, the input / output circuit 33 may acquire data transmitted from the remote monitoring system 5. The input / output circuit 33 also transmits data to the wireless communication device 14. The in-vehicle control device 30 sends and receives data with the remote monitoring system 5 using the wireless communication device 14.

[0045] The input / output circuit 33 acquires driver operation information from the driving operation sensor 15. Based on signals from the input device 18, the input / output circuit 33 also acquires information indicating the driver's operations on the in-vehicle system 1. For example, the input / output circuit 33 can receive instructions from the input device 18 regarding the start and end of remote control.

[0046] The various data acquired sequentially by the input / output circuit 33 are stored in a temporary storage medium such as memory 32 and used by the processor 31. Data acquired after a certain period of time may be discarded. Various data (information) may be acquired by generation, conversion, determination, or calculation based on signals received from other devices. The input / output circuit 33 or the processor 31 may have a function to generate other data based on raw data received from other devices. The processor 31 executes processing related to the driving control of the host vehicle Hv based on the data received by the input / output circuit 33. The processor 31 corresponds to the processing unit.

[0047] <On-board control device> The on-board control device 30 includes two operating modes: a remote driving mode and a manual driving mode. The manual driving mode is an operating mode in which the motion of the host vehicle Hv is controlled according to driver operations on the driving operation members. In the manual driving mode, the on-board control device 30 controls the motion actuators 20 according to driver operation information input from the driving operation sensor 15. That is, the vehicle control device 30 determines the control amount for each of the brake actuator, powertrain, and steering actuator according to the driver operation information, and outputs a control signal to the motion actuators 20 according to the determined control amount. Note that the manual driving mode may include a state in which driver assistance functions such as ACC, which will be described later, are enabled.

[0048] The remote driving mode is an operating mode in which the movement of the host vehicle Hv is controlled according to remote operation information received from the remote monitoring system 5 via the wireless communication device 14. The remote operation information is information indicating the operation content of a predetermined remote control component by the operator 6, as will be described separately later. The remote operation information may also include multiple driving instruction values ​​corresponding to accelerator operation amounts, brake operation amounts, and steering operation amounts. Furthermore, the remote operation information may also include operation instructions for secondary actuators such as the external notification device 19.

[0049] In the following, the remote driving mode may be abbreviated as RD (Remote Driving) mode. RD mode can be understood as the state in which the remote control function of the on-board control device 30 is enabled. In RD mode, the on-board control device 30 controls the motion actuators 20 according to the remote control information. That is, it determines the control amount for each of the brake actuator, powertrain, and steering actuator according to the remote control information, and outputs a control signal to the motion actuators 20 corresponding to the determined control amount.

[0050] Such an in-vehicle control device 30 includes the functional unit shown in Figure 4, which is realized by the processor 31 executing a vehicle control program. Specifically, the in-vehicle control device 30 has a mode manager F1, a manual operation response unit F2, a communication status acquisition unit F3, and a remote control unit F4.

[0051] Mode Manager F1 is a functional module that switches the operating mode of the on-board control unit 30. When the vehicle power is switched from off to on based on the power switch being pressed, the initial operating mode of the on-board control unit 30 may be manual driving mode. Also, when the vehicle power is switched from off to on by receiving a start command from the remote monitoring system 5 (i.e., remote operation), the initial operating mode of the on-board control unit 30 may be RD mode.

[0052] When the operating mode is manual driving mode and certain remote control initiation conditions are met, the mode manager F1 switches the operating mode of the on-board control device 30 from manual driving mode to RD mode. The switch to RD mode may be performed based on having previously sent a remote control request to the remote monitoring system 5 and receiving a positive response from the remote monitoring system 5.

[0053] The conditions for initiating remote control may include the driver exiting the vehicle, the driver's level of consciousness falling below a predetermined level, a request to initiate remote control being received from the driver or a passenger, or the reception of a specific intervention command from the remote monitoring system 5. The state in which the driver's level of consciousness is below a predetermined level may include a state in which the driver is unconscious due to a sudden illness (a so-called dead man's state). Furthermore, the state in which the driver's level of consciousness is below a predetermined level may also include a state in which the driver's drowsiness level is above a predetermined value. The driver's level of consciousness / drowsiness level may be estimated based on the driver's biometric information detected by a DSM or similar.

[0054] Mode manager F1 may periodically determine whether the remote control start condition has been met while manual driving mode is applied. The event registered as the remote control start condition may be referred to as a remote start trigger, etc.

[0055] Furthermore, when the operating mode is RD mode, if the remote control termination condition is met, the mode manager F1 switches the operating mode from RD mode to manual driving mode. Switching to manual driving mode may be performed based on the driver receiving a request to start manual driving via the display 17 or speaker 16 and an affirmative response from the driver. An affirmative response may be pressing the accelerator / brake pedal or gripping the steering wheel.

[0056] The remote control termination conditions may include detection of an override operation by the driver, acceptance of a handover request from the remote monitoring system 5, arrival at a specific location, or a communication failure continuing for a predetermined period of time. An override operation is an input to a driving control component provided in the host vehicle Hv. An override operation may be an accelerator operation, a brake operation, or a steering operation. The mode manager F1 may detect an override operation based on a signal input from the driving control sensor 15.

[0057] The mode manager F1 may periodically determine whether the remote control termination condition has been met while RD mode is applied. The event registered as the remote control termination condition may be referred to as a remote termination trigger, etc. For safety reasons, the vehicle control device 30 may be configured to switch from manual driving mode to RD mode and vice versa only when the vehicle is stopped.

[0058] The manual operation response unit F2 is a functional module that enables manual operation mode. The manual operation response unit F2 is activated when manual operation mode is entered and may be disabled during RD mode. However, the manual operation response unit F2 may remain active even during RD mode in order to respond quickly to driver override operations. During RD mode, the manual operation response unit F2 may remain in a standby state.

[0059] The communication status acquisition unit F3 is a functional module that acquires the communication status between the wireless communication device 14 and the remote monitoring system 5. The communication status between the wireless communication device 14 and the remote monitoring system 5 can be understood as the communication status between the wireless communication device 14 and the remote control device 53, or the communication status between the in-vehicle control device 30 and the remote control device 53.

[0060] The communication status includes round-trip delay time. Round-trip delay time is the time from when the wireless communication device 14 transmits data to the remote monitoring system 5 until a response is received from the remote monitoring system 5. Round-trip delay time may also be referred to as RTT (round-trip time), round-trip latency, or communication delay time. Round-trip delay time may be measured by the in-vehicle control device 30 sending a specific message for connectivity confirmation, such as a Ping command, to the remote monitoring system 5. Round-trip delay time may be measured periodically (for example, every 100 milliseconds).

[0061] In other embodiments, the communication status may include the uplink speed, downlink speed, etc. The communication status may also include the communication method used for data communication (e.g., 3G, LTE, 4G, 5G, 6G). Furthermore, the communication status may include an indicator showing the communication status with the radio base station providing the radio section. The radio base station may be an eNB (evolved Node B) or a gNB (next generation Node B), etc. The indicator showing the communication status with the radio base station may be RSRP, RSSI, or RSRQ, etc. RSRP, RSSI, and RSRQ are abbreviations for Reference Signal Received Power, Received Signal Strength Indicator, and Reference Signal Received Quality, respectively. These indicators may be calculated, for example, by the wireless communication device 14 based on the reception result of the reference signal (RS) transmitted from the radio base station.

[0062] Hereafter, information indicating the communication status with the remote monitoring system 5 will also be referred to as communication status information. As described above, the communication status information includes the round-trip delay time. The communication status information may also include at least one other indicator besides the round-trip delay time. The communication status information generated by the communication status acquisition unit F3 is stored in the memory 32 and referenced by the remote control unit F4. The communication status acquisition unit F3 may also directly provide the communication status information to the remote control unit F4.

[0063] The remote control unit F4 is configured to perform vehicle control based on remote operation information. This vehicle control may include not only the control of the motion actuator 20, but also the control of secondary actuators, the presentation of information to the driver, and the control of the air conditioning system.

[0064] Furthermore, the remote control unit F4 also performs processing associated with remote control. For example, the remote control unit F4 may transmit data indicating the vehicle's driving status to the remote monitoring system 5 as a driving status report. The driving status report includes at least one of the following: vehicle position, driving speed, direction of travel, yaw rate, battery remaining power, battery temperature, and the operating status of the secondary actuator. The vehicle position may be position coordinates or an ego lane number. The ego lane number roughly indicates the position of the host vehicle Hv in the road width direction (so-called lateral position). The ego lane number may be expressed as the number of other lanes present to the left (or right) of the host vehicle Hv.

[0065] The transmission of the driving status report may be performed periodically using the wireless communication device 14. For example, the remote control unit F4 may send a communication packet as a driving status report to the remote monitoring system 5 every 100 milliseconds or 200 milliseconds. The remote control unit F4 may perform periodic transmission of the driving status report at least while in RD mode. The remote control unit F4 may also start periodic transmission of the driving status report from a predetermined time before remote control begins. In other embodiments, the remote control unit F4 may perform periodic transmission of the driving status report even in manual driving mode.

[0066] Furthermore, the remote control unit F4 periodically or continuously transmits external video data and external audio data to the remote monitoring system 5, at least while in RD mode. The external video may include video from the front camera 111, rear camera 112, right camera 113, and left camera 114. The external audio data is audio data collected by the external microphone.

[0067] The remote control unit F4 may transmit video from each in-vehicle camera to the remote monitoring system 5 using any real-time video distribution technology. The transmission of camera video may also include the transmission of audio data. The remote control unit F4 may start transmitting camera video a predetermined time before remote control begins. In other embodiments, the remote control unit F4 may continue transmitting camera video to the remote monitoring system 5 even while in manual driving mode.

[0068] In other embodiments, if the host vehicle Hv is a service vehicle such as an unmanned bus, the remote control unit F4 may continue to transmit in-vehicle video data to the remote monitoring system 5 in addition to the external video data. The in-vehicle video is the video captured by the in-vehicle camera. The in-vehicle video data may include driver / passenger voice data captured by the in-vehicle microphone.

[0069] <Other functions of the in-vehicle control system> The processor 31 recognizes the driving environment of the host vehicle Hv based on sensor data received by the input / output circuit 33. The sensor data may include at least one of map data, detection results from the environmental sensor 11, and data received by the wireless communication device 14. The processor 31 may also recognize the driving environment of the host vehicle Hv by performing sensor fusion processing that integrates the detection results of multiple environmental sensors 11.

[0070] The driving environment includes information about the structure (in other words, configuration) of roads within a predetermined distance in front of the host vehicle Hv. The road structure may include the number of lanes, the position of the road edges, the road width, and the curvature of the road. The driving environment may include at least one of the following: the lane number, the weather, and the road surface condition. The processor 31 also acquires traffic rules around the host vehicle Hv based on the sensor data. Traffic rules may include speed limits and restrictions on lane changes.

[0071] The driving environment includes the position and type of objects present around the host vehicle Hv. The processor 31 may acquire the speed and direction of movement of other detected moving objects. Based on the various data acquired using the input / output circuit 33, the processor 31 recognizes the position and behavior of other vehicles. Thus, the driving environment includes at least the traffic conditions in the direction of movement of the host vehicle Hv (primarily forward). In other words, the processor 31 is configured to recognize the traffic conditions in the direction of movement of the host vehicle Hv using environmental sensors 11, etc.

[0072] The processor 31 may generate an environmental model, which is a three-dimensional model that reproduces (represents) the driving environment of the host vehicle Hv, as data indicating the driving environment. The environmental model may also be called a world model. The environmental model may be a model in which objects detected by the environmental sensor 11, such as moving objects such as other vehicles, lane markers, road edges, traffic lights, etc., are placed in a three-dimensional space based on the host vehicle Hv. The processor 31 may be understood as a configuration that manages data related to the driving environment. Data management here may include data acquisition (generation) and updating.

[0073] The vehicle control device 30 is equipped with an Advanced Emergency Braking (AEB) function. AEB is a control system that avoids collisions with other objects by using the brakes. The processor 31 performs calculations for AEB based on the above-mentioned driving environment data. The processor 31 calculates the time-to-collision (TTC) for each moving object detected in the direction of movement of the host vehicle Hv, in preparation for AEB. Specifically, the processor 31 determines whether there is a preceding vehicle and, if there is, calculates the TTC with the preceding vehicle. In simple terms, the TTC may be calculated by dividing the distance between vehicles by the relative speed. The TTC may also be calculated using other algorithms that take relative acceleration into consideration. Here, the preceding vehicle refers to the vehicle closest to the host vehicle Hv among other vehicles traveling in the same lane as the host vehicle Hv in front of the host vehicle Hv.

[0074] The processor 31 may calculate the TTC not only for the preceding vehicle but also for other objects. If there is an obstacle in the direction of movement of the host vehicle Hv, the processor 31 may also calculate the TTC for that obstacle. The processor 31 may also calculate the TTC for pedestrians or cyclists crossing the road.

[0075] The processor 31 executes brake control as an AEB when there is an object whose TTC is less than a predetermined AEB activation threshold (Ta). In AEB, brake control is executed after an alarm is issued. The on-board control device 30 may have two types of AEB activation thresholds registered: an alarm threshold and a brake threshold. The alarm threshold is the first AEB activation threshold for alarms. The brake threshold is the second AEB activation threshold for initiating braking. The alarm threshold may be set to a value that is a predetermined amount (e.g., 0.8 seconds) greater than the brake threshold. In the following, the AEB activation threshold refers to the brake threshold, but the following AEB activation threshold may also be the alarm threshold.

[0076] The processor 31 can perform AEB processing in the background regardless of the operating mode, that is, in both manual operation mode and RD mode. The AEB processing may include object detection, calculation of TTC, and comparison of TTC with the AEB activation threshold.

[0077] Furthermore, the processor 31 may provide safety functions or driver assistance functions other than AEB. For example, the processor 31 may be configured to perform Advanced Emergency Steering (AES), Adaptive Cruise Control (ACC), and Lane Centering (LC).

[0078] AES is a control system that avoids collisions with objects through steering. ACC is a driver assistance function that drives the host vehicle Hv to follow the preceding vehicle while maintaining its lane. LC is a driver assistance function that automatically controls the steering so that the host vehicle Hv stays in the center of the lane. LC may also be ALKS (Automated Lane Keeping System).

[0079] In manual driving mode, the processor 31 activates at least one of the ACC and LC functions in response to a driver's request to initiate assistance, and performs processing according to the activated driver assistance function. In RD mode, the processor 31 activates at least one of the ACC and LC functions in response to an operator's request to initiate assistance, and performs processing according to the activated driver assistance function.

[0080] The processor 31 may provide various types of information (hereinafter referred to as support information) to at least one of the driver and / or operator as driving assistance. Notification of support information may be implemented using at least one of the following: image display, illumination of indicator lamps, output of voice / notification sound, and application of vibration. Support information may be information relating to at least one of the following: objects that may be involved in collision, the need to change lanes, the path at intersections, speed limits, the status of traffic lights, obstacles, lane baselines, and traffic congestion.

[0081] <Remote monitoring system> This section describes the configuration of the remote monitoring system 5. As shown in Figure 5, the remote monitoring system 5 comprises a communication device 51, an operator HMI 52, and a remote control device 53. HMI in the component names is an abbreviation for Human Machine Interface. Note that the expression "host vehicle Hv" as the communication partner of the remote monitoring system 5 may be read as "in-vehicle system 1" or "in-vehicle control device 30".

[0082] The communication device 51 is communication equipment for performing data communication with the host vehicle Hv (in other words, the in-vehicle system 1) via a wide-area communication network. The communication device 51 may be connected to equipment that constitutes the wide-area communication network, for example, using optical fiber. The communication device 51 may also be configured to be directly wirelessly connected to the wireless communication device 14. The communication device 51 receives data transmitted from the host vehicle Hv and outputs it to the remote control device 53. For example, the communication device 51 outputs driving status reports and camera images from the host vehicle Hv to the remote control device 53. The communication device 51 also transmits data input from the remote control device 53 to the host vehicle Hv. For example, the communication device 51 transmits remote operation information input from the remote control device 53 to the host vehicle Hv.

[0083] The operator's HMI 52 is equipment for operator 6 to check the driving status of the host vehicle Hv and remotely control the host vehicle Hv. The operator's HMI 52 may also be used by operator 6 to communicate with the driver / occupant. The operator's HMI 52 includes a display 521, a microphone 522, a speaker 523, and an input device 524.

[0084] The display 521 displays images based on video signals input from the remote control device 53. For example, the display 521 may display video from each camera of the host vehicle Hv in separate screen windows. There may be one display 521 or multiple displays. The display 521 is located on the operator's desk 6. The display 521 may display a variety of information, such as support information, driving route, and objects detected by the environmental sensor 11.

[0085] Microphone 522 is a microphone for operator 6. Microphone 522 collects the voice spoken by operator 6, converts it into an electrical signal, and outputs it to the remote control device 53. Speaker 523 converts the voice signal input from the remote control device 53 into sound and outputs it. Speaker 523 and microphone 522 may be integrated into a headset for operator 6.

[0086] The input device 524 is a device for receiving instructions and operations from the operator 6 to the host vehicle Hv. The input device 524 can be a keyboard, a dedicated device including multiple switches, or a touch panel stacked on the display 521. The operator's HMI 52 may, for example, include a talk switch as the input device 524. The talk switch is equivalent to a switch that activates the microphone 522 and outputs the operator 6's voice from the speaker 16 of the host vehicle Hv.

[0087] Furthermore, the input device 524 includes driving control components for the operator 6 to remotely control the host vehicle Hv. That is, the input device 524 may include an accelerator pedal, brake pedal, steering wheel, shift lever, and light switch for the operator. The light switch may be a switch that switches the illumination state of at least one of the following: turn signals, headlights, front fog lights (so-called fog lamps), and emergency flashing indicator lights (so-called hazard lights). The concept of a switch is not limited to button-type switches, but also includes lever-type, dial-type, and touch-sensor-type switches. Some or all of the driving control components for the operator may be implemented using a touch panel, keyboard, or computer game / video game controller.

[0088] The remote control device 53 is primarily a computer, comprising a processor 531, memory 532, and input / output circuit 533. The processor 531 is hardware for arithmetic processing, coupled with the memory 532. The processor 531 includes at least one arithmetic core, such as a CPU. The processor 531 performs various processes by accessing the memory 532. The memory 532 includes volatile memory such as RAM. The memory 532 also includes a non-volatile storage medium such as flash memory.

[0089] Memory 532 stores a remote control program as a program executed by processor 531. Execution of the program by processor 531 is equivalent to the execution of a remote control method, which corresponds to the remote control program. The remote control method may include part or all of the vehicle control method. Execution of the remote control program by processor 531 may be equivalent to the execution of part or all of the vehicle control method. The vehicle control method may be implemented by at least one of processors 31 and 531.

[0090] The remote control device 53 is connected to the communication device 51 and the operator's HMI 52 in a communication-enabled manner. Data signals are input to the remote control device 53 from both the communication device 51 and the operator's HMI 52. The remote control device 53 also outputs control signals and data signals to the communication device 51 and the operator's HMI 52 as appropriate.

[0091] For example, the processor 531 displays external video, internal video, and various status information of the host vehicle Hv received by the communication device 51 on the display 521. This process corresponds to vehicle status notification processing, which notifies the operator 6 of the internal or external status of the host vehicle Hv. While the host vehicle Hv is in operation, the processor 531 may display video from each camera and various status information on the display 521. In addition, the processor 31 displays an image (e.g., text or an icon) on the display 521 indicating whether the host vehicle Hv is in RD mode.

[0092] The processor 531 may be configured to display external images on the display 521 when the conditions for starting remote control are met in the host vehicle Hv, or when the likelihood of such conditions being met increases. The processor 531 may be configured to change the display content of the display 521, such as the screen layout, depending on whether it is in RD mode or manual driving mode. For example, in manual driving mode, the processor 531 may mainly display a map image including the current position of the host vehicle Hv and the planned driving route, and may omit the display of camera images or display them in a relatively small size. On the other hand, in RD mode, the processor 31 may display camera images, such as the image from the front camera 111, in a relatively large size.

[0093] The remote control device 53 receives driving operations from the operator 6 to the host vehicle Hv via the input device 524. Specifically, the remote control device 53 acquires information such as accelerator operation amount, brake operation amount, shift position change, and operation instructions for secondary actuators based on input signals from the input device 524. This information pertains to remote operation information. The remote control device 53 transmits the remote operation information to the host vehicle Hv using the communication device 51.

[0094] <Operation of on-board control device> Here, an example of the operation of the in-vehicle control device 30 will be explained using the flowcharts shown in Figures 6 and 7. The series of processes shown in Figure 6 will also be referred to in this disclosure as the communication delay response process. The communication delay response process may include lowering the upper limit of the driving speed in remote control based on an increase in the communication delay time with the remote monitoring system 5. The communication delay response process includes, for example, S11 to S15 and starts from S11.

[0095] S11 may be executed periodically while the operating mode is RD mode. In other embodiments, S11 may also be executed periodically while the operating mode is manual driving mode. Furthermore, if the host vehicle Hv has an automatic driving mode as described later, the processor 31 may execute S11 periodically while the operating mode is automatic driving mode. The description of the processor 31 as the entity executing the following processes may be replaced with the in-vehicle control device 30 or the in-vehicle system 1, etc.

[0096] S11 is a step in which the processor 31 measures the communication status with the remote monitoring system 5 using the wireless communication device 14. The communication status here includes the round-trip delay time, as described above. The round-trip delay time may be measured by actually sending and receiving messages with the remote monitoring system 5. In the diagram, for the sake of simplicity, the round-trip delay time is represented as "Tc". Hereafter, the round-trip delay time may also be referred to as the communication delay time.

[0097] Processor 31 temporarily stores the acquired round-trip delay time in memory 32 along with a timestamp indicating the acquisition time. In S11, Processor 31 may also acquire other indicators of communication status, such as RSRP and downlink speed, in addition to the round-trip delay time (Tc). S12 is executed after S11.

[0098] S12 identifies the maximum delay time based on multiple measurements of round-trip delay time taken within a certain period of time in the past from the present. The maximum delay time is the maximum value (also called the worst value) of the round-trip delay time within a certain period of time in the past from the present. The maximum delay time can be understood as the maximum moving round-trip delay time taken within a certain period of time in the past from the present. This certain period of time may be 1 second, 2 seconds, or 5 seconds, 10 seconds, etc.

[0099] The processor 31 may determine the maximum delay time from the round-trip delay time history stored in memory 32. Alternatively, the processor 31 may be configured to update the value of the maximum delay time by comparing the maximum delay time determined in the previous flow with the round-trip delay time newly acquired in S11. An expiration date may be set for the measured value of the maximum delay time. "Tc_mx" shown in Figure 6, etc., represents the maximum delay time.

[0100] Once the processor 31 determines the maximum delay time, it executes S13. S13 is a step in which the processor 31 compares the maximum delay time with a predetermined threshold (hereinafter referred to as the safety threshold: Tα). The safety threshold is a time parameter and is set to 0.6 seconds, 1.0 seconds, or 1.4 seconds, etc. In Figure 6, etc., "Tα" refers to the safety threshold.

[0101] In this embodiment, as an example, the safety threshold is set to a value that is approximately a predetermined amount (e.g., 0.2 seconds) greater than the AEB activation threshold. The safety threshold may be set based on the AEB activation threshold. In one embodiment, the safety threshold may be set to the same value as the AEB activation threshold. Therefore, the term "safety threshold" may be replaced with "AEB activation threshold".

[0102] If the maximum delay time is less than the safety threshold (S13 YES), the processor 31 performs an AEB determination process in S14. On the other hand, if the maximum delay time is greater than or equal to the safety threshold (S13 NO), the speed limiting process is performed in S15.

[0103] The AEB determination process in S14 is a process that determines whether or not to start brake control as an AEB. The AEB determination process may include S21 to S25 as shown in Figure 7. S21 is a step in which information about objects that may collide with the host vehicle Hv is obtained based on sensor data input from the environmental sensor 11. The information about objects obtained in S21 may be information about objects that may collide with the host vehicle Hv (hereinafter also referred to as target objects).

[0104] The target object is an object in the direction of travel of the host vehicle Hv. In many cases, the target object is a preceding vehicle. The target object may include obstacles such as parked vehicles, fallen objects, barricades for lane closures, and construction signs. Other road users whose paths intersect with the host vehicle Hv may also be included as target objects. The target object may be identified based on the predicted behavior of other road users and the planned path of the host vehicle Hv.

[0105] When the host vehicle Hv is moving forward, the area in front of the host vehicle Hv is included in the direction of movement. When the host vehicle Hv is turning right, the area in front of and to the right of the host vehicle Hv is included in the direction of movement. When the host vehicle Hv is turning left, the area in front of and to the left of the host vehicle Hv is included in the direction of movement. When the host vehicle Hv is moving backward, the area behind the host vehicle Hv is included in the direction of movement. The area in front may include the area diagonally in front, and the area behind may include the area diagonally behind.

[0106] S22 is a step in which it is determined whether or not a target object exists as a result of S21. If no target object, i.e., an object that could potentially come into contact with the host vehicle Hv, is detected (S22 NO), the AEB determination process is terminated. On the other hand, if at least one target object is detected (S22 YES), the processor 31 executes S23 and S24 for each target object.

[0107] S23 is the step in which the processor 31 calculates the Time To Circumference (TTC) for the target object. The method for calculating the TTC is as described above. In S24, it is determined whether the calculated TTC is less than the AEB activation threshold (Ta). "Ta" in the diagram represents the AEB activation threshold. If the TTC is less than the AEB activation threshold, the processor 31 executes brake control as an AEB (also called emergency braking). For example, the processor 31 outputs an execution command for brake control as an AEB to a predetermined ECU or brake actuator.

[0108] Steps S23 to S24, from another perspective, correspond to the step of determining whether or not there is an object whose TTC is below the AEB activation threshold. If there is an object whose TTC is below the AEB activation threshold, the processor 31 decides to execute AEB and inputs a control signal to the brake actuator to generate a predetermined braking force.

[0109] The speed limiting process in S15 of Figure 6 is a process that limits the upper limit of the speed that operator 6 can input in RD mode. If the maximum delay time (Tc_mx) is greater than the safety threshold (or AEB activation threshold), the deceleration instruction by remote control may not be given in time, and the probability of AEB activation by the on-board control device 30 may increase. Frequent AEB activation can lead to a decrease in the ride comfort of the occupants or cause anxiety among other road users. The speed limiting process in S15 is a function introduced (in other words, created) with the above problem in mind. The vehicle control device 30 in this embodiment reduces the driving speed and the probability of AEB activation by lowering the upper limit of the driving speed based on the fact that the maximum delay time is greater than or equal to a predetermined value determined based on the AEB activation threshold. Note that the upper limit of the driving speed (before reduction) when there is no communication delay may be the speed limit set on the road or a pre-registered value.

[0110] The speed limiting process may be, for example, a process that determines the upper limit of the driving speed so that the TTC with respect to the reference vehicle is greater than the maximum delay time (Tc_mx). The reference vehicle here may basically be the preceding vehicle. If there is another vehicle that may cut in between the host vehicle Hv and the preceding vehicle (hereinafter referred to as the cutting vehicle), the reference vehicle may be the cutting vehicle. Of the cutting vehicle and the preceding vehicle, the one with the smaller TTC may be used as the reference vehicle.

[0111] Specifically, if D is the distance to the reference vehicle, Vp is the travel speed of the reference vehicle, and Vh is the travel speed of the host vehicle Hv, then the TTC for the reference vehicle is determined by the following formula (1). Note that ΔV is the approach speed of the host vehicle Hv to the reference vehicle, and is Vh - Vp.

[0112] [Mathematics 1] TTC = D / ΔV = D / (Vh - Vp) ... (1) The condition for Vh such that TTC is greater than Tc_mx is determined by substituting the above equation (1) into the left-hand side of TTC > Tc_mx and rearranging for Vh. That is, Vh such that TTC is greater than Tc_mx can be expressed by the following equation (2).

[0113] [Math 2] Vh <Vp+D / Tc_mx ···(2) If there is another vehicle to be used as the reference vehicle, the processor 31 sets the value determined by the right-hand side of equation (2) above to the speed limit in RD mode. The speed limit in RD mode is the upper limit of the driving speed that the on-board control device 30 can achieve (in other words, the allowable speed) in RD mode. If, in RD mode, the current driving speed has reached the speed limit, even if a further acceleration command is input as remote control information, the processor 31 may be configured to invalidate the command and maintain the speed limit. Furthermore, if the current driving speed exceeds the speed limit in RD mode, the processor 31 may be configured to decelerate to the speed limit at a predetermined deceleration rate.

[0114] In addition to the speed limit value obtained by the above formula (2) (hereinafter also referred to as the TTC-based speed limit value), the processor 31 may be configured to acquire a basic value of the speed limit value in RD mode that is determined according to the driving environment (hereinafter referred to as the basic speed limit). The basic speed limit may be set based on the speed limit set on the road. The basic speed limit may be the same as the speed limit, or it may be set to a value that is a predetermined amount (for example, 10 km / h) greater or less than the speed limit. The basic speed limit may be dynamically determined based on the speed limit of the road on which the host vehicle Hv is traveling.

[0115] The processor 31 may determine the speed limit by recognizing traffic signs using the forward camera 111. Alternatively, the processor 31 may obtain the speed limit based on map data. Furthermore, the processor 31 may determine the speed limit set on the road based on traffic information wirelessly distributed from the roadside unit. In other embodiments, the basic upper speed limit may be a constant value (e.g., 60 km / h).

[0116] The processor 31 may be configured to follow the smaller of the basic upper speed limit and the TTC-based speed limit. If there are no other vehicles that qualify as a reference vehicle, the processor 31 may control the driving speed in RD mode according to the basic upper speed limit. Thus, the processor 31 may be configured to determine the speed limit in RD mode based on the traffic conditions in the direction of travel of the host vehicle Hv (e.g., forward) and the communication status with the remote monitoring system 5.

[0117] Furthermore, the speed limit in RD mode may be determined based on parameters other than the TTC of the reference vehicle. In other embodiments, the processor 31 may be configured to use a value that is a predetermined amount (e.g., 10 km / h) smaller than the basic upper limit speed as the speed limit in RD mode if the delay time is greater than or equal to a predetermined value. In other words, the speed limit processing may be a process that sets a value that is a predetermined amount (e.g., 10 km / h) smaller than the basic upper limit speed determined by the speed limit as the speed limit in RD mode.

[0118] In yet another embodiment, the speed limiting process may be a process that sets the speed limit in RD mode to a value equal to or less than the reference vehicle's travel speed. The processor 31 may be configured to set the speed limit in RD mode to a value equal to or less than the reference vehicle's travel speed. Thus, the processor 31 may be configured to determine the speed limit in RD mode based on the reference vehicle's travel speed when the communication delay time is greater than or equal to a predetermined value.

[0119] With the above configuration, if the communication delay time between the host vehicle Hv and the remote monitoring system 5 exceeds a certain threshold, the upper limit of the driving speed in remote control may be lowered. Therefore, the risk of the TTC with a preceding vehicle falling below the AEB activation threshold can be reduced. In particular, if the first speed limit is determined to a value corresponding to the speed of the reference vehicle, or to a value that takes into account both the speed of the reference vehicle and the maximum delay time, the risk of AEB activating during remote control can be further reduced.

[0120] Furthermore, an upper limit is set on the driving speed during remote control. Therefore, even if the host vehicle Hv is illegally remotely controlled by a third party, the risk of the host vehicle Hv traveling at an excessively high speed can be reduced.

[0121] The above describes a mode in which speed limiting processing is performed based on the communication delay time exceeding a predetermined value, but it is not limited to this. The processor 31 may also be configured to perform speed limiting processing in response to other communication status parameters falling below or exceeding a predetermined value during RD mode. For example, it may be configured to perform speed limiting processing in response to the uplink communication speed, RSRP, RSSI, or RSRQ falling below a predetermined value. The processor 31 may also be configured to adjust the speed limit in RD mode according to the communication status between the wireless communication device 14 and the remote control device 53.

[0122] The remote control device 53 may also have a function to determine the speed limit in RD mode, taking into account the communication status. The remote control device 53 may measure the communication status with the on-board control device 30 (such as the maximum delay time) and determine the speed limit in RD mode based on the measurement results. The remote control device 53 may restrict the operation of operator 6 according to the determined speed limit. For example, if an acceleration command that exceeds the speed limit is input, the remote control device 53 may reject the command. In that case, the remote control device 53 may display an image on the display 521 indicating that the acceleration command has been rejected due to poor communication status.

[0123] <Redundancy of AEB function> The remote control device 53 may also be configured to perform AEB determination processing while remotely controlling the host vehicle Hv. That is, the AEB function may be installed not only in the in-vehicle control device 30 but also in the remote control device 53. The AEB activation threshold used by the remote control device 53 may be set to a different value in consideration of communication delay from the AEB activation threshold used by the in-vehicle control device 30.

[0124] For convenience, hereinafter, the AEB activation threshold used by the in-vehicle control device 30 may be referred to as the in-vehicle AEB threshold. Also, the AEB activation threshold used by the remote control device 53 is referred to as the remote AEB threshold. The remote AEB threshold may be the same value as the in-vehicle AEB threshold. The in-vehicle AEB threshold is stored in the memory 32, and the initial value of the remote AEB threshold is stored in the memory 532.

[0125] As described above, when the remote control device 53 has the AEB function, the remote control device 53 may be configured to dynamically change the remote AEB threshold according to the maximum delay time. FIG. 8 shows an operation example of such a vehicle management system Sys and includes S31 to S38.

[0126] Since S31 to S33 and S34 are the same as S11 to S13 and S15 described above, the description thereof is omitted. When the maximum delay time (Tc_mx) is less than the safety threshold in S33, the remote control device 53 or the in-vehicle control device 30 determines whether the in-vehicle AEB function is valid. The in-vehicle AEB function is the AEB function in the host vehicle Hv and may actually be the AEB function provided by the in-vehicle control device 30.

[0127] A case where the in-vehicle AEB function is not enabled may be when there is a malfunction in the AEB program in the in-vehicle control unit 30. A malfunction in the program may include cases where the program has been tampered with, is outdated, or has been modified in an updated program. Furthermore, a case where the in-vehicle AEB function is not enabled may include a malfunction in the AEB-related hardware in the in-vehicle control unit 30. A malfunction in the AEB-related hardware may include communication errors (e.g., disconnection) or memory sticking. Whether or not the in-vehicle AEB function is enabled may be determined by executing a predetermined test sequence. The test sequence is a process that verifies the operation of programs, etc. A case where the in-vehicle AEB function is not enabled may also include the case where the in-vehicle system 1 does not have an AEB function in the first place. If the host vehicle Hv is a vehicle without an AEB function, S33 may always be judged negative.

[0128] If it is determined that the in-vehicle AEB function is enabled (S35 YES), the processor 31 executes S36. S36 may be the same process as S14 described above. On the other hand, if it is determined that the in-vehicle AEB function is not enabled (S35 NO), the remote control device 53 changes the setting value of the remote AEB threshold (Ta_rm) in S37. "Ta_rm" in the figure represents the remote AEB threshold. Note that the description of the remote control device 53 as the entity executing the step may be appropriately replaced with the processor 531.

[0129] For example, if the initial value of the remote AEB threshold is Ta_i, in S37 the remote control device 53 sets the remote AEB threshold (Ta_rm) to the value obtained by adding the maximum delay time (Tc_mx) to the initial value (Ta_i). Specifically, if the initial value of the remote AEB threshold is 0.7 seconds and the maximum delay time is 0.4 seconds, the remote control device 53 sets the remote AEB threshold to 1.1 seconds. The remote control device 53 may obtain the maximum delay time by wireless communication with the in-vehicle control device 30 and then execute S37.

[0130] Then, the remote control device 53 uses the remote AEB threshold determined in S37 to perform remote AEB determination processing in S38. The remote AEB determination processing may be a process in which the AEB activation threshold (Ta) used in the determination processing in S24 of Figure 7 is replaced with a remote AEB threshold (Ta_rm). The remote control device 53 may calculate the TCC for each target object based on the sensor data received from the host vehicle Hv and compare it with the remote AEB threshold.

[0131] If the calculated TCC is smaller than the remote AEB threshold, the remote control device 53 may send an AEB execution command to the host vehicle Hv via wireless communication. The AEB execution command is a signal that instructs the execution of emergency braking. If the on-board control device 30 receives an AEB execution command from the remote monitoring system 5 via the wireless communication device 14, it may promptly execute brake control as an AEB.

[0132] With the above configuration, the remote control device 53 can determine whether AEB is necessary at an earlier timing, taking into account the communication delay time. This reduces the risk of the host vehicle Hv coming into contact with another object because the brake instruction from AEB is not delivered in time due to communication delay.

[0133] Furthermore, the remote control device 53 may execute S37 and S38 periodically, not only when it is determined that the on-board AEB function is not enabled. In other words, even when the on-board AEB function is enabled, the remote control device 53 may be configured to execute S37 and S38 and determine whether AEB is necessary at a timing that takes the delay time into consideration. In addition, the remote control device 53 may be configured to identify the communication status (e.g., maximum delay time) between the remote monitoring system 5 and the host vehicle Hv, either together with or in place of the on-board control device 30. The remote control device 53 may execute S37 and S38 based on the maximum delay time it has independently determined.

[0134] <Introduction of control to reduce the amount of data transmitted> As shown in Figure 9, the processor 31 may be configured to reduce the number of camera images transmitted to the remote monitoring system 5 if the maximum delay time (Tc_mx) is greater than or equal to a predetermined limit threshold (Tβ) (S41 YES) (S42). In the figure, "Tβ" represents the limit threshold.

[0135] For example, if the maximum delay time is less than the limit threshold, the processor 31 transmits all camera images to the remote monitoring system 5. On the other hand, if the maximum delay time is greater than or equal to the limit threshold, the processor 31 transmits only the camera images related to the direction of movement to the remote monitoring system 5. Camera images related to the direction of movement may be, for example, the images from the front camera 111. In another example, if the maximum delay time is less than the limit threshold, the processor 31 transmits both the front camera images and the in-vehicle camera images to the remote monitoring system 5, while if the maximum delay time is greater than or equal to the limit threshold, it transmits only the images from the front camera 111 to the remote monitoring system 5. By excluding the in-vehicle camera images from uploading when the maximum delay time is greater than or equal to the limit threshold, congestion can be alleviated and the transmission delay of the front camera images can be reduced.

[0136] By changing the combination of camera images to be uploaded according to the communication status in this way, the transmission delay of camera images with high importance or priority can be reduced. In addition to changing the combination of camera images to be transmitted, the processor 31 may also be configured to transmit the images after reducing the image quality. The reduction in image quality may be a reduction in frame rate, a reduction in resolution, or both. Furthermore, the processor 31 may reduce the number of items transmitted in the driving status report if the maximum delay time exceeds a limit threshold. As described above, the setting change to reduce the amount of data transmitted to the remote monitoring system 5 is also called transmission data reduction control. When the communication status deteriorates, transmission data reduction control is executed, which can further reduce the amount of data transmitted per unit time.

[0137] Furthermore, by limiting the data that the in-vehicle control device 30 transmits to the remote monitoring system 5, the information presented to the operator 6 is also limited. This modified control can be understood as a control that narrows the information presented to the operator 6 in response to a deterioration in the communication state between the in-vehicle control device 30 and the remote monitoring system 5 in one situation.

[0138] The processor 31 may also be configured to perform data reduction control in response to other communication status parameters falling below or exceeding predetermined values ​​during RD mode. For example, it may be configured to perform data reduction control in response to uplink speed, RSRP, RSSI, or RSRQ falling below predetermined values. The processor 31 may determine whether the communication status with the remote monitoring system 5 satisfies specific conditions. These specific conditions correspond to a deterioration in the communication status, or in other words, a deviation of communication quality from a predetermined acceptable range. These specific conditions may include, for example, a delay time exceeding a predetermined value, or an uplink speed, RSRP, RSSI, or RSRQ falling below predetermined values. The processor 31 may also be configured to perform data reduction control in response to a determination that the communication status with the remote monitoring system 5 satisfies specific conditions.

[0139] <Control of external notification device> The processor 31 may be configured to perform alert control using the external notification device 19 when the maximum delay time (Tc_mx) is greater than or equal to a predetermined external notification threshold (Tγ) (S51 YES) (S52). In the figure, "Tγ" represents the external notification threshold.

[0140] Warning control is a control that prompts road users (hereinafter referred to as surrounding RUs) present around the host vehicle Hv to pay attention to the behavior of the host vehicle Hv. Warning control may include, for example, at least one of the following: (i) flashing multiple types of lighting devices in a predetermined pattern, (ii) displaying warning information on an outward-facing display, or (iii) outputting an audio message to the outside indicating that a communication problem has occurred. (i) may, for example, involve repeatedly flashing the brake lights along with the hazard lights in a predetermined pattern. The warning information may be text or an image that prompts surrounding RUs to pay attention to the host vehicle Hv. More specifically, the warning information may be text or an image indicating that a communication problem has occurred in the host vehicle Hv.

[0141] In a configuration where the processor 31 implements the above-described warning control in response to a deterioration in communication status with the remote monitoring system 5, measures such as surrounding RUs increasing their distance from the host vehicle Hv may be taken. As a result, the safety of the occupants of the host vehicle Hv or other road users may be enhanced.

[0142] Furthermore, the processor 31 may be configured to execute a warning control when, in RD mode, communication status parameters other than the delay time fall below a predetermined value or exceed a predetermined value. For example, it may be configured to execute a warning control when the uplink communication speed, downlink communication speed, RSRP, RSSI, or RSRQ falls below a predetermined value. The processor 31 may determine whether the communication status with the remote monitoring system 5 satisfies specific conditions. These specific conditions may be that the delay time exceeds a predetermined value, or that the uplink communication speed, downlink communication speed, RSRP, RSSI, or RSRQ falls below a predetermined value. The processor 31 may be configured to execute a warning control when it determines that the communication status with the remote monitoring system 5 satisfies the specific conditions.

[0143] <Routing in remote control devices> The remote control unit 53 / operator 6 will drive the host vehicle Hv toward the destination during remote control. In this case, the remote control unit 53 may determine the driving route of the host vehicle Hv considering the communication quality for each road section. For example, the remote control unit 53 may select a route from the remote control start point of the host vehicle Hv to the destination, based on a pre-generated radio wave map, that minimizes the total delay time at each point, or a route in which there are no sections with delay times exceeding a predetermined value, or a route with as few sections as possible.

[0144] The radio wave map is data that shows the estimated communication quality, or in other words, the communication status, for each location / road section. The radio wave map may be generated and updated based on the past communication history with the host vehicle Hv for each location. To generate the radio wave map, the remote control device 53 may be configured to periodically perform communication with the host vehicle Hv to measure the communication status, and to store the host vehicle Hv's location information and the measurement results in the memory 532. In addition, to generate the radio wave map, the remote control device 53 may be configured to periodically perform wireless communication with the on-board control device 30 in the background even while the host vehicle Hv is being driven manually.

[0145] Furthermore, if there are multiple vehicles managed by the remote monitoring system 5 (hereinafter referred to as remote vehicles), a radio wave map showing the communication quality for each location / road section may be generated from the communication history with multiple remote vehicles. With the above configuration, the remote control device 53 can guide the host vehicle Hv to a road section suitable for remote control based on the communication results with other remote vehicles. In addition, by using the radio wave map, the remote control device 53 can acquire in advance the communication quality along the route that the host vehicle Hv is scheduled to take. As a result, before the communication quality actually deteriorates, the remote control device 53 can instruct the on-board control device 30 to take measures such as reducing speed, issuing warnings, or reducing transmitted data.

[0146] Figure 11 is a flowchart showing an example of the operation of the remote control device 53 corresponding to the above technical concept, and includes steps S61, S62, and S63. S61 is a step of calculating one or more route candidates from the remote control start point to the destination based on map data showing the road connection relationships. The remote control start point is the point where remote control of the host vehicle Hv is started. In practice, the remote control start point may be the position coordinates of the host vehicle Hv at the time when the remote control start conditions are met. The destination may be a point set in advance by the driver or a point set by the operator 6. The destination may be the driver's home or a nearby parking lot. If the host vehicle Hv is a service vehicle, the destination may be a business office, terminal, or station registered as a home. The destination may be the next bus stop. The destination may be changed dynamically.

[0147] Once the calculation of route candidates is complete, the remote control device 53 extracts the congested sections included in the route candidates based on the radio wave map in S62. The congested sections may be sections where the communication delay time is greater than or equal to a predetermined value, or sections where the uplink communication speed, etc., is less than a predetermined value.

[0148] In S63, the remote control device 53 determines the final travel route from among the route candidates, taking into consideration communication quality. For example, the remote control device 53 sets the route candidate with the smallest worst-case delay time from among multiple route candidates as the travel route. If there is only one route candidate, that route candidate may be set as the travel route.

[0149] Note that steps S61 to S63 may be integrated. The remote control device 53 may set the cost for each link based on the degree of communication congestion for each road section and determine the travel route using a predetermined route search method (e.g., Dijkstra's algorithm). Here, "link" refers to a road section on the map data.

[0150] Once the travel route is determined, the remote control device 53 assists the operator 6 in remotely controlling the host vehicle Hv to travel along the travel route. The remote control device 53 may also activate driver assistance functions in the host vehicle Hv (e.g., automatic lane changes) to ensure that the host vehicle Hv travels along the travel route.

[0151] According to the above configuration, the host vehicle Hv will prioritize driving on road sections where communication delays are less likely to occur. Therefore, the problem of communication delays in remote control can be mitigated.

[0152] <Supplementary information on in-vehicle control devices> The on-board control device 30 may be implemented in the form of an automated driving system (ADS) or an automated driving device in one aspect. The on-board control device 30 may have an automated driving mode in addition to a manual driving mode and an RD mode as operating modes. The automated driving mode may be abbreviated as AD mode from now on. AD stands for automated / autonomous driving.

[0153] The mode manager F1 switches the operating mode of the on-board control device 30 based on at least one of the following: the recognition result of the driving environment, the driver's operation, the operator 6's operation, or the determination of the remote control device. In other words, the mode manager F1 switches from manual driving mode or RD mode to AD mode, and switches from AD mode to manual driving mode or RD mode, based on any of the above inputs.

[0154] While in AD mode, the on-board control unit 30 automatically performs dynamic driving tasks to ensure that the host vehicle Hv travels along the planned route to the destination. In other words, while in AD mode, the on-board control unit 30 performs controls to enable the vehicle to drive autonomously, such as recognizing the driving environment, planning the driving trajectory, and motion control. Motion control includes speed adjustment through acceleration and deceleration, and steering control.

[0155] In AD mode, the on-board control device 30 can create a control plan for the host vehicle Hv using the detection results of the environmental sensor 11 and map data stored in the map database. The control plan may include information such as acceleration / deceleration schedule information, steering angle control schedule information, information on lane change locations, and timing for notifying occupants of lane changes or stops. Note that some of the processing related to the creation of the control plan may be handled by the remote monitoring system 5.

[0156] In AD mode, the on-board control unit 30 determines the control amount for each actuator based on the generated control plan and outputs a control signal to each actuator. The on-board control unit 30 also controls the on / off status of turn signals, headlights, hazard lights, etc., based on the created control plan and the external environment. The on-board control unit 30 may transmit data indicating the timing of lane changes, stopping, starting, and right / left turns to the remote control unit 53. The remote control unit 53 may display the automated driving plan for the host vehicle Hv on the display 521 and present it to the operator 6.

[0157] The on-board control unit 30 may send a remote control start request to the remote control unit 53 when certain conditions are met during AD mode. For example, the on-board control unit 30 may send a remote control start request to the remote control unit 53 when it detects or predicts exit from the Operational Design Domain (ODD). The on-board control unit 30 may also send a remote control start request to the remote control unit 53 when it detects a malfunction in a system component related to the execution or maintenance of the automated driving function.

[0158] If the on-board control device 30 detects a system abnormality or system limit, it may first output a TOR (takeover request) to the driver, and then, in response to the driver's failure to respond, send a TOR to the operator 6. A TOR is a request from the on-board control device 30 to the driver or operator 6 to take over the driving operations. A TOR can also be referred to as a handover request, intervention request, or handover request.

[0159] As described above, this disclosure may be applied to autonomously driven vehicles. The in-vehicle control device 30 does not need to consist of a single computer; it may be divided among multiple computers (e.g., ECUs: Electronic Control Units). The in-vehicle control device 30 described above may be implemented using multiple ECUs. In that case, the functional arrangement in the multiple ECUs may be designed as appropriate.

[0160] Furthermore, the on-board control device 30 does not necessarily need to have a manual driving mode. The on-board control device 30 may not have a manual driving mode and may only have AD mode and RD mode. In other words, the host vehicle Hv may be an unmanned bus or the like. Also, the host vehicle Hv and the on-board control device 30 may be a vehicle / device dedicated to remote control that does not have a manual driving mode or an automatic driving mode.

[0161] <Additional remark (1)> This disclosure includes the following technical concepts. It also includes methods, programs, and storage media on which programs are recorded that correspond to these technical concepts.

[0162] [Technical thought 1] A vehicle control device used in a vehicle configured to be remotely controllable, A communication unit (33) for communicating with one or more other devices mounted on the vehicle, The system includes a processing unit (31) that performs processing related to vehicle driving control based on data received by the communication unit, The one or more other devices include a wireless communication device (14) configured to wirelessly communicate with a remote control device, which is an external device used to remotely control the vehicle. The aforementioned processing unit, Based on the data received by the communication unit from the wireless communication device, the communication unit acquires the communication status between the remote control device and the wireless communication device. A vehicle control device configured to dynamically adjust a speed limit, which is an upper limit of the driving speed in the remote control of the vehicle, according to the aforementioned communication state.

[0163] [Technical thought 2] The aforementioned communication unit is configured to communicate with multiple other devices, The aforementioned plurality of other devices include, in addition to the wireless communication device, an environmental sensor (11) that outputs data indicating traffic conditions in the direction of the vehicle's movement. The aforementioned processing unit, The communication unit acquires the traffic conditions in the direction of movement based on the data received from the environmental sensor. A vehicle control device according to technical concept 1, configured to determine the speed limit in the remote control based on the communication state and the traffic conditions in the direction of movement.

[0164] [Technical thought 3] The aforementioned communication unit is configured to communicate with multiple other devices, The plurality of other devices include, in addition to the wireless communication device, an environmental sensor that outputs data indicating the speed of other moving objects in the direction of movement of the vehicle. The aforementioned processing unit, Based on the data received by the communication unit from the environmental sensor, the communication unit obtains the moving speed of the other moving body in the direction of movement. A vehicle control device according to technical concept 1, configured to determine the speed limit in the remote control based on the communication state and the speed of the other moving body.

[0165] [Technical thought 4] The aforementioned communication unit is configured to communicate with multiple other devices, The plurality of other devices include, in addition to the wireless communication device, an environmental sensor that outputs data indicating the relative speed of other moving objects in the direction of movement of the vehicle. The aforementioned communication state is the communication delay time. The aforementioned processing unit, Based on the data received by the communication unit from the environmental sensor, the relative velocity of the moving object in the direction of movement is obtained. Based on the relative velocity, the collision margin time with the moving body is calculated. A vehicle control device according to technical concept 1, configured to determine the speed limit such that the collision margin time is greater than the communication delay time.

[0166] [Technical thought 5] The aforementioned processing unit, A vehicle control device according to technical concept 4, configured to adjust the speed limit so that the collision margin time is greater than the communication delay time when the communication delay time is greater than or equal to a predetermined value.

[0167] [Technical Thought 6] The processing unit is configured to periodically or continuously transmit data relating to the vehicle's driving status to the remote control device using the wireless communication device while the remote control is enabled. The aforementioned processing unit further, Determine whether the aforementioned communication state satisfies specific conditions, A vehicle control device according to any one of technical concepts 1 to 5, configured to perform control to reduce the amount of data per unit time transmitted to the remote control device upon determining that the communication state satisfies the specific conditions.

[0168] [Technical Thought 7] The aforementioned communication unit is configured to communicate with multiple other devices, The aforementioned multiple other devices include, in addition to the wireless communication device, a plurality of in-vehicle cameras, The aforementioned processing unit, If the aforementioned communication state satisfies the aforementioned specific conditions, the video from two or more in-vehicle cameras is transmitted to the remote control device. A vehicle control device according to technical concept 6, configured to reduce the number of in-vehicle cameras that transmit video to the remote control device compared to when the communication state does not satisfy the specific conditions.

[0169] [Technical Thought 8] The aforementioned communication unit is configured to communicate with multiple other devices, The aforementioned plurality of other devices include, in addition to the wireless communication device, an external notification device (19), The aforementioned processing unit, Determine whether the aforementioned communication state satisfies specific conditions, A vehicle control device according to any one of technical concepts 1 to 7, which, upon determining that the communication state satisfies the specific conditions, is configured to use the external notification device to alert other road users present in the vicinity of the vehicle.

[0170] [Technical Thought 9] The aforementioned external notification device includes a display device configured to display information outside the vehicle, The aforementioned processing unit, Determine whether the aforementioned communication state satisfies specific conditions, A vehicle control device according to technical concept 8, configured to display information to prompt attention using the display device after determining that the communication state satisfies the specific conditions.

[0171] Furthermore, the adjustment of the speed limit in remote control is not a mandatory element in Technical Concept 6, and the following technical concepts may also be included in this disclosure. According to the following technical concepts, even when communication conditions are poor, the upload speed of relatively important data can be more easily maintained. As a result, the function of the remote control device to check the status of the controlled vehicle can be maintained, and safety in remote control can be enhanced.

[0172] [Technical thought] A vehicle control device used in a vehicle configured to be remotely controllable, A communication unit (33) for communicating with one or more other devices mounted on the vehicle, The system includes a processing unit (31) that performs processing related to vehicle driving control based on data received by the communication unit, The one or more other devices include a wireless communication device (14) configured to wirelessly communicate with a remote control device, which is an external device used to remotely control the vehicle. The processing unit is configured to periodically or continuously transmit data relating to the vehicle's driving status to the remote control device using the wireless communication device while the remote control is enabled. The aforementioned processing unit further, Based on the data input from the communication unit, the communication status between the remote control device and the wireless communication device is acquired. Determine whether the aforementioned communication state satisfies specific conditions, A vehicle control device configured to perform control to reduce the amount of data per unit time transmitted to the remote control device upon determining that the communication state satisfies the specific conditions.

[0173] Furthermore, the adjustment of the speed limit in remote control is not a mandatory element in Technical Concept 8, and the following technical concepts may also be included in this disclosure. According to the following technical concept, if the communication conditions are not good, an external notification device is used to alert nearby road users. This can enhance the safety of the remotely controlled vehicle or road users in the vicinity.

[0174] [Technical thought] A vehicle control device used in a vehicle configured to be remotely controllable, A communication unit (33) for communicating with multiple other devices mounted on the vehicle, The system includes a processing unit (31) that performs processing related to vehicle driving control based on data received by the communication unit, The aforementioned plurality of other devices include a wireless communication device (14) configured to be wirelessly connected to a remote control device, which is an external device used to remotely control the vehicle, and an external notification device (19), The aforementioned processing unit, Based on the data input from the communication unit, the communication status between the remote control device and the wireless communication device is acquired. Determine whether the aforementioned communication state satisfies specific conditions, A vehicle control device configured to alert other road users in the vicinity of the vehicle using the external notification device, upon determining that the communication state satisfies the specific conditions.

[0175] <Additional remarks (2)> The various flowcharts shown in this disclosure are all examples, and the number of steps constituting the flowchart and the order of execution of processes can be changed as appropriate. The controls shown in each flowchart may be combined and executed in parallel to the extent that they do not contradict each other. Expressions such as acquisition, determination, detection, generation, and calculation may be used interchangeably. When a device acquires certain data, it may also include the device generating that data based on signals input from other devices / sensors.

[0176] The devices, systems, and methods described in this disclosure may be implemented by a dedicated computer comprising a processor programmed to perform one or more functions embodied by a computer program. The devices and methods described in this disclosure may be implemented using dedicated hardware logic circuits. The devices and methods described in this disclosure may be implemented by one or more dedicated computers comprising a combination of a processor that executes a computer program and one or more hardware logic circuits. The processor may be any arithmetic core, such as a CPU, MPU, GPU, or DFP (Data Flow Processor). Some or all of the functions of the vehicle management system may be implemented as hardware. Some or all of the functions of the in-vehicle control device or remote control device may be implemented using a system-on-chip (SoC), integrated circuit (IC), or field-programmable gate array (FPGA).

[0177] A computer program includes instructions that are executed by a computer. A computer program may be stored on a computer-readable, non-transitory tangible storage medium. The storage medium for a computer program may be a variety of media, such as an HDD (Hard-disk drive), an SSD (Solid State Drive), or flash memory. [Explanation of Symbols]

[0178] 11 Environmental sensor, 14 Wireless communication device, 19 External notification device, 30 In-vehicle control device (vehicle control device), 31 Processor (processing unit), 32 Memory, 33 Input / output circuit (communication unit), 5 Remote management system (external system), 53 Remote control device, 531 Processor, 532 Memory, 521 Display

Claims

1. A vehicle control device used in a vehicle configured to be remotely controllable, A communication unit (33) for communicating with one or more other devices mounted on the vehicle, The system includes a processing unit (31) that executes processing related to vehicle driving control based on data received by the communication unit, The one or more other devices include a wireless communication device (14) configured to be wirelessly connected to a remote control device, which is an external device used to remotely control the vehicle. The aforementioned processing unit, Based on the data received by the communication unit from the wireless communication device, the communication unit acquires the communication status between the remote control device and the wireless communication device. A vehicle control device configured to dynamically adjust a speed limit, which is an upper limit of the driving speed in the remote control of the vehicle, according to the aforementioned communication state.

2. The aforementioned communication unit is configured to communicate with multiple other devices, The aforementioned plurality of other devices include, in addition to the wireless communication device, an environmental sensor (11) that outputs data indicating traffic conditions in the direction of the vehicle's movement. The aforementioned processing unit, The communication unit acquires the traffic conditions in the direction of movement based on the data received from the environmental sensor. The vehicle control device according to claim 1, configured to determine the speed limit in the remote control based on the communication state and the traffic conditions in the direction of movement.

3. The aforementioned communication unit is configured to communicate with multiple other devices, The plurality of other devices include, in addition to the wireless communication device, an environmental sensor that outputs data indicating the speed of other moving objects in the direction of movement of the vehicle. The aforementioned processing unit, Based on the data received by the communication unit from the environmental sensor, the communication unit obtains the moving speed of the other moving body in the direction of movement. The vehicle control device according to claim 1, configured to determine the speed limit in the remote control based on the communication state and the speed of the other moving body.

4. The aforementioned communication unit is configured to communicate with multiple other devices, The plurality of other devices include, in addition to the wireless communication device, an environmental sensor that outputs data indicating the relative speed of other moving objects in the direction of movement of the vehicle. The aforementioned communication state is the communication delay time. The aforementioned processing unit, Based on the data received by the communication unit from the environmental sensor, the relative velocity of the moving object in the direction of movement is obtained. Based on the relative velocity, the collision margin time with the moving body is calculated. The vehicle control device according to claim 1, configured to determine the speed limit value such that the collision margin time is greater than the communication delay time.

5. The aforementioned processing unit, The vehicle control device according to claim 4, configured to adjust the speed limit so that the collision margin is greater than the communication delay time when the communication delay time is greater than or equal to a predetermined value.

6. The processing unit is configured to periodically or continuously transmit data relating to the vehicle's driving status to the remote control device using the wireless communication device while the remote control is enabled. The aforementioned processing unit further, Determine whether the aforementioned communication state satisfies specific conditions, The vehicle control device according to claim 1, configured to perform control to reduce the amount of data per unit time transmitted to the remote control device when it is determined that the communication state satisfies the specific conditions.

7. The aforementioned communication unit is configured to communicate with multiple other devices, The aforementioned multiple other devices include, in addition to the wireless communication device, a plurality of in-vehicle cameras, The aforementioned processing unit, If the aforementioned communication state satisfies the aforementioned specific conditions, the video from two or more in-vehicle cameras is transmitted to the remote control device. The vehicle control device according to claim 6, wherein, if the communication state does not satisfy the specified conditions, the number of in-vehicle cameras that transmit video to the remote control device is reduced compared to when the communication state does not satisfy the specified conditions.

8. The aforementioned communication unit is configured to communicate with multiple other devices, The aforementioned plurality of other devices include, in addition to the wireless communication device, an external notification device (19), The aforementioned processing unit, Determine whether the aforementioned communication state satisfies specific conditions, The vehicle control device according to claim 1, wherein, upon determining that the communication state satisfies the specific conditions, the external notification device is configured to alert other road users present in the vicinity of the vehicle.

9. The aforementioned external notification device includes a display device configured to display information outside the vehicle, The aforementioned processing unit, Determine whether the aforementioned communication state satisfies specific conditions, The vehicle control device according to claim 8, configured to display information to prompt attention using the display device when it is determined that the communication state satisfies the specific conditions.

10. A vehicle control device (30) used in a vehicle configured to be remotely controllable, A remote control system including an external system (5) located outside the vehicle for remotely controlling the vehicle, The aforementioned vehicle control device is A communication unit (33) for communicating with a wireless communication device (14) configured to be able to communicate wirelessly with the external system, The system includes a processing unit (31) that executes processing related to vehicle driving control based on data received by the communication unit, The aforementioned processing unit, Based on the data received by the communication unit from the wireless communication device, the communication unit acquires the communication status between the external system and the wireless communication device. The system is configured to dynamically adjust the speed limit, which is the upper limit of the driving speed in the remote control of the vehicle, according to the aforementioned communication state. The aforementioned external system A communication device (51) for communicating with the vehicle control device, The communication device includes a remote control device (53) that performs processing for remotely controlling the vehicle based on the data received by the communication device, The remote control device is Using the aforementioned communication device, the communication status is acquired in association with the vehicle's location information, The data of the communication status for each location is stored in a predetermined memory (532), A remote control system configured to determine the vehicle's travel route based on the communication status data for each location stored in the memory.

Images