Remote support method and remote support device

The remote support system addresses communication delays by determining vehicle presence and setting an expanded viewing range, enabling safe and timely lane changes in autonomous vehicles.

JP2024167838APending Publication Date: 2024-12-04NISSAN MOTOR CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023084193
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-12-04

AI Technical Summary

Technical Problem

Existing remote support systems for autonomous vehicles fail to provide appropriate lane change assistance when communication delays occur, leading to incomplete or unsafe driving scenarios.

Method used

A remote support method and device that determines the presence of vehicles in the target vehicle's path, calculates delay times, and sets an expanded viewing range to ensure safe lane changes by displaying relevant vehicles to the operator, even in the presence of communication delays.

Benefits of technology

Enables effective remote support for lane changes and navigation through complex driving scenarios by accurately identifying and highlighting critical vehicles, ensuring safe and timely operator intervention.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167838000001_ABST
    Figure 2024167838000001_ABST
Patent Text Reader

Abstract

To provide a remote support method and a remote support device that can execute appropriate remote support even if delays occur in communication.SOLUTION: When a remote support device 20 located outside a subject vehicle 10 supports the travel control of the subject vehicle 10 entering another lane L2 different from a travel lane L1 on which the subject 10 is traveling, and when it is determined that there is another vehicle traveling toward an entry range in the another lane L2 into which the subject vehicle 10 is entering, the remote support device 20 external to the subject vehicle 10 calculates the delay time from the time when the surroundings of the subject vehicle 10 are captured by an image capturing device 11 of the subject vehicle 10 until an image of the surroundings of the subject vehicle 10 is displayed to an operator operating the remote support device 20, sets a gazing range A3, which is extended by a length based on the traveling speed of the other vehicle and the delay time, on the opposite side of the traveling direction of the other vehicle, as an entry range, and outputs an indication to a display unit 22 indicating to the operator that the other vehicle traveling in the gazing range A3 is the vehicle to be gazed at.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a remote assistance method and a remote assistance device. [Background technology]

[0002] A remote assistance system is known that limits the remote assistance that an operator can provide to an autonomous vehicle based on at least one of the state of communication between an autonomous driving assistance center and the autonomous vehicle and the state of the environment around the autonomous vehicle (Patent Document 1). As an example, in this remote assistance system, if there is a delay in the display of video of the left front or right front, assistance for changing lanes to the left or right cannot be provided. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-68132 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technology, even in a driving scene where a lane change is necessary, if there is a delay in communication between the vehicle and the remote assistance device, lane change assistance cannot be performed, resulting in the problem that remote assistance appropriate for the driving scene cannot be performed when a delay in communication occurs.

[0005] An object of the present invention is to provide a remote assistance method and a remote assistance device that are capable of providing appropriate remote assistance even if a delay occurs in communication. [Means for solving the problem]

[0006] In the present invention, when a remote assistance device outside the target vehicle assists in the driving control of a target vehicle entering a lane different from the lane in which the target vehicle is currently traveling, when it determines that there is another vehicle traveling toward the entry range in the other lane into which the target vehicle is entering, it calculates the delay time from when the surroundings of the target vehicle are imaged by the target vehicle's imaging device to when an image of the surroundings of the target vehicle is displayed to the operator operating the remote assistance device, and sets a gaze range that expands the entry range by a length based on the other vehicle's traveling speed and delay time on the opposite side of the other vehicle's direction of travel, and outputs a display on the display device to indicate to the operator that the other vehicle traveling in the gaze range is the gazed vehicle, thereby solving the above problem. Effect of the Invention

[0007] According to the present invention, appropriate remote support can be performed even if a delay occurs in communication. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of an embodiment of a remote assistance system according to the present invention. [Diagram 2] 2 is a plan view showing an example of a driving scene in which driving assistance is performed by the remote assistance system shown in FIG. 1. [Diagram 3] 3 is a diagram showing an example of an image displayed to an operator in the driving scene of FIG. 2. [Figure 4] FIG. 3 is a plan view showing an example of driving assistance executed in the driving scene of FIG. 2 (part 1). [Diagram 5] FIG. 3 is a plan view showing an example of driving assistance executed in the driving scene of FIG. 2 (part 2). [Figure 6] 1. FIG. 4 is a plan view showing another example of a driving scene in which driving assistance is performed by the remote assistance system shown in FIG. [Figure 7] 7 is a diagram showing an example of an image displayed to an operator in the driving scene of FIG. 6. [Figure 8] 1. FIG. 4 is a plan view showing still another example of a driving scene in which driving assistance is performed by the remote assistance system shown in FIG. [Figure 9] 9 is a diagram showing an example of an image displayed to an operator in the driving scene of FIG. 8. [Figure 10] 1. FIG. 4 is a plan view showing yet another example of a driving scene in which driving assistance is performed by the remote assistance system shown in FIG. [Figure 11] 11 is a diagram showing an example of an image displayed to an operator in the driving scene of FIG. 10. FIG. [Figure 12] 2 is a flowchart showing an example of a processing procedure in the remote support system of FIG. 1. [Figure 13] 13 is a flowchart showing another example of the processing procedure in the remote support system of FIG. 1 (part 1). [Figure 14] 13 is a flowchart showing another example of the processing procedure in the remote support system of FIG. 1 (part 2). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the following description is based on the assumption that the vehicle is driven on the left side of the road in a country that has a left-hand traffic law. In a country that has a right-hand traffic law, the left and right in the following description should be interpreted as symmetrical.

[0010] [Configuration of remote support system] 1 is a block diagram showing a remote assistance system 1 according to the present invention. The remote assistance system 1 is a system for an operator outside the vehicle to assist in driving the vehicle. For example, when a vehicle running under autonomous driving control is no longer able to continue running under autonomous driving control, an operator outside the vehicle remotely operates the vehicle using the remote assistance system 1. The outside of the vehicle is not particularly limited as long as it is a location away from the body of the vehicle, and may be a remote location such as a management center that centrally manages the running of the vehicle.

[0011] Autonomous driving control refers to autonomously controlling the driving operations of a vehicle using a vehicle controller, and the driving operations include all driving operations such as acceleration, deceleration, starting, stopping, and steering. Furthermore, autonomously controlling driving operations refers to the vehicle controller controlling the driving operations using the vehicle's devices. The controller controls these driving operations within a predetermined range, and driving operations that are not controlled by the controller are manually operated by the driver. When the vehicle travels under manual driving by the driver without autonomous driving control, the controller does not autonomously control the driving operations, and the vehicle's driving operations are controlled by the driver's operation.

[0012] As shown in Fig. 1, the remote support system 1 includes a vehicle 10 and a remote support device 20. The vehicle 10 and the remote support device 20 exchange information with each other via a server (not shown). The vehicle 10, the remote support device 20, and the server are connected to a network for exchanging information with each other. The network refers to a telecommunication network such as the Internet, and wired communication, wireless communication, mobile communication, etc. can be used for communication.

[0013] The vehicle 10 includes an imaging device 11, a first controller 12, and a vehicle control device 13. These devices are connected by a controller area network (CAN) or other in-vehicle LAN, and can exchange information with each other. Hereinafter, the vehicle 10 is also referred to as a target vehicle 10.

[0014] The imaging device 11 is a device for capturing images of objects around the vehicle 10, and is, for example, a camera equipped with an imaging element such as a CCD, an infrared camera, or the like. In order to reduce blind spots where an object cannot be captured, a plurality of imaging devices 11 are arranged on the front grill, side mirrors, rear bumper, etc. of the vehicle 10. Images captured by the imaging devices 11 are acquired by the first controller 12 as necessary. The vehicle 10 may also be equipped with a ranging device (such as a millimeter wave radar or a LiDAR (light detection and ranging) unit) (not shown).

[0015] The objects detected by the imaging device 11 and the distance measuring device include lane boundaries, center lines, road markings, medians, guard rails, curbs, road signs, traffic lights, and crosswalks. The objects also include obstacles that may affect the travel of the vehicle 10, such as automobiles (other vehicles) other than the vehicle 10, motorcycles, bicycles, and pedestrians. The detection results of the imaging device 11 and the distance measuring device are acquired by the first controller 12 as necessary. The first controller 12 recognizes the objects and the travel environment around the vehicle 10 from the acquired detection results.

[0016] The first controller 12 is a device that controls the traveling of the vehicle 10 by controlling and cooperating with each device of the vehicle 10, and drives the vehicle 10 to a set destination. The first controller 12 is, for example, a computer, and includes a CPU (Central Processing Unit) that is a processor, a ROM (Read Only Memory) that stores programs, and a RAM (Random Access Memory) that functions as an accessible storage device. The CPU is an operating circuit that executes the programs stored in the ROM and realizes the functions of the first controller 12.

[0017] The vehicle control device 13 is an on-board computer such as an electronic control unit, and electronically controls on-board devices that govern the driving of the vehicle 10. The vehicle control device 13 autonomously controls the operation of the drive device and steering device of the vehicle 10 in response to a control signal input from the first controller 12.

[0018] The remote assistance device 20 is a device used by an operator outside the vehicle 10 to assist in driving the vehicle 10, and includes a second controller 21, a display device 22, and an input device 23. These devices are connected by a LAN or the like so as to be able to transmit and receive information to and from each other.

[0019] The second controller 21 is a device for providing necessary information to the operator by controlling and cooperating with each device of the remote support device 20, and for supporting the operator in controlling the driving of the vehicle 10. The driving control of the vehicle 10 includes the above-mentioned autonomous driving control, and driving control other than the autonomous driving control, such as control for stopping the vehicle 10, control for evacuating the vehicle 10, and control by manual driving by the driver. The second controller 21 is, for example, a computer, and includes a CPU, a ROM, and a RAM, similar to the first controller 12. The CPU is an operating circuit for executing a program stored in the ROM and realizing the functions of the second controller 21.

[0020] The display device 22 is a device for displaying images to the operator who operates the remote support device 20, and examples of such devices include a liquid crystal display and a projector. Images include moving images and still images. The operator will be described in detail later.

[0021] The input device 23 is a device for an operator of the remote assistance device 20 to input assistance information necessary for driving assistance of the vehicle 10. The input device 23 is composed of a touch panel, a keyboard, a steering wheel, an accelerator pedal, a brake pedal, etc. Details of the assistance information will be described later.

[0022] [Function block functions] The second controller 21 has a remote support function of transmitting support information to the first controller 12 to cause the vehicle 10 to travel autonomously. A program stored in the ROM of the second controller 21 is a program for realizing the remote support function, and the remote support function is realized by the CPU of the second controller 21 executing the program stored in the ROM. Fig. 1 shows, for convenience, a determination unit 31, a calculation unit 32, a setting unit 33, a detection unit 34, a display unit 35, and a support unit 36, which are functional blocks that realize the remote support function.

[0023] The functions of each functional block shown in FIG. 1 will be described below with reference to FIG.

[0024] Fig. 2 is a plan view showing an example of a driving scene in which the second controller 21 executes driving control of the vehicle 10 by the remote assistance function. The road shown in Fig. 2 is a two-lane road having lanes L1 and L2. A vehicle traveling on lane L1 travels from the left side to the right side of the drawing, and a vehicle traveling on lane L2 travels from the right side to the left side of the drawing. Hereinafter, lane L1 is also referred to as a driving lane, and lane L2 is also referred to as another lane or an adjacent lane.

[0025] In the driving scene shown in FIG. 2, the vehicle 10 is driving at position P1 in the driving lane L1 toward the destination Px on the right side of the drawing. In addition, another vehicle V1 is driving at position Q1 in the other lane L2, and another vehicle V2 is parked at position R1 in the driving lane L1. To continue driving under the autonomous driving control, the vehicle 10 needs to enter the other lane L2 to avoid the other vehicle V2, and then return to the driving lane L1 after avoiding the other vehicle V2. However, if the detection device (image capture device, distance measurement device, etc.) of the vehicle 10 cannot recognize an obstacle in the other lane L2 due to a blind spot in the detection range, the first controller 12 cannot determine whether the vehicle 10 can enter the other lane L2, and the vehicle 10 stops (gets stuck) behind the other vehicle V2.

[0026] When the vehicle 10 gets stuck, the vehicle 10 is driven manually, enters the other lane L2 at a slow speed (for example, at a driving speed of 5 km / h or less), and passes the other vehicle V2 while avoiding contact with the other vehicle V1. In places with many parked vehicles, such as urban areas, the vehicle 10 may frequently get stuck, and if the driver has to manually drive the vehicle to avoid the parked vehicles every time the vehicle gets stuck, the driver may feel uncomfortable due to the intermittent avoidance of the parked vehicles. Therefore, when the vehicle 10 gets stuck, the first controller 12 transmits a request for remote assistance to the remote assistance device 20 so that the operator performs driving operations to avoid the parked vehicles instead of the driver.

[0027] 2, when a driver with little driving experience drives the vehicle 10 manually, the driver may feel anxious about contact with the other vehicle V1 when avoiding the other parked vehicle V2. In such a case, an operator drives the vehicle 10 instead of the driver, and a request for remote assistance is sent to the remote assistance device 20 by the driver's operation.

[0028] The first controller 12 has a function of transmitting a request for remote assistance to the remote assistance device 20. Remote assistance refers to an operator outside the vehicle 10 remotely operating the vehicle 10 from outside the vehicle 10 via the first controller 12, or the operator providing information to the first controller 12 or the driver of the vehicle 10. The operator is, for example, in a management center that manages the traveling of one or more vehicles 10, and provides necessary information to the vehicle 10 in response to a request from the vehicle 10 or regardless of a request from the vehicle 10, and remotely operates the vehicle 10.

[0029] The request for remote assistance includes information indicating that the autonomous driving control of the vehicle 10 has stopped, and information necessary for the operator to provide driving assistance for the vehicle 10. The request for remote assistance includes, for example, the driving status of the vehicle 10. The driving status means, for example, a driving scene, and includes a driving state, a driving plan, and a surrounding driving environment.

[0030] The driving conditions include vehicle speed, acceleration, yaw rate, steering angle, etc., and are obtained from various sensors (not shown) such as a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, and a steering angle sensor. The driving plan is information for the vehicle 10 to travel to a destination, and includes a driving route, etc. The driving route includes at least information on the road on which the vehicle 10 travels, the driving lane, and the traveling direction of the vehicle 10. The driving plan is obtained from a navigation device (not shown). The surrounding driving environment refers to the state of objects (especially obstacles) around the vehicle 10, and is obtained by using a method such as pattern matching from the detection results of the imaging device 11, a distance measuring device, etc.

[0031] 2, the first controller 12 acquires the vehicle speed of the vehicle 10 and transmits to the remote support device 20 information indicating that the vehicle 10 is stopped together with the vehicle speed of the vehicle 10. The first controller 12 also transmits position information of the destination Px and the set driving route to the remote support device 20. Furthermore, the first controller 12 acquires an image of the surroundings of the vehicle 10 including the other lane L2 from the imaging device 11 and transmits it to the remote support device 20. Then, upon receiving the request for remote support, the remote support device 20 starts driving support for the vehicle 10 using the remote support function of the second controller 21.

[0032] The determination unit 31 has a function of determining whether or not there is another vehicle traveling toward the entry range of the other lane L2 into which the vehicle 10 will enter, when the vehicle 10 enters the other lane L2 different from the driving lane L1 in which the vehicle 10 is traveling. The second controller 21 determines whether or not the vehicle 10 will enter the other lane L2 from the driving lane L1 based on the traveling situation of the vehicle 10. For example, when a parked vehicle is detected in front of the vehicle 10 and it is necessary to travel on the other lane L2 to pass the side of the parked vehicle, the second controller 21 determines that the vehicle 10 will enter the other lane L2 from the driving lane L1. On the other hand, even if an obstacle is detected in front of the vehicle 10, if the vehicle 10 can continue traveling along the driving lane L1 while avoiding the obstacle, the second controller 21 determines that the vehicle 10 will not enter the other lane L2 from the driving lane L1.

[0033] When the vehicle 10 returns to the driving lane L1 after entering the other lane L2, such as when avoiding an obstacle on the driving lane L1, the second controller 21 sets the range in which the vehicle 10 travels from entering the other lane L2 to exiting the other lane L2 as the entry range. On the other hand, when the vehicle 10 does not return to the driving lane L1 after entering the other lane L2, such as when changing lanes along the driving route, the second controller 21 sets the range in which the vehicle 10 travels from entering the other lane L2 to starting to travel along the other lane L2 as the entry range.

[0034] 2, the second controller 21 sets an entry range A1 in the other lane L2 based on a distance D1 from the vehicle 10 to the other vehicle V2, which is an obstacle, a distance D2 corresponding to the length of the other vehicle V2 along the traveling direction of the vehicle 10 (left and right direction in the drawing), and a distance D3 required to change lanes from the other lane L2 to the driving lane L1 after passing the side of the other vehicle V2. Specifically, the second controller 21 calculates an end position Py on the driving lane L1 where the vehicle 10 ends avoiding the other vehicle V2 from the distances D1, D2, and D3, and sets the entry range A1 in the other lane L2 from a position P1a corresponding to the current position P1 of the vehicle 10 to a position Pya corresponding to the end position Py.

[0035] Distance D1 is obtained from the detection result of the distance measuring device of vehicle 10. A predetermined value is set for distance D2 for each vehicle type, and the distance according to the vehicle type of the recognized other vehicle V2 is obtained from a storage medium such as a ROM. For example, distance D2 for a standard vehicle is set to 3 to 5 m, and distance D2 for a large vehicle such as a truck is set to 5 to 10 m. Distance D3 is preset to a certain value (for example, 15 to 30 m). Alternatively, distance D3 may be calculated based on the traveling speed of vehicle 10 and the widths of traveling lane L1 and other lane L2.

[0036] After setting the entry range A1, the second controller 21 detects other vehicles traveling around the vehicle 10. In the traveling scene shown in FIG. 2, the other vehicle V1 traveling in the other lane L2 is detected, and the traveling direction of the other vehicle V1 is acquired. Then, it is determined whether the traveling direction of the other vehicle V1 is toward the entry range A1. If it is determined that the traveling direction of the other vehicle V1 is not toward the entry range A1, the second controller 21 avoids the other vehicle V2 by normal autonomous traveling control. On the other hand, if it is determined that the traveling direction of the other vehicle V1 is toward the entry range A1, the second controller 21 executes processing by the function of the calculation unit 32.

[0037] The calculation unit 32 has a function of calculating a delay time from when the imaging device 11 captures the surroundings of the vehicle 10 until the image of the surroundings of the vehicle 10 is displayed to the operator operating the remote support device 20 (i.e., until the image is displayed on the display device 22). When the remote support device 20 determines, by the function of the determination unit 31, that there is another vehicle V1 traveling toward the entry range A1, it acquires information on the imaging time when the imaging device 11 captured the surroundings of the vehicle 10 together with the image acquired by the imaging device 11. Next, it calculates the transfer time of the image from the vehicle 10 to the remote support device 20 from the difference between the imaging time and the reception time when the remote support device 20 (or the second controller 21) received the image. Then, it calculates the delay time by adding the display time from when the remote support device 20 receives the image to when it displays the image to the operator to the transfer time.

[0038] As the information on the image capturing time, the second controller 21 acquires at least one of a code indicating the image capturing time superimposed on the image captured by the image capturing device 11 and a number indicating the image capturing time superimposed on the image. The remote support device 20 may acquire an image from an outside image capturing device installed on the road on which the vehicle 10 is traveling, instead of the image capturing device 11 included in the vehicle 10. By using the outside image capturing device, remote support can be performed based on information that cannot be acquired from the image capturing device 11 and the sensor of the vehicle 10. In this case, the delay time is calculated as the time from when the outside image capturing device installed on the road acquires an image to when the image is displayed to the operator. The delay time mainly depends on the communication speed of the network. The display time may be set in advance to a certain time according to the processing capacity of the second controller 21 and the display capacity of the display device 22.

[0039] The setting unit 33 has a function of setting a gaze range, which is a range in which other vehicles whose travel will be obstructed by the vehicle 10 when the vehicle 10 enters the other lane L2 travel. The second controller 21 sets the gaze range to a range obtained by expanding the approach range A1 on the opposite side of the traveling direction of the other vehicle traveling on the other lane L2. The second controller 21 expands the approach range A1 by a length based on the traveling speed of the other vehicle and the delay time calculated by the function of the calculation unit 32. Specifically, the second controller 21 detects the traveling speed of the other vehicle traveling on the other lane L2, and calculates the expansion distance by multiplying the traveling speed of the other vehicle by the delay time. Then, the second controller 21 sets a gaze range in which the approach range A1 is expanded by the expansion distance along the other lane L2 on the opposite side of the traveling direction of the other vehicle.

[0040] In the traveling scene shown in FIG. 2, the second controller 21 detects the traveling speed of the other vehicle V1 traveling on the other lane L2 using a distance measuring device or the like, and calculates the extended distance Dx by multiplying the traveling speed of the other vehicle V1 by the delay time. Then, a gaze range is set by extending the approach range A1 by the extended distance Dx along the other lane L2 on the opposite side to the traveling direction of the other vehicle V1 (to the right side in the drawing). The traveling direction of the other vehicle V1 is obtained from, for example, a change in the traveling position of the other vehicle V1 or a change in the distance between the vehicle 10, detected by a distance measuring device or the like. FIG. 2 shows the approach range A1 and the extended range A2, which is the extended part. The gaze range is a range consisting of the approach range A1 and the extended range A2 shown in FIG. 2.

[0041] The detection unit 34 has a function of detecting a watched vehicle traveling in the gaze range among other vehicles traveling in the other lane L2. The second controller 21 extracts a watched vehicle from other vehicles traveling in the other lane L2 by the function of the detection unit 34. In the traveling scene shown in FIG. 2, the other vehicle V1 traveling at position Q1 in the other lane L2 enters the extended range A2, so the second controller 21 detects the other vehicle V1 as a watched vehicle. If the position of the other vehicle V1 is within the gaze range, the second controller 21 determines that the other vehicle V1 is traveling in the gaze range, and if the position of the other vehicle V1 is not within the gaze range, the second controller 21 determines that the other vehicle V1 is not traveling in the gaze range.

[0042] The display unit 35 has a function of displaying the watched vehicle to the operator in a manner that distinguishes it from other vehicles. The second controller 21 outputs to the display device 22 a display that indicates to the operator that the other vehicle traveling in the gaze range is the watched vehicle. The display is not particularly limited, and examples thereof include images, characters, and figures. As the display to be output to the display device 22, the second controller 21 executes, for example, at least one of displaying a frame around the watched vehicle and displaying the other vehicle with a higher transparency than the watched vehicle. In addition, the second controller 21 may acquire or generate a video that displays images acquired from the imaging device 11 in a time series, execute an editing process on the video to distinguish the watched vehicle from the other vehicles, and display the video that has been edited using the display device 22 to the operator. Examples of the editing process include a first editing process that displays a display portion corresponding to the gaze range that has a color different from the surroundings, and a second editing process that displays a line on the video at a position corresponding to the end of the gaze range on the opposite side of the traveling direction of the other vehicle. The second controller 21 executes at least one of a first editing process and a second editing process.

[0043] FIG. 3 is an example of an image displayed to the operator in the driving scene of FIG. 2. The lane on the left side of FIG. 3 is the driving lane L1, the lane on the right side is the other lane L2, and the upper side of the drawing is the direction toward the destination Px. The driving lane L1 displays the vehicle 10 and the other lane L2 in which the vehicle is parked, and the other lane L2 displays the other vehicle V1 traveling from the top to the bottom of the drawing. In addition, the gaze range A3 is displayed on the other lane L2 as a rectangular display portion with hatching different from the surroundings. As shown in FIG. 3, since the other vehicle V1 has entered the gaze range A3, a frame X is displayed around the other vehicle V1, which is the gaze vehicle. This allows the operator to easily and intuitively grasp the presence of the other vehicle V1, whose traveling is obstructed by the vehicle 10. In addition, by expanding the entry range A1 according to the delay time, remote support that does not obstruct the traveling of the other vehicle V1 can be performed even if a delay occurs in communication.

[0044] 1, the support unit 36 ​​has a function of transmitting support information necessary for remote support to the first controller 12. The operator inputs the support information using the input device 23 after grasping the traveling situation of the vehicle 10 from the image shown in FIG.

[0045] The support information is information required for remote support input by the operator to the input device 23, and includes, for example, an execution instruction for the first controller 12 to execute a certain driving operation by autonomous driving control, and judgment information for the first controller 12 to set the driving operation by autonomous driving control. The execution instruction includes an instruction to start the vehicle 10 from a stopped state, an instruction to change the vehicle 10 from the driving lane L1 to an adjacent lane, an instruction to blink the turn signal of the vehicle 10, etc. As an example, in the driving scene shown in Fig. 2, when the operator causes the vehicle 10 to enter the other lane L2 at a slow speed, the first controller 12 receives operation information of the accelerator pedal (input device 23) as support information and starts the vehicle 10 from a stopped state.

[0046] On the other hand, the judgment information includes at least one of the following in addition to the driving conditions of the vehicle 10: a new destination, a new driving route, and a type of obstacle present around the vehicle 10. The new destination and the new driving route are acquired from a navigation device (not shown), and the type of obstacle is acquired from a detection result of a detection device of the vehicle 10. The type of obstacle is an attribute of the obstacle, and refers to, for example, the type, size, operation, and effect of the obstacle on the driving of the vehicle 10. As an example, in the driving scene shown in FIG. 2, when the vehicle 10 enters the other lane L2 at a slow speed, the driving conditions of the other lane L2 can be acquired by the imaging device 11. In this case, the first controller 12 receives information on whether or not the operator can enter the other lane L2 as judgment information.

[0047] When the first controller 12 receives the support information from the second controller 21, the first controller 12 outputs a control signal to the vehicle control device 13 to execute autonomous driving control based on the support information. In the driving scene shown in Fig. 2, another vehicle V1 traveling in the gaze range A3 is detected, so the operator operates the brake pedal of the input device 23 so that the vehicle 10 stops at position P1 and waits until the other vehicle V1 reaches position Q2 from position Q1, as shown in Fig. 4. Then, support information according to the operator's operation of the brake pedal is transmitted to the first controller 12, and the first controller 12, which has received the support information, outputs a control signal to the vehicle control device 13 to operate the brakes.

[0048] Then, after the other vehicle V1 passes beside the vehicle 10, the operator operates the steering wheel and accelerator pedal of the input device 23 so as to travel along the travel trajectory T1 shown in Fig. 5 and avoid the other vehicle V2. Assistance information according to the operator's operation of the steering wheel and accelerator pedal is transmitted to the first controller 12, and the first controller 12, which receives the assistance information, outputs control signals for operating the drive device and steering device to the vehicle control device 13. In this way, the vehicle 10 travels from position P1 to position P2, and continues traveling under autonomous travel control while avoiding the other vehicle V2.

[0049] The remote support device 20 may detect the traveling speed of the other vehicle V1, calculate the passing time from when the vehicle 10 enters the entry range A1 until when it exits, calculate the traveling distance of the other vehicle V1 by multiplying the traveling speed of the other vehicle V1 by the sum of the passing time and the delay time, and set the gaze range A3 along the other lane L2 having a length corresponding to the traveling distance of the other vehicle V1. After calculating the passing time of the vehicle 10, the remote support device 20 may multiply the traveling speed of the vehicle 10 by the sum of the passing time and the delay time to calculate the traveling distance of the vehicle 10, and set the gaze range A3 along the other lane L2 having a length corresponding to the traveling distance of the vehicle 10. In this case, the traveling speed of the vehicle 10 is acquired before entering the entry range A1, and the time required from when the vehicle 10 enters the entry range A1 until when it exits when the vehicle 10 travels at that traveling speed is calculated as the passing time.

[0050] Fig. 6 is a plan view showing another example of a driving scene in which the second controller 21 autonomously controls the driving of the vehicle 10 by using the remote assistance function. The road shown in Fig. 6 is a two-lane road having a driving lane L1 and an other lane L2, and unlike the driving scene shown in Fig. 2, the vehicles traveling on the driving lane L1 and the other lane L2 travel from the left side to the right side of the drawing. In other words, the driving scene in Fig. 6 is a driving scene in which a vehicle enters an overtaking lane from a driving lane, unlike the driving scene in Fig. 2 in which a vehicle enters an oncoming lane.

[0051] In the driving scene shown in FIG. 6, the vehicle 10 is driving at position P3 in the driving lane L1 toward the destination Px on the right side of the drawing. In addition, another vehicle V1 is driving at position Q3 in the other lane L2, and another vehicle V2 is parked at position R2 in the driving lane L1. In order to continue driving under the autonomous driving control, the vehicle 10 needs to enter the other lane L2 to avoid the other vehicle V2, and then return to the driving lane L1. However, when the vehicle 10 is driving at a relatively high speed, the driver or the first controller 12 is late in determining whether or not the vehicle 10 can enter the other lane L2, and the vehicle 10 stops behind the other vehicle V2 without being able to enter the other lane L2. In this case, the first controller 12 cannot continue driving under the autonomous driving control, so it transmits a request for remote assistance to the remote assistance device 20.

[0052] The second controller 21, which has received the request for remote assistance, sets the entry range A1 in the same manner as in the driving scene shown in Fig. 2, using the function of the determination unit 31. After setting the entry range A1, the second controller 21 detects other vehicles traveling around the vehicle 10. In the driving scene shown in Fig. 6, the other vehicle V1 traveling in the other lane L2 is detected, and the traveling direction of the other vehicle V1 is to the right in the drawing, heading toward the entry range A1, so it is determined that there is another vehicle V1 traveling toward the entry range A1.

[0053] Next, the second controller 21, using the function of the calculation unit 32, acquires the image capture time from the code of the image acquired from the imaging device 11, calculates the image transfer time from the difference between the image capture time and the image reception time, and calculates the delay time by adding the display time to the transfer time. Next, the second controller 21, using the function of the setting unit 33, detects the traveling speed of the other vehicle V1, and calculates the extended distance by multiplying the traveling speed of the other vehicle V1 by the delay time. Then, it sets a gaze range that expands the entry range A1 along the other lane L2 on the opposite side to the traveling direction of the other vehicle V1 (to the left side in the drawing).

[0054] FIG. 7 is an example of an image displayed to the operator in the driving scene of FIG. 6. The lane on the right side of FIG. 7 is the driving lane L1, the lane on the left side is the other lane L2, and the bottom of the drawing is the direction toward the destination Px. The vehicle 10 is displayed in the driving lane L1, and the other vehicle V1 traveling from the top to the bottom of the drawing is displayed in the other lane L2. In addition, the gaze range A3 is displayed on the other lane L2 as a rectangular display portion with hatching different from the surroundings. In FIG. 7, the other vehicle V1 is not traveling in the gaze range A3 and is not a gaze vehicle, so it is displayed with a higher transparency than the vehicle 10 (or other vehicles other than the other vehicle V1). In this way, the remote assistance function of this embodiment can also be applied to a driving scene in which a vehicle enters an overtaking lane from a driving lane.

[0055] Fig. 8 is a plan view showing yet another example of a driving scene in which the second controller 21 autonomously controls the driving of the vehicle 10 by using the remote assistance function. In the driving scene shown in Fig. 8, there is a road K1 extending in the vertical direction of the drawing and a road K2 extending in the horizontal direction of the drawing, and an intersection C exists at the intersection where the two roads K1 and K2 intersect. The roads K1 and K2 are one-lane roads on each side. Vehicles traveling on lanes L1 to L4 of the road K1 and lanes L5 to L8 of the road K2 can turn right, turn left, or go straight at the intersection C.

[0056] In the driving scene shown in Fig. 8, the vehicle 10 is driving at position P4 in the driving lane L1 toward the destination Px on the left side of the drawing. In addition, another vehicle V1 is driving at position Q4 in the lane L7. The vehicle 10 turns left at the intersection C to drive toward the destination Px, and drives to position P5 along the driving trajectory T2. In this case, if the situation of the lane L5 cannot be detected from position P4 due to an obstacle or the like, and a left turn cannot be executed by the autonomous driving control, the first controller 12 transmits a request for remote assistance to the remote assistance device 20.

[0057] The second controller 21, which has received the request for remote assistance, sets, by the function of the determination unit 31, the range in which the vehicle 10 travels from entering the intersection C until it starts traveling along the other lane L5 as the entry range A1. In the traveling scene shown in FIG. 8, the vehicle 10 starts traveling along the other lane L5 at position P5, so the entry range A1 shown in FIG. 8 is set. After the entry range A1 is set, the second controller 21 detects other vehicles traveling around the vehicle 10. In the traveling scene shown in FIG. 8, the other vehicle V1 traveling at position Q4 of the other lane L7 is detected, and the traveling direction of the other vehicle V1 is the left direction in the drawing, heading toward the entry range A1, so it is determined that there is another vehicle V1 traveling toward the entry range A1.

[0058] Next, the second controller 21 calculates the delay time using the function of the calculation unit 32, detects the traveling speed of the other vehicle V1 using the function of the setting unit 33, and calculates the extended distance by multiplying the traveling speed of the other vehicle V1 by the delay time. Then, the second controller 21 sets a gaze range that extends the entry range A1 along the other lane L7 on the opposite side (to the right in the drawing) to the traveling direction of the other vehicle V1 entering the intersection C by going straight.

[0059] Fig. 9 is an example of an image displayed to the operator in the driving scene of Fig. 8. A vehicle 10 is displayed in the driving lane L1. The gaze range A3 is displayed as a rectangular display portion with hatching different from the surroundings, spanning another lane L5, the intersection C, and another lane L7. In this way, the remote assistance function of this embodiment can also be applied to a driving scene in which the vehicle 10 enters an intersection.

[0060] The driving scene shown in FIG. 10 is different from the driving scene shown in FIG. 8 in that the vehicle 10 turns right at an intersection C. In the driving scene shown in FIG. 10, the vehicle 10 is driving at position P4 in the driving lane L1 toward the destination Px on the right side of the drawing. The vehicle 10 turns right at the intersection C to head toward the destination Px, and drives to position P6 along the driving trajectory T3. In addition, another vehicle V1 is driving at position Q5 in the lane L6. In this case, when the display of the traffic light at the intersection C frequently changes and it is difficult to turn right at the intersection C under autonomous driving control, the first controller 12 transmits a request for remote assistance to the remote assistance device 20.

[0061] The second controller 21, which has received the request for remote assistance, sets, by the function of the determination unit 31, the range in which the vehicle 10 travels from entering the intersection C until it starts traveling along the other lane L8 as the entry range A1. In the traveling scene shown in FIG. 10, the vehicle 10 starts traveling along the other lane L8 at position P6, so the entry range A1 shown in FIG. 10 is set. After the entry range A1 is set, the second controller 21 detects other vehicles traveling around the vehicle 10. In the traveling scene shown in FIG. 10, the other vehicle V1 traveling at position Q5 on the other lane L6 is detected, and the traveling direction of the other vehicle V1 is to the right in the drawing and toward the entry range A1, so it is determined that there is another vehicle V1 traveling toward the entry range A1.

[0062] Next, the second controller 21 calculates the delay time using the function of the calculation unit 32, detects the traveling speed of the other vehicle V1 using the function of the setting unit 33, and calculates the extended distance by multiplying the traveling speed of the other vehicle V1 by the delay time. Then, the second controller 21 sets a gaze range that is extended along the other lane L6 in which the other vehicle V1 is traveling, on the opposite side of the traveling direction of the other vehicle V1 going straight into the intersection C (to the left side in the drawing), for a portion of the entry range A1 that corresponds to the other lane L8 in which the vehicle 10 is entering.

[0063] The second controller 21 may include in the gaze range A3 a range in which other vehicles turning left and entering the intersection C are traveling. For example, the range A2a of the lane L4 shown in Fig. 10 is included in the gaze range A3 as an extended range. The length of the range A2a in the lane L4 direction is set to a length (e.g., 15 to 30 m) corresponding to the range in front of the intersection C where lane changes are not permitted.

[0064] Fig. 11 is an example of an image displayed to the operator in the driving scene of Fig. 10. A vehicle 10 is displayed in a driving lane L1. Also, a gaze range A3 is displayed as a rectangular display portion with hatching different from the surroundings.

[0065] [Processing in remote support system] 12 to 14, a procedure for processing information by the second controller 21 will be described. Fig. 12 is an example of a flowchart showing information processing executed in the remote support system 1 of this embodiment. The processing described below is executed by a processor (CPU) included in the remote support device 20.

[0066] First, in step S1, the function of the determination unit 31 is used to acquire the driving conditions around the vehicle 10, and in the subsequent step S2, it is determined whether or not the vehicle 10 will enter another lane L2 different from the driving lane L1. If it is determined that the vehicle 10 will not enter the other lane L2, the process proceeds to step S1. On the other hand, if it is determined that the vehicle 10 will enter the other lane L2, the process proceeds to step S3.

[0067] In step S3, the entry range A1 is set in the other lane L2, and in the following step S4, it is determined whether or not there is another vehicle traveling toward the entry range A1 based on the detection result of the detection device of the vehicle 10. If it is determined that there is no other vehicle traveling toward the entry range A1, the vehicle travels according to normal autonomous driving control. On the other hand, if it is determined that there is another vehicle traveling toward the entry range A1, the process proceeds to step S5.

[0068] In step S5, the delay time is calculated by the function of the calculation unit 32, and in step S6, the setting unit 33 sets a gaze range by expanding the approach range A1 to the opposite side of the traveling direction of the other vehicle. In step S7, the detection unit 34 detects the gazed vehicle traveling in the gaze range, and in step S8, the display unit 35 distinguishes between the gazed vehicle and other vehicles and displays them to the operator. Thereafter, the execution of the routine is terminated.

[0069] 13 and 14 are another example of a flowchart showing information processing executed in the remote support system 1 of this embodiment. The processing described below is executed by a processor (CPU) included in the remote support device 20.

[0070] First, in step S11 of Fig. 13, the function of the determination unit 31 is used to determine whether or not a request for assistance has been received from the vehicle 10. If it is determined that a request for assistance has not been received from the vehicle 10, step S11 is repeated. In contrast, if it is determined that a request for assistance has been received from the vehicle 10, the process proceeds to step S12. In step S12, an image of the surroundings of the vehicle 10 is acquired from the imaging device 11, and in the following step S13, obstacles around the vehicle 10 are detected from the acquired image. In the following step S14, the travel route of the vehicle 10 is acquired from a navigation device (not shown).

[0071] In the next step S15, it is determined whether or not the vehicle 10 will enter the adjacent lane based on the positions of obstacles around the vehicle 10 and the travel route. If it is determined that the vehicle 10 will not enter the adjacent lane, the process proceeds to step S11. On the other hand, if it is determined that the vehicle 10 will enter the adjacent lane, the process proceeds to step S16.

[0072] In step S16, the end position on the travel lane L1 where the vehicle 10 ends the obstacle avoidance is calculated, and in the following step S17, an entry range A1 is set in the adjacent lane based on the current position and the end position. In the following step S18, it is determined whether or not there is another vehicle traveling toward the entry range A1. If it is determined that there is no other vehicle traveling toward the entry range A1, the process proceeds to step S30. On the other hand, if it is determined that there is another vehicle traveling toward the entry range A1, the process proceeds to step S19.

[0073] In step S19, the calculation unit 32 uses its function to obtain the imaging time when the imaging device captured the image of the surroundings of the vehicle 10 from a code displayed on the image, etc., and in the following step S20, the calculation unit 32 calculates the transfer time of the image from the difference between the imaging time and the reception time of the image, and in the following step S21, the calculation unit 32 calculates the delay time by adding the display time for displaying the received image on the display device 22 to the transfer time.

[0074] 14, in step S22, the setting unit 33 detects the traveling speed of the other vehicle, and in the following step S23, the traveling speed of the other vehicle is multiplied by the delay time to calculate an extended distance, and in the following step S24, a gaze range is set by extending the entry range A1 by the extended distance. In step S25, the detection unit 34 detects a gazed vehicle traveling in the gaze range A3, and in step S26, the display unit 35 generates a video that displays images in chronological order from the image data acquired from the imaging device 11. In the following step S27, the generated video is displayed to the operator.

[0075] In step S28, it is determined whether or not there is a watched vehicle traveling in the gaze range A3. If it is determined that there is no watched vehicle, the process proceeds to step S27, and the display of the generated video continues. On the other hand, if it is determined that there is a watched vehicle, the process proceeds to step S29, and a frame X is displayed around the detected watched vehicle.

[0076] After displaying the video to the operator, in step S30, the support unit 36 ​​acquires input information input by the operator to the input device 23, and in the following step S31, the support information is transmitted to the first controller 12 of the vehicle 10, and autonomous driving control based on the support information is executed. Then, execution of the routine is terminated.

[0077] [Embodiments of the present invention] As described above, according to the present embodiment, in the remote assistance method of the remote assistance device 20 that assists in driving control of the target vehicle 10 from outside the target vehicle 10, when the target vehicle 10 enters another lane L2 different from the driving lane L1 in which the target vehicle 10 is traveling, the remote assistance device 20 determines whether or not there is another vehicle V1 traveling toward the entry range A1 into which the target vehicle 10 will enter in the other lane L2, and when it is determined that there is the other vehicle V1 traveling toward the entry range A1, The remote assistance method calculates a delay time from when the surroundings of the target vehicle 10 are imaged by the imaging device 11 to when an image of the surroundings of the target vehicle 10 is displayed to an operator operating the remote assistance device 20, sets a gaze range A3 by expanding the entry range A1 on the opposite side to the traveling direction of the other vehicle V1 by a length based on the traveling speed of the other vehicle V1 and the delay time, and outputs a display to the display device 22 to show the operator that the other vehicle traveling in the gaze range A3 is a gazed vehicle. This makes it possible to perform appropriate remote assistance even if a delay occurs in communication.

[0078] According to the remote assistance method of the present embodiment, the remote assistance device 20 detects the traveling speed of the other vehicle V1, calculates the extended distance Dx by multiplying the traveling speed of the other vehicle V1 by the delay time, and sets the gaze range A3 by extending the entry range A1 along the other lane L2 by the extended distance Dx on the opposite side of the traveling direction of the other vehicle V1. This allows the gaze range A3 to be set according to the traveling state of the other vehicle V1.

[0079] According to the remote assistance method of the present embodiment, the remote assistance device 20 detects the traveling speed of the other vehicle V1, calculates the passing time from when the target vehicle 10 enters the entry range A1 until when it exits the entry range A1, calculates the traveling distance of the other vehicle V1 by multiplying the traveling speed of the other vehicle V1 by the sum of the passing time and the delay time, and sets the gaze range A3 along the other lane L2 having a length corresponding to the traveling distance of the other vehicle. This allows the gaze range A3 to be set according to the driving scene.

[0080] According to the remote assistance method of the present embodiment, the remote assistance device 20 calculates the passing time from when the target vehicle 10 enters the entry range A1 until when it exits the entry range A1, calculates the traveling distance of the target vehicle 10 by multiplying the traveling speed of the target vehicle 10 by the sum of the passing time and the delay time, and sets the gaze range A3 along the other lane L2 having a length corresponding to the traveling distance of the target vehicle 10. This allows the gaze range A3 to be set according to the traveling scene.

[0081] According to the remote assistance method of the present embodiment, the remote assistance device 20 acquires the traveling speed of the target vehicle 10 before the target vehicle 10 enters the entry range A1, and calculates the time required for the target vehicle 10 to enter and exit the entry range A1 when the target vehicle 10 is traveling at the traveling speed as the passing time. This makes it possible to calculate the passing time more accurately.

[0082] According to the remote assistance method of the present embodiment, the remote assistance device 20 acquires information on the image capturing time when the imaging device 11 captured the image of the surroundings of the target vehicle 10 together with the image, calculates the transfer time of the image from the difference between the image capturing time and the reception time when the image was received, and calculates the delay time by adding the display time from when the image was received to when the image was displayed on the display device 22 to the transfer time. This makes it possible to calculate the delay time more accurately.

[0083] According to the remote assistance method of the present embodiment, when the other lane L2 is an adjacent lane to the driving lane L1 and the target vehicle 10 travels on the adjacent lane to avoid an obstacle on the driving lane L1, the remote assistance device 20 calculates an end position Py on the driving lane L1 where the target vehicle 10 ends the obstacle avoidance based on a distance D1 from the target vehicle 10 to the obstacle, a distance D2 corresponding to the length of the obstacle in the traveling direction of the target vehicle 10, and a distance D3 for changing lanes from the adjacent lane to the driving lane L1 after passing the side of the obstacle, and sets the approach range A1 from a position P1a corresponding to the current position P1 of the target vehicle 10 to a position Pya corresponding to the end position Py on the adjacent lane. This allows remote assistance suitable for a driving scene in which an obstacle on the driving lane L1 is avoided.

[0084] According to the remote assistance method of the present embodiment, when the target vehicle 10 turns right or left at an intersection C to enter the other lane, the remote assistance device 20 sets the range in which the target vehicle 10 travels from entering the intersection C to starting to travel along the other lane as the entry range A1. This makes it possible to perform remote assistance in a driving scene in which the target vehicle 10 turns right or left at the intersection C.

[0085] According to the remote assistance method of the present embodiment, the remote assistance device 20 acquires the image from an outside image capture device installed on the road on which the target vehicle 10 is traveling, and calculates the delay time as the time from when the outside image capture device acquires the image to when the image is displayed to the operator. This makes it possible to perform remote assistance using an image capture device 11 installed on the road.

[0086] According to the remote assistance method of the present embodiment, the remote assistance device 20 executes at least one of displaying a frame X around the watched vehicle and displaying the other vehicle V1 with a higher transparency than the watched vehicle. This allows the watched vehicle and the other vehicles to be displayed in a distinguishable manner.

[0087] According to the remote assistance method of the present embodiment, the remote assistance device 20 acquires a video displaying the images in chronological order, executes a first editing process to display a display portion of the video that has a different color from the surroundings and corresponds to the gaze range A3, and / or executes a second editing process to display a line in the video at a position corresponding to the end of the gaze range A3 on the opposite side in the traveling direction of the other vehicle V1, and displays the video on which at least one of the first editing process and the second editing process has been executed to the operator. This allows the gazed vehicle and the other vehicles to be displayed in a distinguishable manner.

[0088] According to the present embodiment, the remote assistance device 20 is located outside the target vehicle 10 and assists in driving control of the target vehicle 10. When the target vehicle 10 enters a lane L2 different from the driving lane L1 in which the target vehicle 10 is currently traveling, the remote assistance device 20 includes a determination unit 31 that determines whether or not there is a other vehicle V1 traveling in the other lane L2 toward an entry range A1 into which the target vehicle 10 is to enter, and when the determination unit 31 determines that there is a other vehicle V1 traveling toward the entry range A1, the remote assistance device 20 captures an image of the surroundings of the target vehicle 10 by an imaging device 11 of the target vehicle 10 and then detects the presence of the other vehicle V1. The remote support device 20 includes a calculation unit 32 that calculates a delay time until an image of the surroundings of the target vehicle 10 is displayed to an operator who operates the remote support device 20, a setting unit 33 that sets a gaze range A3 by expanding the entry range A1 on the opposite side of the traveling direction of the other vehicle V1 by a length based on the traveling speed of the other vehicle V1 and the delay time calculated by the calculation unit 32, and a display unit 35 that outputs a display to a display device 22 to show to the operator that the other vehicle traveling in the gaze range A3 set by the setting unit 33 is a gazed vehicle. This makes it possible to perform appropriate remote support even if a delay occurs in communication. [Explanation of symbols]

[0089] 1... remote support system, 10... vehicle (target vehicle), 11... imaging device, 12... first controller, 13... vehicle control device, 20... remote support device, 21... second controller, 22... display device, 23... input device, 31... determination unit, 32... calculation unit, 33... setting unit, 34... detection unit, 35... display unit, 36... support unit, A1... entry range, A2... extension range, A3... gaze range, D1, D2, D3...distance, Dx...extended distance, K1, K2...road, L1...driving lane, L2, L3, L4, L5, L6, L7, L8...other lanes, P1, P2, P3, P4, P5, P6...position, Px...destination, Py...end position, P1a, Pya...position, Q1, Q2, Q3, Q4, Q5...position, R1, R2...position, T1, T2, T3...driving trajectory, V1, V2...other vehicles, X...frame

Claims

1. A remote support method for a remote support device that supports driving control of a target vehicle from outside the target vehicle, comprising: The remote support device includes: When the target vehicle is entering a lane different from the lane in which the target vehicle is currently traveling, it is determined whether or not there is another vehicle traveling toward an entry range into which the target vehicle is to enter in the other lane; When it is determined that the other vehicle traveling toward the entry range is present, a delay time is calculated from when an image of the surroundings of the target vehicle is captured by an imaging device of the target vehicle until an image of the surroundings of the target vehicle is displayed to an operator operating the remote assistance device; A gaze range is set by expanding the entry range on the opposite side of the traveling direction of the other vehicle by a length based on the traveling speed of the other vehicle and the delay time, A remote assistance method, comprising: outputting, to a display device, a display indicating to the operator that the other vehicle traveling within the gaze range is a gazed vehicle.

2. The remote support device includes: Detecting the traveling speed of the other vehicle; multiplying the travel speed of the other vehicle by the delay time to calculate an extended distance; The remote assistance method according to claim 1 , further comprising setting the gaze range by expanding the entry range along the other lane by the expansion distance on the opposite side of the traveling direction of the other vehicle.

3. The remote support device includes: Detecting the traveling speed of the other vehicle; Calculating a transit time of the target vehicle from when it enters the entry range until when it exits the entry range; multiplying the traveling speed of the other vehicle by the sum of the passing time and the delay time to calculate a traveling distance of the other vehicle; The remote assistance method according to claim 1 , further comprising: setting the gaze range along the other lane, the gaze range having a length corresponding to the travel distance of the other vehicle.

4. The remote support device includes: Calculating a transit time of the target vehicle from when it enters the entry range until when it exits the entry range; multiplying the travel speed of the target vehicle by the sum of the passing time and the delay time to calculate the travel distance of the target vehicle; The remote assistance method according to claim 1 , further comprising: setting the gaze range along the other lanes, the gaze range having a length corresponding to the travel distance of the target vehicle.

5. The remote support device includes: acquiring the travel speed of the target vehicle before the target vehicle enters the entry range; The remote assistance method according to claim 4 , wherein the passing time is calculated as a time required for the target vehicle to enter and exit the entry range when traveling at the travel speed.

6. The remote support device includes: Acquire information on an image capturing time when the imaging device captured an image of the surroundings of the target vehicle together with the image, calculating a transfer time of the image from a difference between the image capture time and a reception time of the image; The remote assistance method according to claim 1 , further comprising the step of: calculating the delay time by adding a display time from when the image is received until when the image is displayed on the display device to the transfer time.

7. The remote support device includes: When the other lane is an adjacent lane to the driving lane and the target vehicle is traveling on the adjacent lane to avoid an obstacle in the driving lane, an end position on the driving lane at which the target vehicle will end its avoidance of the obstacle is calculated based on a distance from the target vehicle to the obstacle, a distance in the traveling direction of the target vehicle corresponding to a length of the obstacle, and a distance required for the target vehicle to change lanes from the adjacent lane to the driving lane after passing beside the obstacle; The remote assistance method according to claim 1 , wherein the approach range is set to a range from a position corresponding to the current position of the target vehicle to a position corresponding to the end position in the adjacent lane.

8. The remote support device includes: The remote assistance method according to claim 1, wherein when the target vehicle turns right or left at an intersection to enter the other lane, the range in which the target vehicle travels from entering the intersection to starting to travel along the other lane is set as the entry range.

9. The remote support device includes: acquiring the image from an external image capturing device installed on a road along which the target vehicle is traveling; The remote assistance method according to claim 1 , further comprising the step of calculating, as the delay time, a time from when the outside-vehicle image capturing device captures the image to when the image is displayed on the display device.

10. The remote support device includes: The remote assistance method according to any one of claims 1 to 9, further comprising at least one of displaying a frame around the watched vehicle and displaying the other vehicle with a higher transparency than the watched vehicle.

11. The remote support device includes: obtaining a video showing the images in chronological order; execute a first editing process to display a display portion corresponding to the gaze range having a color different from the surroundings in the video, and / or execute a second editing process to display a line in the video at a position corresponding to the opposite end of the gaze range in the traveling direction of the other vehicle, among the ends of the gaze range; The remote assistance method according to claim 1 , further comprising displaying to the operator the moving image on which at least one of the first editing process and the second editing process has been executed.

12. A remote assistance device that is located outside a target vehicle and assists in driving control of the target vehicle, a determination unit that, when the target vehicle enters a lane different from the lane in which the target vehicle is currently traveling, determines whether or not there is another vehicle traveling in the other lane toward an entry range into which the target vehicle will enter; a calculation unit that calculates a delay time from when an image of the surroundings of the target vehicle is captured by an imaging device of the target vehicle until an image of the surroundings of the target vehicle is displayed to an operator who operates the remote support device, when the determination unit determines that the other vehicle traveling toward the entry range is present; and A setting unit that sets a gaze range by expanding the entry range on the opposite side of the traveling direction of the other vehicle by a length based on the traveling speed of the other vehicle and the delay time calculated by the calculation unit; A remote assistance device comprising: a display unit that outputs to a display device a display indicating to the operator that the other vehicle traveling within the gaze range set by the setting unit is a gazed vehicle.

Citation Information

Patent Citations

  • Remote support system, on-vehicle device, remote support method, and remote support program

    JP2021068132A