Driving support method and driving support device
By calculating delay times and preventing unsuitable autonomous driving control based on remote support information, the method ensures that vehicle operations align with the current driving scene, addressing the issue of delayed instructions in autonomous vehicle support systems.
Patent Information
- Application Number
- JP2023071675
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-25
- Publication Date
- 2024-11-07
AI Technical Summary
Existing remote support systems for autonomous vehicles do not adequately address the issue of executing instructions that are not suitable for the current driving scene due to communication delays between the autonomous driving support center and the vehicle.
A method and device that calculate the delay time between receiving support information from a remote source and determine if it exceeds a predetermined threshold, thereby preventing the execution of autonomous driving control if the delay is excessive, ensuring the instructions align with the current driving scene.
This approach effectively suppresses the execution of unsuitable remote support by preventing autonomous driving control during communication delays, maintaining vehicle safety and operational suitability.
Smart Images

Figure 2024157355000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a driving assistance method and a driving assistance device. [Background technology]
[0002] A remote assistance system is known that limits the remote assistance that an operator can provide to an autonomous vehicle based on at least one of the communication state between an autonomous driving assistance center and the autonomous vehicle and the environmental state around the autonomous vehicle (Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2021-68132 Summary of the Invention [Problem to be solved by the invention]
[0004] In the above conventional technology, the remote assistance that the operator of the autonomous driving assistance center can perform is limited depending on the communication state between the autonomous driving assistance center and the autonomous vehicle, but the autonomous driving control of the autonomous vehicle is not limited. Therefore, in the above conventional technology, when there is a delay in communication from the autonomous driving assistance center to the autonomous vehicle, when an instruction is received from the autonomous driving assistance center, there is a problem that remote assistance that is not suitable for the current driving scene of the autonomous vehicle is executed.
[0005] The problem that the present invention aims to solve is to provide a driving assistance method and a driving assistance device that can prevent remote assistance that is not suitable for the current driving scene of the vehicle from being executed when a delay occurs in communication. [Means for solving the problem]
[0006] In the present invention, when assistance information is received from a remote assistance device outside the vehicle, a delay time is calculated from the difference between the transmission time when the assistance information is transmitted and the reception time when the assistance information is received, and if it is determined that the delay time is longer than a predetermined time, autonomous driving control of the vehicle based on the assistance information is not executed, thereby solving the above problem. Effect of the Invention
[0007] According to the present invention, when a delay occurs in communication, it is possible to prevent remote assistance that is not suitable for the current driving scene of the vehicle from being executed. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram showing an example of an embodiment of a remote assistance system according to the present invention. [Diagram 2] 2 is a plan view showing an example of a driving scene in which driving assistance is performed by the remote assistance system shown in FIG. 1 (part 1). FIG. [Diagram 3] 2 is a plan view showing an example of a driving scene in which driving assistance is performed by the remote assistance system shown in FIG. 1 (part 2). FIG. [Figure 4] 3 is a plan view showing an example of a driving scene in which driving assistance is performed by the remote assistance system shown in FIG. 1 (part 3). FIG. [Diagram 5] 4 is a plan view showing an example of a driving scene in which driving assistance is performed by the remote assistance system shown in FIG. 1 (part 4). FIG. [Figure 6] 2 is a flowchart showing an example of a processing procedure in the remote support system of FIG. 1. [Figure 7] 13 is a flowchart showing another example of the processing procedure in the remote support system of FIG. 1 (part 1). [Figure 8] 13 is a flowchart showing another example of the processing procedure in the remote support system of FIG. 1 (part 2). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0010] [Configuration of remote support system] 1 is a block diagram showing a remote assistance system 1 according to the present invention. The remote assistance system 1 is a system in which an operator outside the vehicle assists in driving the vehicle. For example, when a vehicle running under autonomous driving control is no longer able to continue running under autonomous driving control, an operator outside the vehicle remotely operates the vehicle using the remote assistance system 1. The outside of the vehicle is not particularly limited as long as it is a location away from the body of the vehicle, and may be a remote location such as a management center that centrally manages the running of the vehicle.
[0011] Autonomous driving control refers to autonomously controlling the driving operation of a vehicle using a driving assistance device of the vehicle, and the driving operation includes all driving operations such as acceleration, deceleration, starting, stopping, and steering. Furthermore, autonomously controlling driving operations refers to the driving assistance device controlling the driving operation using the device of the vehicle. The driving assistance device controls these driving operations within a predetermined range, and driving operations that are not controlled by the driving assistance device are manually operated by the driver. When the vehicle runs by manual driving by the driver without relying on autonomous driving control, the driving assistance device does not autonomously control the driving operation, and the driving operation of the vehicle is controlled by the driver's operation.
[0012] As shown in Fig. 1, the remote support system 1 includes a vehicle 10 and a remote support device 30. The vehicle 10 and the remote support device 30 exchange information with each other via a server (not shown). The vehicle 10, the remote support device 30, and the server are connected to a network for exchanging information with each other. The network refers to a telecommunication network such as the Internet or a LAN (local area network), and wired communication, wireless communication, mobile communication, etc. can be used for communication.
[0013] The vehicle 10 includes an imaging device 11, a distance measuring device 12, a vehicle state detection device 13, a navigation device 14, a vehicle control device 15, and a driving support device 20. These devices are connected by a CAN (Controller Area Network) or other in-vehicle LAN and can transmit and receive information to and from each other.
[0014] The imaging device 11 is a device that captures images of objects around the vehicle 10, and is, for example, a camera equipped with an imaging element such as a CCD, an infrared camera, etc. In order to reduce blind spots where an object cannot be captured, a plurality of imaging devices 11 are arranged on the front grill, side mirrors, rear bumper, etc. of the vehicle 10.
[0015] The distance measuring device 12 is a device that detects the relative distance and relative speed between the vehicle and an object, and is a laser radar, a millimeter wave radar, a LiDAR (light detection and ranging) unit, etc. In order to reduce blind spots where an object cannot be detected, a plurality of distance measuring devices 12 are provided on one vehicle.
[0016] The objects detected by the imaging device 11 and the distance measuring device 12 are lane boundaries, center lines, road markings, medians, guard rails, curbs, road signs, traffic lights, crosswalks, etc. The objects also include obstacles that may affect the travel of the vehicle 10, such as automobiles (other vehicles) other than the vehicle 10, motorcycles, bicycles, pedestrians, etc.
[0017] The detection results of the imaging device 11 and the distance measuring device 12 are acquired by the driving assistance device 20 as necessary. The driving assistance device 20 recognizes objects and the driving environment around the vehicle 10 from the acquired detection results. In addition, the driving assistance device 20 may integrate or synthesize (so-called sensor fusion) the detection results of the imaging device 11 and the distance measuring device 12.
[0018] The vehicle state detection device 13 is a device that detects the running state of the vehicle 10, and includes sensors for detecting the running state, such as a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, and a steering angle sensor. Any known sensor can be used without any particular limitation, and the arrangement and number of sensors can be appropriately set within a range in which the running state of the vehicle can be appropriately detected. The detection results of each sensor are acquired by the driving assistance device 20 as necessary.
[0019] The navigation device 14 is a device that refers to map information such as a high-precision map and calculates a driving route from the current position of the vehicle 10 detected by a positioning system such as a GPS (Global Positioning System) unit to a destination set by a user. The driving route includes at least information on the road on which the vehicle 10 is traveling, the driving lane, and the traveling direction of the vehicle 10, and is displayed, for example, linearly. The calculated driving route is acquired by the driving assistance device 20. The high-precision map is also called an HD (High-Definition) map.
[0020] The vehicle control device 15 is an on-board computer such as an electronic control unit (ECU), and electronically controls on-board devices that govern the driving of the vehicle 10. The vehicle control device 15 autonomously controls the operation of the drive device and steering device of the vehicle 10 in response to a control signal input from the driving assistance device 20.
[0021] The driving support device 20 is a device that controls the traveling of the vehicle 10 by controlling and cooperating with each device of the vehicle 10, and drives the vehicle 10 to a set destination. The driving support device 20 is, for example, a computer, and includes a CPU (Central Processing Unit) that is a processor, a ROM (Read Only Memory) that stores programs, and a RAM (Random Access Memory) that functions as an accessible storage device. The CPU is an operating circuit that executes the programs stored in the ROM and realizes the functions of the driving support device 20.
[0022] The remote assistance device 30 is a device used by an operator outside the vehicle 10 to assist in driving the vehicle 10, and includes a display unit 31 and an assistance unit 32. These devices are connected by a LAN or the like so as to be able to transmit and receive information to and from each other.
[0023] The display unit 31 is a device that displays images to the operator who operates the remote support device 30, and examples of such devices include a liquid crystal display and a projector. The images include moving images and still images. The operator will be described in detail later.
[0024] The support unit 32 is a device into which the operator of the remote support device 30 inputs support information necessary for driving support of the vehicle 10. The support unit 32 is composed of a touch panel, a keyboard, a steering wheel, an accelerator pedal, a brake pedal, and the like. The touch panel may form a touch screen together with a display (display unit 31). Details of the support information will be described later.
[0025] [Function block functions] The driving assistance device 20 has a driving assistance function that causes the vehicle 10 to drive autonomously based on assistance information received from a remote assistance device 30 outside the vehicle 10. A program stored in the ROM of the driving assistance device 20 is a program for realizing the driving assistance function, and the driving assistance function is realized by the CPU of the driving assistance device 20 executing the program stored in the ROM. Fig. 1 shows a transmission unit 21, a reception unit 22, a calculation unit 23, a determination unit 24, and a control unit 25, which are functional blocks that realize the driving assistance function, extracted for convenience.
[0026] The functions of each functional block shown in FIG. 1 will be described below with reference to FIG.
[0027] Fig. 2 is a plan view showing an example of a driving scene in which the driving assistance device 20 autonomously controls the driving of a vehicle by using the driving assistance function. The road shown in Fig. 2 is a two-lane road having lanes L1 and L2. A vehicle traveling on lane L1 travels from the left side to the right side of the drawing, and a vehicle traveling on lane L2 travels from the right side to the left side of the drawing. Hereinafter, lane L1 is also referred to as the own lane, and lane L2 is also referred to as the adjacent lane or the oncoming lane.
[0028] In the driving scene shown in Fig. 2, the vehicle 10 is driving at position P1 in lane L1 toward a destination (not shown) on the right side of the drawing. Another vehicle Vx is parked at position Px in lane L1, and another vehicle Vy is driving at position Py in lane L2. Note that, although the vehicle 10 is driving by autonomous driving control in the driving scene shown in Fig. 2, the driving assistance function of this embodiment can also be applied to driving scenes in which the vehicle 10 is driving manually by the driver.
[0029] In order to continue traveling under the autonomous driving control, the vehicle 10 needs to enter the lane L2 to avoid the other vehicle Vx, and then return to the lane L1 after avoiding the other vehicle Vx. However, if an obstacle in the lane L2 cannot be recognized from the detection results of the imaging device 11 and the distance measuring device 12 due to a blind spot in the detection range, or if the vehicle 10 does not have a function to determine whether it can enter the lane L2, the driving assistance device 20 cannot determine whether the vehicle 10 can enter the lane L2, and the vehicle 10 stops (gets stuck) behind the other vehicle Vx. In this case, the driving assistance device 20 cannot continue traveling under the autonomous driving control, and therefore transmits a request for remote assistance to the remote assistance device 30.
[0030] The transmission unit 21 has a function of transmitting a request for remote assistance to the remote assistance device 30. Remote assistance refers to an operator outside the vehicle 10 remotely operating the vehicle 10 from outside the vehicle 10 via the driving assistance device 20, or the operator providing information to the driving assistance device 20 or the driver of the vehicle 10. The operator is, for example, located in a management center that manages the traveling of the vehicles 10 constituting the remote assistance system 1, and provides necessary information to the vehicle 10 in response to a request from the vehicle 10 or regardless of a request from the vehicle 10, and remotely operates the vehicle 10.
[0031] The request for remote assistance includes information indicating that the autonomous driving control of the vehicle 10 has stopped, and information necessary for the operator to provide driving assistance for the vehicle 10. For example, the request for remote assistance includes the driving status of the vehicle 10. The driving status of the vehicle 10 is, for example, a driving scene of the vehicle 10, and includes the driving state of the vehicle 10, a driving plan for the vehicle 10, and the driving environment around the vehicle 10.
[0032] The driving state of the vehicle 10 includes the vehicle speed, acceleration, yaw rate, steering angle, etc. of the vehicle 10, and is obtained from the vehicle state detection device 13. The driving plan of the vehicle 10 is information for the vehicle 10 to travel to a destination, and includes a driving route, etc. The driving plan is obtained from the navigation device 14. The driving environment around the vehicle 10 refers to the state of objects (particularly obstacles) around the vehicle 10, and is obtained from the detection results of the imaging device 11 and the distance measuring device 12 using a method such as pattern matching.
[0033] In the driving scene shown in Fig. 2, the driving assistance device 20 acquires the vehicle speed of the vehicle 10 from the vehicle state detection device 13 by the function of the transmission unit 21, and transmits to the assistance unit 32 the information that the vehicle 10 is stopped together with the vehicle speed of the vehicle 10. The driving assistance device 20 also transmits to the assistance unit 32 location information of the destination acquired from the navigation device 14 and the set driving route. Furthermore, the driving assistance device 20 acquires an image of the surroundings of the vehicle 10 including the lanes L1 and L2 from the imaging device 11, and transmits it to the display unit 31. Similarly, the driving assistance device 20 acquires the distance between the vehicle 10 and another vehicle Vx from the distance measurement device 12, and transmits it to the assistance unit 32.
[0034] When transmitting a request for remote assistance, the driving assistance device 20 may stop the vehicle 10, and after the vehicle 10 has stopped, transmit the request for remote assistance to the remote assistance device 30. In addition, the driving assistance device 20 may instruct the operator operating the remote assistance device 30 to cause the display unit 31 to display the transmitted image.
[0035] The remote assistance device 30 that has received the request for remote assistance prompts the operator to perform the remote assistance, for example, by displaying an image on the display unit 31. For example, the operator notifies the driver of the vehicle 10 from the assistance unit 32 of the start of remote assistance, and when the operator obtains consent from the driver, starts remote assistance by the assistance unit 32. After grasping information such as the image displayed on the display unit 31, the operator transmits assistance information necessary for remote assistance to the driving assistance device 20.
[0036] The receiving unit 22 has a function of receiving support information from the remote support device 30. The support information is information necessary for remote support inputted to the support unit 32 by the operator, and includes an execution instruction for the driving support device 20 to execute a certain driving operation by autonomous driving control, and judgment information for the driving support device 20 to set the driving operation by autonomous driving control. The driving operation related to the execution instruction is not particularly limited. The execution instruction includes, for example, an instruction to start the vehicle 10 from a stopped state, an instruction to change lanes of the vehicle 10 from the own lane to an adjacent lane, an instruction to flash the turn indicator of the vehicle 10, and the like.
[0037] Meanwhile, the judgment information includes, in addition to the driving conditions of the vehicle 10, at least one of destination information, driving route information, and information on the type of obstacle present around the vehicle 10. The destination information and driving route information include at least one of information input by the operator to the support unit 32 and information acquired from the navigation device 14. The information on the type of obstacle present around the vehicle 10 includes at least one of information acquired from the detection results of the imaging device 11 and the distance measuring device 12 and information acquired by the remote support device 30 from a camera (not shown). When the operator updates the destination and driving route, the driving support device 20 acquires the new destination information and new driving route information from the support unit 32.
[0038] The type of an obstacle is an attribute of the obstacle, and refers to, for example, the type, size, and action of the obstacle, and the effect it has on the travel of the vehicle 10. For example, the type of the other vehicle Vx shown in Fig. 2 may be a medium-sized vehicle, stationary because it is parked, and having a large effect on the travel of the vehicle 10 because it needs to be avoided.
[0039] The calculation unit 23 has a function of calculating a delay time from when the remote support device 30 transmits support information to when the driving support device 20 receives the support information. The driving support device 20 acquires the transmission time when the remote support device 30 transmits the support information together with the support information, and calculates the delay time from the difference between the transmission time when the remote support device 30 transmits the support information and the reception time when the driving support device 20 receives the support information.
[0040] The determination unit 24 has a function of determining whether the delay time calculated by the function of the calculation unit 23 is longer than a predetermined time. The predetermined time can be set as an appropriate time within a range that can suppress the execution of a driving operation that is not suitable for the driving scene of the vehicle 10 when a delay or interruption occurs in the communication between the driving support device 20 and the remote support device 30, and is, for example, 1 to 5 seconds.
[0041] The control unit 25 has a function of not executing autonomous driving control of the vehicle 10 based on the support information when the function of the determination unit 24 determines that the delay time is longer than a predetermined time. When the delay time from when the support information is transmitted from the remote support device 30 to when the driving support device 20 receives the support information is longer than a predetermined time, the driving support device 20 outputs a control signal to the vehicle control device 15 not to execute autonomous driving control based on the support information, since there is a risk that the driving situation of the vehicle 10 changes during the delay time and a remote operation that is not suitable for the driving scene of the vehicle 10 is executed. On the other hand, when the delay time is determined to be equal to or shorter than the predetermined time, the control unit 25 outputs a control signal to the vehicle control device 15 to execute autonomous driving control based on the support information.
[0042] In the driving scene shown in Fig. 2, the operator transmits an execution instruction to move the vehicle 10 to the right in the travel direction within the lane L1 from the support unit 32. The driving support device 20 receives this execution instruction using the function of the receiving unit 22, and acquires the transmission time at which this execution instruction was transmitted from the remote support device 30. Then, the delay time is calculated from the difference between the transmission time at which the execution instruction was transmitted and the reception time at which the execution instruction was received.
[0043] If the calculated delay time is equal to or less than a predetermined time, the vehicle 10, which is autonomously driven by remote control of the operator, drives along the driving trajectory T1 to position P2 shown in Fig. 3. The autonomous driving control of the vehicle 10 at this time is performed by the function of the control unit 25. This allows the imaging device 11 to detect another vehicle Vy traveling in lane L2. On the other hand, if the calculated delay time is longer than a predetermined time, the vehicle 10 does not drive by remote control, and remains stopped at position P1.
[0044] Furthermore, when the operator is presented with information in the request for remote assistance that it is not possible to determine whether the other vehicle Vx is parked or stopped, and therefore it is not possible to determine whether it is necessary to avoid the other vehicle Vx, the operator transmits information to the driving assistance device 20 that the other vehicle Vx is a parked vehicle as the type of obstacle. In other words, the operator transmits information that the other vehicle Vx will not move for a while, and that the operator cannot resume traveling even if the operator waits behind the other vehicle Vx. In contrast, when the other vehicle Vx is stopped, the operator transmits information that the operator can resume traveling if the operator waits behind the other vehicle Vx.
[0045] The vehicle 10, which has traveled to position P2 by remote control, stops at position P2 as shown in Fig. 4. The operator recognizes another vehicle Vy traveling on lane L2 from the image of the imaging device 11 displayed on the display unit 31. The operator stops the vehicle 10 at position P2 until the other vehicle Vy travels from position Py to position Pz and passes by the side of the vehicle 10.
[0046] After the other vehicle Vy passes beside the vehicle 10, the operator transmits an execution instruction from the support unit 32 to enter the lane L2 to avoid the other vehicle Vx. The driving support device 20 acquires the transmission time at which this execution instruction was transmitted from the remote support device 30 by the function of the receiving unit 22. Then, the delay time is calculated from the difference between the transmission time at which the execution instruction was transmitted and the reception time at which the execution instruction was received.
[0047] If the calculated delay time is equal to or less than a predetermined time, the operator remotely controls the vehicle 10 to autonomously travel along the travel path T2 from position P2 to position P3 shown in Fig. 5. On the other hand, if the calculated delay time is longer than a predetermined time, the vehicle 10 does not travel by remote control, and remains stopped at position P2.
[0048] When the assistance information is an execution instruction, if the driving assistance device 20 determines that the delay time is longer than a predetermined time before starting the driving operation related to the execution instruction, the driving assistance device 20 may maintain the driving state at the time of receiving the execution instruction. On the other hand, if the driving assistance device 20 determines that the delay time is longer than a predetermined time after starting the driving operation related to the execution instruction, the driving assistance device 20 does not immediately stop the driving operation, but continues the driving operation and detects a parking space in which the vehicle 10 can be parked. Then, the driving assistance device 20 parks the vehicle 10 in the detected parking space. Alternatively, the driving assistance device 20 may continue the driving operation and request the remote assistance device 30 to update the assistance information.
[0049] The driving assistance device 20 may set in advance an exceptional instruction to execute autonomous driving control based on the assistance information even when it is determined that the delay time of the assistance information is longer than a predetermined time. The exceptional instruction is an instruction to execute a driving operation that needs to be executed with priority over other driving operations, and is included in the execution instruction described later. Examples of driving operations that need to be executed with priority include stopping the vehicle 10 to deal with an accident that has occurred in front of the vehicle 10, turning on lights to deal with a sudden change in weather, and the like. In addition, the exceptional instruction may be transmitted from the remote assistance device 30 regardless of a request for remote assistance from the user of the vehicle 10.
[0050] When the driving assistance device 20 determines that the delay time is longer than a predetermined time, the driving assistance device 20 determines whether the assistance information includes an exception instruction. When the driving assistance device 20 determines that the assistance information includes an exception instruction, the driving assistance device 20 executes autonomous driving control based on the assistance information. On the other hand, when the driving assistance device 20 determines that the assistance information does not include an exception instruction, the driving assistance device 20 does not execute autonomous driving control based on the assistance information.
[0051] In addition, the driving assistance device 20 may transmit a request for remote assistance to the remote assistance device 30, determine whether the assistance information corresponds to the transmitted request for remote assistance, and when it is determined that the assistance information does not correspond to the request for remote assistance, execute autonomous driving control based on the assistance information even when it is determined that the delay time is longer than a predetermined time. The information corresponding to the request for remote assistance is information for realizing the assistance requested by the user to the operator, for example, an execution instruction for realizing the driving operation requested by the user to the operator.
[0052] For example, in the driving scene shown in Fig. 2, after transmitting a request for remote assistance to avoid another vehicle Vx, assistance information to stop at position P1 is received from the remote assistance device 30. In this case, since the assistance information to stop at position P1 is not assistance information (execution instruction) for realizing the request to avoid the other vehicle Vx, the driving assistance device 20 stops at position P1 even when it determines that the delay time is longer than the predetermined time.
[0053] When the driving assistance device 20 determines that the delay time is longer than a predetermined time, the driving assistance device 20 may determine whether the assistance information is an execution instruction or judgment information. When the driving assistance device 20 determines that the assistance information is an execution instruction, the driving assistance device 20 does not execute autonomous driving control based on the assistance information. On the other hand, when the driving assistance device 20 determines that the assistance information is judgment information, the driving assistance device 20 executes autonomous driving control using the received assistance information. In this case, the driving assistance device 20 may use the driving situation of the vehicle 10 and the driving plan acquired from the navigation device 14 and / or the remote assistance device 30 in combination.
[0054] 5, when an instruction to run along the running path T2 is received from the remote support device 30, if it is determined that the delay time is longer than a predetermined time, the autonomous running control to run along the running path T2 is not executed, and the vehicle 10 is stopped at position P2. On the other hand, when the determination information that the vehicle 10 can enter the lane L2 is received from the remote support device 30, the autonomous running control is executed based on the detection results of the imaging device 11 and the distance measuring device 12 and the running route acquired from the navigation device 14.
[0055] When the driving assistance device 20 transmits a request for remote assistance to the remote assistance device 30 before receiving assistance information, the driving assistance device 20 stores the driving conditions of the vehicle 10 at the time the request was transmitted. Then, when it is determined that the delay time is longer than a predetermined time, it may determine whether the driving conditions of the vehicle 10 at the time the assistance information was received are suitable for executing remote assistance at the request, and when it is determined that the driving conditions of the vehicle 10 at the time the assistance information was received are suitable for executing remote assistance at the request, it may execute autonomous driving control based on the assistance information.
[0056] The driving conditions of the vehicle 10 when the support information is received are suitable for performing remote support upon request means that there is no change in the driving scene of the vehicle 10 between the time the request for remote support is transmitted and the time the support information is received. For example, if the positional relationship between the vehicle 10 and other vehicles does not change between the time the request for remote support is transmitted and the time the support information is received, it is determined that the driving conditions of the vehicle 10 when the support information is received are suitable for performing remote support upon request. In addition, if performing remote support upon request upon receiving the support information would not interfere with traffic of other vehicles, it may be determined that the driving conditions of the vehicle 10 when the support information is received are suitable for performing remote support upon request.
[0057] For example, in the driving scene shown in Fig. 2, when a request for remote assistance to avoid another vehicle Vx is transmitted, the driving status of the vehicle 10 at the time the request was transmitted is stored in a server or the like. As an example, it is stored that the other vehicle Vx in front of the vehicle 10 is parked. Then, when it is determined that the delay time is longer than a predetermined time, the driving status of the vehicle 10 at the time the assistance information was received is acquired from the imaging device 11 or the like. In this case, if the other vehicle Vx is still parked when the assistance information is received, it is determined that the stored driving status is the same as the current driving status, and the vehicle 10 is caused to travel along the driving trajectory T1.
[0058] On the other hand, if it is determined that the driving situation of the vehicle 10 when the support information is received is not suitable for executing remote support in response to a request, the autonomous driving control based on the support information is not executed. For example, in the driving scene shown in FIG. 2, if it is determined that the delay time is longer than a predetermined time, the driving situation of the vehicle 10 when the support information is received is acquired from the imaging device 11 or the like, and if it is detected that the other vehicle Vx has already started moving when the support information is received, the positional relationship between the vehicle 10 and the other vehicle Vx has changed, so the driving support device 20 determines that the driving situation of the vehicle 10 when the support information is received is not suitable for executing remote support in response to a request. In this case, the driving support device 20 causes the vehicle 10 to travel along the lane L1, rather than along the travel trajectory T1.
[0059] [Processing in remote support system]
[0060] 6 to 8, a procedure for information processing by the driving support device 20 will be described. Fig. 6 is an example of a flowchart showing information processing executed in the remote support system 1 of this embodiment. The processing described below is executed by a processor (CPU) included in the driving support device 20.
[0061] First, in step S1, the function of the receiving unit 22 receives support information from the remote support device 30. In step S2, the function of the calculation unit 23 calculates a delay time from the difference between the transmission time acquired from the remote support device 30 and the reception time when the support information was received. In step S3, the function of the determination unit 24 determines whether the delay time is longer than a predetermined time.
[0062] If it is determined that the delay time is equal to or shorter than the predetermined time, the process proceeds to step S4, where autonomous driving control based on the assistance information is executed by the function of the control unit 25. On the other hand, if it is determined that the delay time is longer than the predetermined time, the process proceeds to step S5, where autonomous driving control based on the assistance information is not executed.
[0063] 7 and 8 are another example of a flowchart showing information processing executed in the remote assistance system 1 of this embodiment. The processing described below is executed by a processor (CPU) included in the driving assistance device 20 at a predetermined time interval (for example, every 0.1 to 1 second).
[0064] 7, the function of the determination unit 24 acquires the driving status of the vehicle 10 from the detection results of the imaging device 11 detected at a predetermined time interval, and in the following step S12, it is determined whether or not the autonomous driving control of the vehicle 10 can be continued. If it is determined that the autonomous driving control of the vehicle 10 can be continued, the process proceeds to step S11. On the other hand, if it is determined that the autonomous driving control of the vehicle 10 cannot be continued, for example, when the vehicle 10 cannot avoid an obstacle, the process proceeds to step S13, and the function of the control unit 25 causes the vehicle 10 to stop.
[0065] In step S14, a request for remote support is transmitted to the remote support device 30 using the function of the transmission unit 21, and in the following step S15, the traveling status of the vehicle 10 is stored in a server (not shown) or the like. In step S16, an image is acquired from the imaging device 11 using the function of the transmission unit 21, and in the following step S17, the image is transmitted to the remote support device 30.
[0066] In step S18, support information is received from the remote support device 30 using the function of the receiving unit 22, and in the following step S19, the transmission time is acquired from the remote support device 30. In step S20, the reception time when the support information was received is acquired using the function of the calculation unit 23, and in the following step S21, the delay time is calculated from the difference between the transmission time acquired from the remote support device 30 and the reception time when the support information was received.
[0067] 8, in step S22, the function of the determination unit 24 is used to determine whether the delay time is longer than a predetermined time. If it is determined that the delay time is equal to or shorter than the predetermined time, the process proceeds to step S28, and the function of the control unit 25 is used to output a control instruction based on the assistance information to the vehicle control device 15, thereby executing autonomous driving control. On the other hand, if it is determined that the delay time is longer than the predetermined time, the process proceeds to step S23.
[0068] In step S23, it is determined whether or not the support information is an execution instruction by the function of the determination unit 24. If it is determined that the support information is not an execution instruction, the process proceeds to step S28. On the other hand, if it is determined that the support information is an execution instruction, the process proceeds to step S24.
[0069] In step S24, the determination unit 24 determines whether the support information corresponds to the request for remote support. If it is determined that the support information does not correspond to the request for remote support or that the request for remote support has not been transmitted, the process proceeds to step S28. On the other hand, if it is determined that the support information corresponds to the request for remote support, the process proceeds to step S25, where the current traveling status of the vehicle 10 is acquired from the detection result of the imaging device 11, etc.
[0070] Then, in step S26, the function of the determination unit 24 is used to determine whether or not the current driving conditions of the vehicle 10 are suitable for executing remote assistance in response to a request. If it is determined that the current driving conditions of the vehicle 10 are suitable for executing remote assistance in response to a request, such as when the driving environment around the vehicle 10 (for example, the positional relationship with other vehicles) has not changed, the process proceeds to step S28. On the other hand, if it is determined that the current driving conditions of the vehicle 10 are not suitable for executing remote assistance in response to a request, the process proceeds to step S27, and autonomous driving control based on the assistance information is not executed.
[0071] [Embodiments of the present invention] As described above, according to the present embodiment, in a driving assistance method executed by the driving assistance device 20 of the vehicle 10, which causes the vehicle 10 to autonomously drive based on assistance information received from a remote assistance device 30 outside the vehicle 10, the driving assistance device 20, when receiving the assistance information from the remote assistance device 30, calculates a delay time from the difference between a transmission time at which the remote assistance device transmits the assistance information and a reception time at which the driving assistance device 20 receives the assistance information, determines whether the delay time is longer than a predetermined time, and when it is determined that the delay time is longer than the predetermined time, does not execute autonomous driving control of the vehicle 10 based on the assistance information. This makes it possible to suppress remote assistance that is not suitable for the current driving scene of the vehicle 10 from being executed when a delay occurs in communication.
[0072] According to the driving assistance method of the present embodiment, the driving assistance device 20 presets an exceptional instruction for executing the autonomous driving control based on the assistance information even when it is determined that the delay time is longer than the predetermined time, and when it is determined that the delay time is longer than the predetermined time, it determines whether the assistance information includes the exceptional instruction, and when it is determined that the assistance information includes the exceptional instruction, it executes the autonomous driving control based on the assistance information, and when it is determined that the assistance information does not include the exceptional instruction, it does not execute the autonomous driving control based on the assistance information. This makes it possible to appropriately execute driving operations that need to be executed with priority over other driving operations.
[0073] According to the driving assistance method of the present embodiment, the driving assistance device 20 transmits a request for remote assistance to the remote assistance device 30, determines whether the assistance information corresponds to the request, and when it determines that the assistance information does not correspond to the request, executes the autonomous driving control based on the assistance information even if it determines that the delay time is longer than the predetermined time. This allows driving operations that need to be executed with priority over other driving operations to be executed appropriately.
[0074] Furthermore, according to the driving assistance method of this embodiment, when the driving assistance device 20 determines that the delay time is longer than the predetermined time, it determines whether the assistance information is an execution instruction to perform a driving operation by the autonomous driving control or judgment information for setting the driving operation by the autonomous driving control, and when it determines that the assistance information is the execution instruction, it does not execute the autonomous driving control based on the assistance information, and when it determines that the assistance information is the judgment information, it executes the autonomous driving control using the assistance information. This makes it possible to determine whether or not to execute autonomous driving control depending on the content of the assistance information.
[0075] Moreover, according to the driving assistance method of the present embodiment, the execution instruction includes an instruction for the vehicle 10 to start moving from a stopped state, thereby making it possible to resolve the state in which the vehicle 10 is stuck.
[0076] According to the driving assistance method of the present embodiment, the driving assistance device 20 acquires, as the determination information, at least one of destination information, driving route information, and information on the type of obstacle present around the vehicle 10 from the remote assistance device 30. This allows the driving assistance device 20 to perform appropriate driving operations.
[0077] According to the driving assistance method of the present embodiment, when the driving assistance device 20 transmits a request for remote assistance to the remote assistance device 30 before receiving the assistance information, the driving assistance device 20 stores the driving situation of the vehicle 10 at the time of transmitting the request, and when it is determined that the delay time is longer than the predetermined time, it determines whether the driving situation of the vehicle 10 at the time of receiving the assistance information is suitable for executing the remote assistance at the request, and when it is determined that the driving situation of the vehicle 10 at the time of receiving the assistance information is suitable for executing the remote assistance at the request, it executes the autonomous driving control based on the assistance information, and when it is determined that the driving situation of the vehicle 10 at the time of receiving the assistance information is not suitable for executing the remote assistance at the request, it does not execute the autonomous driving control based on the assistance information. This makes it possible to suppress remote assistance that is not suitable for the current driving scene of the vehicle 10 from being executed when the driving situation of the vehicle 10 changes.
[0078] According to the driving assistance method of the present embodiment, when the driving assistance device 20 determines that the driving situation of the vehicle 10 at the time of receiving the assistance information is suitable for executing the remote assistance in response to the request, the driving assistance device 20 determines whether the assistance information is an execution instruction to perform a driving operation by the autonomous driving control or judgment information for setting the driving operation by the autonomous driving control, and when it determines that the assistance information is the execution instruction, it does not execute the autonomous driving control based on the assistance information, and when it determines that the assistance information is the judgment information, it executes the autonomous driving control using the assistance information. This makes it possible to determine whether or not to execute autonomous driving control depending on the content of the assistance information.
[0079] Moreover, according to the driving assistance method of the present embodiment, the traveling situation of the vehicle 10 includes the traveling state of the vehicle 10, the traveling plan of the vehicle 10, and the traveling environment around the vehicle 10. This makes it possible to more appropriately recognize the traveling situation of the vehicle 10.
[0080] According to the present embodiment, the driving support device 20 that causes the vehicle 10 to autonomously drive based on support information received from a remote support device 30 outside the vehicle 10 includes a calculation unit 23 that calculates a delay time from the difference between a transmission time at which the remote support device 30 transmits the support information and a reception time at which the driving support device 20 receives the support information when the support information is received from the remote support device 30, a determination unit 24 that determines whether the delay time calculated by the calculation unit 23 is longer than a predetermined time, and a control unit 25 that outputs an instruction not to execute autonomous driving control of the vehicle 10 based on the support information when the determination unit 24 determines that the delay time is longer than the predetermined time. This makes it possible to prevent remote support that is not suitable for the current driving scene of the vehicle 10 from being executed when a delay occurs in communication. [Explanation of symbols]
[0081] 1. Remote support system 10. Vehicle 11...Imaging device 12…Distance measuring device 13... Vehicle status detection device 14. Navigation device 15...Vehicle control device 20...Driving assistance device 21...Transmitter 22...Receiver 23...Calculation section 24…Judgment section 25...Control section 30...Remote support device 31...Display section 32…Support Department L1, L2...Lane P1,P2,P3,Px,Py,Pz…Position T1, T2…Travel trajectory Vx, Vy...other vehicles
Claims
1. A driving assistance method executed by a driving assistance device of a vehicle, the driving assistance method comprising: The driving assistance device includes: When the assistance information is received from the remote assistance device, a delay time is calculated from a difference between a transmission time at which the remote assistance device transmits the assistance information and a reception time at which the driving assistance device receives the assistance information; determining whether the delay time is greater than a predetermined time; When it is determined that the delay time is longer than the predetermined time, autonomous driving control of the vehicle based on the assistance information is not executed.
2. The driving assistance device includes: An exception instruction is set in advance to execute the autonomous driving control based on the assistance information even when it is determined that the delay time is longer than the predetermined time, When it is determined that the delay time is longer than the predetermined time, it is determined whether or not the support information includes the exception instruction; When it is determined that the support information includes the exception instruction, the autonomous driving control is executed based on the support information. The driving assistance method according to claim 1 , wherein when it is determined that the assistance information does not include the exception instruction, the autonomous driving control based on the assistance information is not executed.
3. The driving assistance device includes: Sending a request for remote assistance to the remote assistance device; determining whether the support information corresponds to the request; 3. The driving assistance method according to claim 1, further comprising: executing the autonomous driving control based on the assistance information when it is determined that the assistance information is not the information corresponding to the request, even if it is determined that the delay time is longer than the predetermined time.
4. The driving assistance device includes: When it is determined that the delay time is longer than the predetermined time, it is determined whether the support information is an execution instruction to perform a driving operation by the autonomous driving control or judgment information for setting the driving operation by the autonomous driving control; When it is determined that the support information is the execution instruction, the autonomous driving control based on the support information is not executed, The driving assistance method according to claim 1 , further comprising the step of: executing the autonomous driving control using the assistance information when it is determined that the assistance information is the determination information.
5. The driving assistance method according to claim 4 , wherein the execution instruction includes an instruction for the vehicle to start moving from a stopped state.
6. The driving assistance device includes: The driving assistance method according to claim 4 , further comprising acquiring, as the determination information, at least one of destination information, driving route information, and information on a type of obstacle present around the vehicle from the remote assistance device.
7. The driving assistance device includes: If a request for remote assistance is transmitted to the remote assistance device before receiving the assistance information, the vehicle's running status at the time of transmitting the request is stored; When it is determined that the delay time is longer than the predetermined time, it is determined whether or not the driving situation of the vehicle at the time of receiving the assistance information is suitable for executing the remote assistance in response to the request; When it is determined that the driving situation of the vehicle at the time of receiving the assistance information is suitable for executing the remote assistance in response to the request, the autonomous driving control is executed based on the assistance information; 3. The driving assistance method according to claim 1, further comprising: if it is determined that the driving conditions of the vehicle at the time when the assistance information is received are not suitable for executing the remote assistance in response to the request, not executing the autonomous driving control based on the assistance information.
8. The driving assistance device includes: When it is determined that the driving situation of the vehicle at the time of receiving the assistance information is suitable for executing the remote assistance in response to the request, it is determined whether the assistance information is an execution instruction to perform a driving operation by the autonomous driving control or judgment information for setting the driving operation by the autonomous driving control; When it is determined that the support information is the execution instruction, the autonomous driving control based on the support information is not executed, The driving assistance method according to claim 7 , further comprising the step of: executing the autonomous driving control using the assistance information when it is determined that the assistance information is the determination information.
9. The driving assistance method according to claim 7 , wherein the driving conditions of the vehicle include a driving state of the vehicle, a driving plan of the vehicle, and a driving environment around the vehicle.
10. A driving assistance device that causes a vehicle to autonomously drive based on assistance information received from a remote assistance device outside the vehicle, a calculation unit that calculates a delay time from a difference between a transmission time at which the remote support device transmits the support information and a reception time at which the driving support device receives the support information when the support information is received from the remote support device; a determination unit that determines whether the delay time calculated by the calculation unit is longer than a predetermined time; a control unit that outputs an instruction not to execute autonomous driving control of the vehicle based on the assistance information when the determination unit determines that the delay time is longer than the predetermined time.
Citation Information
Patent Citations
Remote support system, on-vehicle device, remote support method, and remote support program
JP2021068132A
Cited By
Control system and control method for autonomous ships
JP7908716B1
