Remote support system, remote support method, and non-transitory storage medium

Abstract

The present disclosure relates to a remote support system, a remote support method, and a non-transitory storage medium. The remote support method includes: issuing a remote operation request to a remote operator to remotely operate a vehicle configured for autonomous driving when it is predicted that it will be difficult to continue autonomous driving; performing a determination process to determine whether it will be difficult for the vehicle to continue autonomous driving based on a future driving trajectory of the vehicle generated based on driving environment information; determining whether to issue the remote operation request based on a determination result obtained when the determination process is performed based on a first evaluation criterion; and determining whether to correct the driving trajectory based on a determination result obtained when the determination process is performed based on a second evaluation criterion.

Description

Technical Field

[0001] The present disclosure relates to a remote support system, a remote support method, and a non-transitory storage medium for issuing a remote operation request for a vehicle to a remote operator. Background Art

[0002] Japanese Patent Application Laid-Open No. 2018-77649 discloses a technology related to a remote driving control device for enabling a remote operator to perform remote driving. In this device, a remote operator remotely controls a vehicle in response to a remote control request sent from an autonomous vehicle.

[0003] Consider the following scenario: When a vehicle operating autonomous driving is predicted to have difficulty continuing autonomous driving, a remote control request is issued and the driving plan is revised to await remote control. In this case, if the prediction of whether autonomous driving can continue is prioritized, the avoidance object can be detected and the remote control request issued at an earlier timing. However, this may result in deceleration from the earlier timing, which could compromise driving efficiency and comfort. On the other hand, if the prediction of whether autonomous driving can continue is prioritized, the timing of issuing the remote control request is expected to be delayed. In this case, the vehicle may remain stopped on the road while awaiting remote control, potentially disrupting traffic flow. Thus, there is room for improvement in balancing safety and driving efficiency when remote control requests are issued from a vehicle during autonomous driving. Summary of the Invention

[0004] The present disclosure provides a remote support system, a remote support method, and a non-transitory storage medium that can prevent a reduction in driving efficiency and issue a remote operation request in a safety-oriented manner.

[0005] A first aspect of the present disclosure is a remote support system configured to issue a remote operation request to a remote operator requesting remote operation of a vehicle configured for autonomous driving when it is predicted that it will be difficult to continue autonomous driving. The remote support system includes: a storage device storing commands; and at least one processor connected to the storage device. The at least one processor is configured to obtain driving environment information of the vehicle and to perform a determination process to determine whether it will be difficult for the vehicle to continue autonomous driving based on a future driving trajectory of the vehicle generated based on the driving environment information. The at least one processor is configured to determine whether to issue the remote operation request based on a determination result when the determination process is performed based on a first evaluation criterion, and to determine whether to correct the driving trajectory based on a determination result when the determination process is performed based on a second evaluation criterion. The determination process includes: performing the determination process based on the first evaluation criterion, which is an evaluation criterion that prioritizes vehicle safety over the second evaluation criterion; and performing the determination process based on the second evaluation criterion, which is an evaluation criterion that prioritizes vehicle driving efficiency over the first evaluation criterion.

[0006] In the first aspect, the determination process may include a predicted trajectory generation process for generating a predicted future trajectory of the avoidance object based on the driving environment information. Alternatively, the at least one processor may be configured to determine whether it is difficult for the vehicle to continue autonomous driving based on the predicted trajectory and the driving trajectory.

[0007] In the first embodiment, the evaluation criteria may include an evaluation criteria associated with the accuracy of the avoidance target. Alternatively, the determination process may include obtaining a likelihood index value indicating the accuracy of the detected object target detected based on the driving environment information, and setting the detected object target as the avoidance target if the obtained likelihood index value exceeds a predetermined determination threshold. Alternatively, in the determination process, the determination threshold of the first evaluation criteria may be lower than the determination threshold of the second evaluation criteria.

[0008] In the first aspect, the evaluation criteria may include an evaluation criteria associated with the avoidance object's compliance probability with traffic regulations. Alternatively, the predicted trajectory generation process may use the compliance probability to generate the predicted trajectory of the avoidance object. Alternatively, the first compliance probability may be lower than the second compliance probability, the first compliance probability being the compliance probability based on the first evaluation criteria, and the second compliance probability being the compliance probability based on the second evaluation criteria.

[0009] In the first embodiment, the evaluation criteria may include an evaluation criteria related to the likelihood of maintaining the avoidance object's current state. Alternatively, the predicted trajectory generation process may use the likelihood of maintaining the predicted trajectory of the avoidance object. Alternatively, the first likelihood of maintaining the predicted trajectory may be lower than the second likelihood of maintaining the predicted trajectory, the first likelihood of maintaining the predicted trajectory being the first criterion, and the second likelihood of maintaining the predicted trajectory being the second criterion.

[0010] In the first embodiment, the determination process may include an intersection determination process for determining whether the predicted trajectory and the driving trajectory intersect. Alternatively, the at least one processor may be configured to determine whether it is difficult for the vehicle to continue autonomous driving based on the determination result of the intersection determination process. Alternatively, the evaluation criteria may include an evaluation criteria related to an allowable error range in the intersection determination process. Alternatively, in the intersection determination process, the error range of the first evaluation criteria may be narrower than the error range of the second evaluation criteria.

[0011] The second scheme of the present disclosure is a remote support method. The remote support method includes: when a vehicle configured to perform autonomous driving is predicted to be difficult to continue autonomous driving, a processor issues a remote operation request to a remote operator to entrust remote operation of the vehicle; and performs a judgment process, which is a process of judging whether it is difficult for the vehicle to continue autonomous driving based on the future driving trajectory of the vehicle generated based on driving environment information. The remote support method includes: judging whether to issue a remote operation request based on a judgment result when the judgment process is performed based on a first evaluation criterion; and judging whether to correct the driving trajectory based on a judgment result when the judgment process is performed based on a second evaluation criterion. The judgment process includes: performing a judgment process based on a first evaluation criterion that emphasizes the safety of the vehicle compared to the second evaluation criterion; and performing a judgment process based on a second evaluation criterion that emphasizes the driving efficiency of the vehicle compared to the first evaluation criterion.

[0012] The third aspect of the present disclosure is a non-transitory storage medium that stores commands that can be executed by one or more processors and cause the one or more processors to perform the following functions. The functions include: issuing a remote operation request to a remote operator to request remote operation of the vehicle when a vehicle configured for autonomous driving is predicted to be difficult to continue autonomous driving; obtaining driving environment information of the vehicle; and performing a determination process that determines whether it is difficult for the vehicle to continue autonomous driving based on the future driving trajectory of the vehicle generated based on the driving environment information. The functions include: determining whether to issue a remote operation request based on a determination result when the determination process is performed based on a first evaluation criterion; and determining whether to correct the driving trajectory based on a determination result when the determination process is performed based on a second evaluation criterion. The determination process includes: performing a determination process based on the first evaluation criterion that prioritizes vehicle safety over the second evaluation criterion; and performing a determination process based on the second evaluation criterion that prioritizes vehicle driving efficiency over the first evaluation criterion.

[0013] According to the first, second, and third schemes of the present disclosure, in the determination process for determining whether to issue a remote operation request, it is possible to determine whether it is difficult for a vehicle that is performing autonomous driving to continue autonomous driving based on a first evaluation criterion that prioritizes safety. In addition, as to whether the driving trajectory should be corrected, it is possible to determine whether it is difficult for a vehicle that is performing autonomous driving to continue autonomous driving based on a second evaluation criterion that prioritizes driving efficiency. According to such a configuration, it is possible to prioritize driving efficiency when determining whether to correct the driving trajectory, and it is possible to prioritize safety when determining whether a remote operation request is necessary. As a result, a reduction in driving efficiency can be prevented, and a remote operation request can be issued in a manner that prioritizes safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Hereinafter, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, wherein like reference numerals denote like elements, and wherein:

[0015] Figure 1 This is a block diagram showing a configuration example for explaining the outline of the remote support system according to the first embodiment.

[0016] Figure 2 This is a block diagram showing an example of the configuration of an autonomous driving vehicle.

[0017] Figure 3 This is a diagram for explaining an example of a situation in which remote operation is performed in the remote support system according to the first embodiment.

[0018] Figure 4This is a diagram for explaining an example of a situation in which remote operation is performed in the remote support system according to the first embodiment.

[0019] Figure 5 This is a functional block diagram showing part of the functions of the automatic driving control device.

[0020] Figure 6 This is a flowchart of the processing performed in the automatic driving control device.

[0021] Figure 7 This is a diagram for explaining a modified example of the automatic driving control device of embodiment 1.

[0022] Figure 8 This is a diagram for explaining another modified example of the automatic driving control device of embodiment 1. DETAILED DESCRIPTION

[0023] The following describes embodiments of the present disclosure with reference to the accompanying drawings. Where numerical values, such as the number, quantity, amount, and range of various components are mentioned in the embodiments shown below, the present disclosure is not limited to such numerical values ​​unless otherwise specified or clearly established in principle. Furthermore, with respect to structures, steps, and the like described in the embodiments shown below, these are not necessarily required in the present disclosure unless otherwise specified or clearly established in principle.

[0024] Implementation Method 1

[0025] 1-1. Overall Configuration of Remote Support System in Embodiment 1

[0026] First, the schematic configuration of the remote support system according to the first embodiment will be described. Figure 1 This is a block diagram showing a configuration example for explaining the outline of the remote support system according to the first embodiment. Figure 1 The illustrated remote support system 100 is a system for remotely controlling the driving of the autonomous vehicle 10 when it is predicted that the autonomous vehicle 10 will have difficulty continuing autonomous driving. Hereinafter, the autonomous vehicle 10 used in the remote support system 100 will be referred to simply as "vehicle 10."

[0027] Remote operation not only refers to remote driving, including instructions for any of the following: acceleration, deceleration, and steering of the vehicle, but also includes driving assistance that assists with a portion of the vehicle 10's understanding or judgment of its surroundings. Remote operation is performed by a remote operator on standby at a remote location. There is no limit on the number of remote operators used in the remote support system 100. Furthermore, there is no limit on the number of vehicles 10 used in the remote support system 100.

[0028] like Figure 1 As shown, remote support system 100 includes a vehicle 10 and a remote operation device 2. Remote operation device 2 includes a remote server 4 and a remote operator interface 6 for inputting remote operation commands. Remote server 4 is communicatively connected to vehicle 10 via a communication network N. Various information is transmitted from vehicle 10 to remote server 4. Remote operator interface 6 includes, for example, input devices simulating a vehicle's steering wheel, accelerator pedal, and brake pedal. Alternatively, remote operator interface 6 includes an input device for inputting judgment results during driving assistance.

[0029] In the remote support system 100, when it is predicted that the autonomous vehicle 10 will find it difficult to continue autonomous driving, a remote operation request is issued from the vehicle 10 requesting remote operation of the vehicle 10. In the remote support system 100, in response to the remote operation request issued from the vehicle 10, a remote operator performs remote operation via the remote operation device 2. Typically, the remote operator inputs the remote operation instructions into the remote operator interface 6. The remote server 4 issues remote operation instructions to the vehicle 10 via the communication network N. The vehicle 10 travels in accordance with the remote operation instructions sent from the remote operation device 2. It should be noted that the configuration of the remote operation device 2 can employ related technologies, and therefore a detailed description thereof will be omitted.

[0030] 1-2. Configuration of the Autonomous Driving Vehicle in Embodiment 1

[0031] Next, an example of a configuration of autonomous driving of the autonomous driving vehicle 10 applied to the remote support system 100 according to the first embodiment will be described. Figure 2 This is a block diagram showing an example of the configuration of an autonomous vehicle 10. Vehicle 10 is capable of autonomous driving. The autonomous driving described here assumes Level 3 or higher, as defined by the Society of Automotive Engineers (SAE). It should be noted that there are no restrictions on the power source of vehicle 10.

[0032] Vehicle 10 includes an automatic driving control device 40. Automatic driving control device 40 has functions for automatically driving vehicle 10 and for remotely controlling vehicle 10 in accordance with remote control instructions from a remote operator. Information acquisition device 30, communication device 50, and travel device 60 are connected to automatic driving control device 40.

[0033] The information acquisition device 30 is configured to include a vehicle position sensor 31 , a peripheral condition sensor 32 , and a vehicle state sensor 33 .

[0034] The vehicle position sensor 31 detects the position and orientation of the vehicle 10. For example, the vehicle position sensor 31 includes a GPS (Global Positioning System) sensor. The GPS sensor receives signals transmitted from multiple GPS satellites and calculates the position and orientation of the vehicle 10 based on the received signals. The vehicle position sensor 31 can also perform localization processing to improve the accuracy of the current position of the vehicle 10. The information detected by the vehicle position sensor 31 is constantly transmitted to the automatic driving control device 40 as part of the surrounding environment information.

[0035] The surrounding situation sensor 32 identifies information about the surroundings of the vehicle 10. Examples of the surrounding situation sensor 32 include a camera (photographic device), a laser radar (LIDAR), and a radar. The surrounding information includes information about objects identified by the surrounding situation sensor 32. Examples of objects include surrounding vehicles, pedestrians, roadside objects, obstacles, white lines, and traffic lights. Object information includes the relative position and relative speed of the object relative to the vehicle 10. The information identified by the surrounding situation sensor 32 is constantly transmitted to the automatic driving control device 40 as part of the surrounding environment information.

[0036] The vehicle state sensor 33 detects vehicle information indicating the state of the vehicle 10. Examples of the vehicle state sensor 33 include a vehicle speed sensor, a lateral acceleration sensor, and a yaw rate sensor. The information detected by the vehicle state sensor 33 is constantly transmitted to the automatic driving control device 40 as part of the vehicle motion information.

[0037] The communication device 50 communicates with the outside of the vehicle 10. For example, the communication device 50 sends and receives various information with the remote control device 2 via the communication network N. In addition, the communication device 50 communicates with external devices such as roadside devices, surrounding vehicles, and surrounding infrastructure. Roadside devices are, for example, beacon devices that transmit congestion information, traffic information for each lane, control information such as temporary stops, information on traffic conditions at blind spots, etc. In addition, when the external device is a surrounding vehicle, the communication device 50 performs vehicle-to-vehicle communication (V2V communication) with the surrounding vehicles. Moreover, when the external device is surrounding infrastructure, the communication device 50 performs vehicle-to-road communication (V2I communication) with the surrounding infrastructure.

[0038] The travel device 60 includes a steering device, a drive device, and a braking device. The steering device steers the wheels of the vehicle 10. The drive device is a driving source that generates driving force for the vehicle 10. Examples of the drive device include an engine or an electric motor. The braking device generates braking force for the vehicle 10. The travel device 60 controls the travel of the vehicle 10 based on travel control variables related to steering, acceleration, and deceleration of the vehicle 10.

[0039] The autonomous driving control device 40 is an information processing device that performs various processes for autonomous driving and remote autonomous driving. Typically, the autonomous driving control device 40 is a microcomputer equipped with at least one processor 42, at least one storage device 44, and at least one input / output interface 48. The autonomous driving control device 40 is also called an ECU (Electronic Control Unit).

[0040] The storage device 44 stores driving environment information 46 related to the driving environment of the vehicle 10. The driving environment information 46 includes the aforementioned surrounding environment information and vehicle motion information. Examples of the storage device 44 include volatile memory, nonvolatile memory, and an HDD (Hard Disk Drive).

[0041] The storage device 44 stores a map database 47. The map database 47 is a database that stores map information. The driving environment information 46 includes map information from the map database 47. This map information includes traffic environment information such as road location information, road shape information, the number of lanes, lane widths, intersection and fork location information, and road priority. It should be noted that the map database 47 may also be stored on a server that can communicate with the vehicle 10, such as the remote server 4 of the remote control device 2.

[0042] Processor 42 includes a CPU (Central Processing Unit). Processor 42 is coupled to a storage device 44 and an input / output interface 48. Storage device 44 stores at least one program 45, including a remote support program related to autonomous driving and remote autonomous driving. Processor 42 implements the various functions of autonomous driving control device 40 by reading and executing program 45 stored in storage device 44.

[0043] The input / output interface 48 is an interface for exchanging information with the remote operation device 2 . Various information generated in the automatic driving control device 40 and a remote operation request described later are output to the remote operation device 2 via the input / output interface 48 .

[0044] 1-3. Features of the remote support system according to Embodiment 1

[0045] First, an example of a situation in which remote operations by a remote operator are performed in the remote support system 100 according to the first embodiment will be described. Figure 3 and Figure 4 This is a diagram for explaining an example of a situation in which remote operation is performed in the remote support system 100 according to the first embodiment.

[0046] exist Figure 3 and Figure 4 Vehicle 10 is depicted traveling in lane L1. Lane L2, adjacent to lane L1, is a lane for vehicles traveling in the opposite direction of vehicle 10. Lanes L1 and L2 intersect with lane L3 at intersection I1. Vehicle 10 is scheduled to turn right at intersection I1 and proceed to lane L3. Oncoming vehicle V1 is depicted traveling in lane L2 on the opposite side of intersection I1 from vehicle 10. Like vehicle 10, oncoming vehicle V1 is scheduled to enter intersection I1.

[0047] In the remote support system 100, when it is predicted that the vehicle 10 will have difficulty in continuing the automatic driving, a remote operation request RQ is issued to the remote operation device 2. Typically, in the remote support system 100, when it is predicted that the vehicle 10 will collide with an object to be avoided, a remote operation request RQ is issued to the remote operation device 2. Figure 3 In the example of the traffic environment shown, the oncoming vehicle V1 is an object to be avoided. The automatic driving control device 40 generates a driving trajectory TR1 for the vehicle 10 and a predicted trajectory TR2 for the oncoming vehicle V1. Then, the automatic driving control device 40 determines whether the vehicle 10 is predicted to collide with the oncoming vehicle V1 based on the driving trajectory TR1 and the predicted trajectory TR2. This determination process is hereinafter referred to as the "collision determination process." In the collision determination process, for example, an intersection determination process is performed, which is a process for determining whether the driving trajectory TR1 and the predicted trajectory TR2 intersect. Then, when it is determined in the intersection determination process that the driving trajectory TR1 and the predicted trajectory TR2 are reversed, it is determined that the vehicle 10 is predicted to collide with the oncoming vehicle V1.

[0048] If the automatic driving control device 40 determines that the vehicle 10 is predicted to collide with the oncoming vehicle V1, the automatic driving control device 40 issues a remote operation request RQ to the remote operation device 2 and transmits the driving environment information 46 acquired by the information acquisition device 30 as information required for remote control of the vehicle 10. The remote operation device 2, having received the remote operation request RQ, performs remote automatic driving by the remote operator. Typically, the remote operator determines whether to pass through the intersection I1 based on the driving environment information 46 received from the automatic driving control device 40. The remote operator then issues a remote operation instruction to the vehicle 10 by operating the remote operator interface 6. This remote operation instruction can be an instruction on the amount of operation of the driving device 60 or an instruction on a determination result such as "go," "stop," or "turn right." The vehicle 10 passes through the intersection I1 in accordance with the remote operation instruction transmitted from the remote operation device 2.

[0049] Here, in the calculation of the collision determination process, various parameters that define the evaluation criteria are used. These parameters are stored in the storage device 44. The parameters include, for example, parameter A, which serves as a second evaluation criterion that emphasizes driving efficiency, and parameter B, which serves as a first evaluation criterion that emphasizes safety. Compared to parameter B, parameter A has a higher benchmark for treating the detected object as an avoidance target. Alternatively, compared to parameter B, parameter A treats the avoidance target as an object with a high probability of complying with traffic rules or customs. Alternatively, compared to parameter B, parameter A treats the avoidance target as an object with a high probability of maintaining the current action state. Alternatively, compared to parameter B, parameter A has a narrower allowable error range when determining whether there is a collision with the avoidance target.

[0050] exist Figure 3 The example shown illustrates a case where the automatic driving control device 40 executes collision determination processing using parameter A that prioritizes driving efficiency. While using parameter A, which prioritizes driving efficiency, prioritizes driving efficiency compared to using parameter B, which prioritizes safety, there is also a tendency for the frequency of successful collision determinations to decrease or for the timing of successful collision determinations to be delayed.

[0051] If the collision determination is delayed, the issuance of the remote operation request RQ is also delayed. It takes approximately 10 to 15 seconds for the remote operator to begin remote operation from the issuance of the remote operation request RQ. Therefore, if the remote operation request RQ is delayed, the vehicle 10 must wait at a designated stop position before the predicted collision location for the remote operator to begin remote operation. This waiting time on the road can disrupt traffic flow or cause rear-end collisions.

[0052] On the other hand, Figure 4The example shown shows a case where the automatic driving control device 40 executes collision determination processing using the safety-focused parameter B. In collision determination processing using the safety-focused parameter B, the timing of the collision determination tends to be earlier, so the timing of issuing the remote operation request RQ is not delayed. However, if the timing of issuing the remote operation request RQ becomes earlier, a driving trajectory may be generated that causes the vehicle to decelerate or stop at the earlier timing, potentially impairing driving efficiency and comfort.

[0053] Thus, in order to ensure the remote operator's judgment time, it is preferable to perform the collision determination process for issuing the remote operation request at an early timing. However, on the other hand, in order to improve driving efficiency, it is preferable to perform the collision determination process for correcting the driving trajectory at a late timing.

[0054] Therefore, the remote support system 100 of Embodiment 1 is characterized by using different parameters for collision determination processing depending on the purpose of the vehicle 10 during autonomous driving. Typically, the autonomous driving control device 40 uses parameter B, which prioritizes safety, in the first collision determination process used to determine whether a remote operation request is necessary. On the other hand, the autonomous driving control device 40 uses parameter A, which prioritizes driving efficiency, in the second collision determination process used to determine whether a driving trajectory correction is necessary. This differentiated use of parameters in the collision determination process can prevent a decrease in autonomous driving efficiency while enabling determination of whether a remote operation request is necessary, which prioritizes safety.

[0055] Hereinafter, the functional configuration and specific processing of the automatic driving control device 40 of the remote support system 100 according to the first embodiment will be described.

[0056] 1-4. Functional Structure of the Autonomous Driving Control System

[0057] Next, an example of the functional configuration of the automatic driving control device 40 will be described. Figure 5 4 is a functional block diagram showing a portion of the functions of the automatic driving control device 40. The automatic driving control device 40 includes a surrounding environment information acquisition unit 402, a vehicle motion information acquisition unit 404, a map information acquisition unit 406, a travel planning unit 410, and a remote operation request unit 460.

[0058] The surrounding environment information acquisition unit 402 and the vehicle motion information acquisition unit 404 are functional blocks for respectively acquiring surrounding environment information and vehicle motion information detected by the information acquisition device 30 . The map information acquisition unit 406 is a functional block for acquiring map information stored in the map database 47 .

[0059] The travel planning unit 410 performs a first determination process of determining whether a remote operation request is to be issued, and a second determination process of determining whether a travel trajectory for automatic driving of the vehicle 10 is to be corrected.

[0060] Typically, the driving plan unit 410 includes a driving trajectory generating unit 412 , a target object behavior predicting unit 414 , a driving plan generating unit 420 , a driving trajectory selecting unit 450 , and a remote operation request determining unit 452 .

[0061] The travel trajectory generator 412 generates a travel trajectory TR1 of the vehicle 10 using the target route, position, and speed of the vehicle 10 calculated based on the destination, vehicle motion information, and map information. The generated travel trajectory TR1 is sent to the travel plan generator 420 .

[0062] The object movement prediction unit 414 performs a predicted trajectory generation process, which is a process for generating a predicted trajectory TR2 for an avoidance object that may collide with the autonomous driving vehicle 10. Examples of avoidance objects include oncoming vehicles, preceding vehicles, and crossing vehicles traveling on intersecting roads. The object movement prediction unit 414 uses the position and speed of the avoidance object calculated based on the surrounding environment information and map information to calculate the predicted trajectory TR2 for the avoidance object. It should be noted that when there are multiple predictions for the avoidance object, such as right turn, left turn, and straight driving, multiple predicted trajectories TR2 can also be calculated. The calculated predicted trajectory TR2 is sent to the driving plan generation unit 420.

[0063] The driving plan generation unit 420 includes a collision determination unit 430 and a driving trajectory correction unit 440. The collision determination unit 430 calculates the determination result of whether the vehicle 10 will collide with the avoidance object in the future and the predicted collision position CP in the event of a collision. Typically, the collision determination unit 430 calculates the area where the driving trajectory TR1 and the predicted trajectory TR2 intersect. Then, the collision determination unit 430 uses the evaluation criterion of parameter A to calculate the determination result of whether the vehicle 10 will collide with the avoidance object in the future and the predicted collision position CPA. This process is a collision determination process using parameter A as a second evaluation criterion and is therefore also referred to as the "second collision determination process." The determination result of the second collision determination process and the calculated predicted collision position CPA are sent to the driving trajectory correction unit 440.

[0064] Furthermore, collision determination unit 430 uses the evaluation criteria for parameter B to calculate a result of determining whether vehicle 10 will collide with the avoidance object in the future and a predicted collision position CPB. This process is also referred to as "first collision determination processing" because it uses parameter B as the first evaluation criteria. The result of the first collision determination processing and the predicted collision position CPB are transmitted to remote operation request determination unit 452.

[0065] The first collision determination process using the evaluation criteria of parameter B prioritizes safety over the evaluation criteria of parameter A. Furthermore, the second collision determination process using the evaluation criteria of parameter A prioritizes driving efficiency over the evaluation criteria of parameter B. Several examples of specific evaluation criteria for parameters A and B are described below.

[0066] <Evaluation Criteria for Avoidance Treatment>

[0067] The driving planning unit 410 treats detected objects included in the input surrounding environment information as avoidance targets if they are predicted to collide with the vehicle 10. Parameters A and B are assigned evaluation criteria for treating detected objects as avoidance targets. Typically, parameter A treats detected objects detected by both the lidar and camera sensors in the plurality of surrounding situation sensors 32 as avoidance targets. In contrast, parameter B treats detected objects detected only by the camera sensors in the plurality of surrounding situation sensors 32 as avoidance targets. Thus, the evaluation criteria for parameter B result in more avoidance targets being treated than for parameter A.

[0068] Alternatively, the surrounding environment information includes a likelihood index value indicating the accuracy (likelihood) of the detected object. Driving planning unit 410 processes detected objects included in the input surrounding environment information by designating those objects whose likelihood index values ​​exceed a predetermined threshold as avoidance targets. Parameters A and B have thresholds for likelihood index values. Parameter A has a higher threshold for likelihood index values ​​than parameter B. Therefore, based on the evaluation criteria for parameter B, more avoidance targets are processed compared to parameter A.

[0069] <Evaluation criteria related to the likelihood of compliance with traffic laws and regulations or traffic customs>

[0070] The driving planning unit 410 predicts the likelihood of the detected object complying with traffic laws or traffic customs, thereby generating a predicted trajectory TR2 for the avoidance target. Parameters A and B are defined as evaluation criteria related to the likelihood of the avoidance target complying with traffic laws or traffic customs. Parameter A, compared to parameter B, treats the avoidance target as one with a higher likelihood of complying with traffic laws or traffic customs. Typically, if the detected object is a non-priority vehicle stopped at a temporary stop line, parameter A predicts that the non-priority vehicle will not start, thereby determining the avoidance target or generating the predicted trajectory TR2. In contrast, parameter B considers the possibility that the non-priority vehicle will start disregarding traffic laws or traffic customs, thereby determining the avoidance target or generating the predicted trajectory TR2.

[0071] Alternatively, if the detected object is an oncoming vehicle in the opposite lane ahead, waiting to turn right, parameter A predicts that the oncoming vehicle will comply with traffic regulations or customs and continue to wait to turn right, thereby determining avoidance action or generating predicted trajectory TR2. Conversely, parameter B also considers the possibility that the oncoming vehicle will disregard traffic regulations or customs and turn right, thereby determining avoidance action or generating predicted trajectory TR2.

[0072] <Evaluation Criteria Regarding the Ability to Maintain the Avoidance Target's Movement>

[0073] The driving planning unit 410 predicts the likelihood of the avoidance object maintaining its future behavior and generates a predicted trajectory TR2. Parameters A and B are assigned evaluation criteria related to the likelihood of the avoidance object maintaining its future behavior. Parameter A predicts a higher likelihood of the avoidance object maintaining its current behavior than parameter B, thereby determining the avoidance object's handling or generating the predicted trajectory TR2. Typically, when the detected target object is a vehicle, parameter A assumes the vehicle will maintain its current behavior and determines the avoidance object's handling or generates the predicted trajectory TR2. In contrast, parameter B predicts the vehicle will not only maintain its current behavior but also perform a different behavior, thereby determining the avoidance object's handling or generating the predicted trajectory TR2.

[0074] Alternatively, if the detected object is a pedestrian walking near a crosswalk, parameter A assumes the pedestrian continues walking and determines whether the pedestrian has reached the crosswalk. Parameter B, on the other hand, assumes the pedestrian begins running instead of walking and determines whether the pedestrian has reached the crosswalk.

[0075] <Evaluation criteria related to tolerance>

[0076] When the vehicle 10's driving trajectory TR1 intersects the predicted trajectory TR2 of the avoidance object, the driving planning unit 410 predicts that the vehicle 10 may collide with the avoidance object. Parameters A and B are defined as evaluation criteria related to the tolerance for track intersection determination. Typically, parameter A considers an error ellipse within a standard deviation of 2σ to determine track intersection. Parameter B considers an error ellipse within a standard deviation of 3σ to determine track intersection.

[0077] Return again Figure 5The trajectory correction unit 440 corrects the trajectory TR1 based on the results of the second collision determination process using the evaluation criteria of parameter A. Typically, if the second collision determination process indicates a collision, the trajectory correction unit 440 generates a trajectory TRA1 that corrects the trajectory TR1 based on the vehicle 10's position and speed, map information, the trajectory TR1, the predicted trajectory TR2, and the predicted collision position CPA, so that the vehicle 10 avoids collision with the avoidance target. On the other hand, if the second collision determination process indicates a collision-free state, the trajectory correction unit 440 generates the trajectory TR1 as is. Furthermore, in parallel with the generation of the trajectory TRA1, the trajectory correction unit 440 generates a trajectory TRA2 that corrects the trajectory TR1 based on the vehicle 10's position and speed, map information, the trajectory TR1, the predicted trajectory TR2, and the predicted collision position CPA, so that the vehicle 10 stops at a predetermined stop position immediately before the predicted collision position CPA or before entering the intersection and awaits remote control. The generated trajectories TRA1 and TRA2 are transmitted to the trajectory selection unit 450.

[0078] Remote operation request determination unit 452 determines whether to issue a remote operation request based on the results of the first collision determination process using the evaluation criteria of parameter B. Typically, if the result of the first collision determination process indicates no collision, that is, if there is no predicted collision position CPB input to remote operation request determination unit 452, remote operation request determination unit 452 determines that a remote operation request is not necessary. Furthermore, if the result of the first collision determination process indicates a collision, remote operation request determination unit 452 uses map information or surrounding environment information to determine the traffic priority status at predicted collision position CPB. If the traffic priority status at predicted collision position CPB indicates that vehicle 10 should be prioritized, remote operation request determination unit 452 determines that a remote operation request is not necessary. If the traffic priority status at predicted collision position CPB indicates that vehicle 10 is not prioritized, remote operation request determination unit 452 determines that a remote operation request is necessary. The result of the determination on whether a remote operation request is necessary is transmitted to remote operation request unit 460. Furthermore, the result of the determination on whether a remote operation request is necessary is also transmitted to driving trajectory selection unit 450.

[0079] The travel track selection unit 450 selects a travel track TR to be transmitted to the travel device 60 based on the result of determining whether a remote operation request is required. Typically, if a remote operation request is determined to be required, the travel track selection unit 450 selects the travel track TRA2 as the travel track TR. Alternatively, if a remote operation request is determined not to be required, the travel track selection unit 450 selects the travel track TRA1 as the travel track TR. The selected travel track TR is transmitted to the travel device 60.

[0080] The remote operation request unit 460 is a device for issuing a remote operation request to a remote operator operating the remote operation device 2 via the communication network N. The remote operation request unit 460 issues a remote operation request RQ to the remote operation device 2 in accordance with the remote operation necessity request sent from the remote operation request determination unit 452 .

[0081] 1-5. Specific Processing Executed by the Automatic Driving Control Device

[0082] Figure 6 4 is a flowchart of the processing executed in the automatic driving control device 40. During the automatic driving of the vehicle 10, the automatic driving control device 40 repeatedly executes Figure 6 The routine shown.

[0083] exist Figure 6 In step S100 of the illustrated routine, the travel trajectory generator 412 generates a travel trajectory TR1 for the vehicle 10. In the following step S102, the target object behavior predictor 414 generates a predicted trajectory TR2 for the avoidance target. While step S102 is being performed, the first determination process from steps S110 to S116 and the second determination process from steps S120 to S130 are performed in parallel.

[0084] In the first determination process, first, in step S110, a first collision determination process is performed using the safety-focused parameter B evaluation criteria. Typically, the first collision determination process determines whether a collision has occurred between the avoidance object and the vehicle 10 and calculates a predicted collision position CPB. In the next step, S112, a determination is made as to whether a collision has occurred between the avoidance object and the vehicle 10. Here, if a valid predicted collision position CPB is calculated, the collision determination unit 430 determines that a collision has occurred between the avoidance object and the vehicle 10.

[0085] The remote operation request can be determined to be unnecessary if it is determined in step S112 that there is no collision between the avoidance object and the vehicle 10. In this case, the remote operation request determination unit 452 determines that a remote operation request is unnecessary and ends this routine without issuing the remote operation request RQ.

[0086] On the other hand, if it is determined in step S112 that a collision with the avoidance object is imminent, the process proceeds to step S114. In step S114, remote operation request determination unit 452 determines, based on the map information and predicted collision position CPB, whether vehicle 10 is non-priority relative to the traffic environment of the avoidance object at predicted collision position CPB. If vehicle 10 is not non-priority at predicted collision position CPB, remote operation request determination unit 452 determines that a collision with the avoidance object is unlikely, and therefore determines that a remote operation request is not necessary.

[0087] On the other hand, if vehicle 10 is not prioritized at predicted collision position CPB, it is determined that vehicle 10 is predicted to collide with the avoidance object. In this case, the process proceeds to step S116. In step S116, remote operation request determination unit 452 determines that a remote operation request is necessary and issues a remote operation request RQ.

[0088] In the second determination process, first, in step S120, a second collision determination process is performed using the evaluation criteria of parameter A that prioritizes driving efficiency. Typically, the second collision determination process determines whether a collision has occurred between the avoidance object and the vehicle 10 and calculates a predicted collision position CPA. In the next step, S122, a determination is made as to whether a collision has occurred between the avoidance object and the vehicle 10. Here, if a valid predicted collision position CPA is calculated, the collision determination unit 430 determines that a collision has occurred between the avoidance object and the vehicle 10.

[0089] If it is determined in step S122 that the avoidance object has not collided with the vehicle 10, it is determined that the travel trajectory TR1 generated in step S100 does not need to be corrected. In this case, the travel trajectory correction unit 440 generates the travel trajectory TR1 as the travel trajectory TRA1, and the process proceeds to step S126. On the other hand, if it is determined that the avoidance object has collided with the vehicle 10, the process proceeds to step S124.

[0090] In step S124, the travel trajectory correction unit 440 generates a travel trajectory TRA1 obtained by correcting the travel trajectory TR1 in consideration of the collision with the avoidance object at the predicted collision position CPA. After the process of step S124 is completed, the process proceeds to step S126.

[0091] In step S126, it is determined whether a remote operation request RQ has been issued in the process of step S116. If the remote operation request RQ has not been issued, the travel track TRA1 is selected as the travel track TR, and the process proceeds to step S130. On the other hand, if the remote operation request RQ has been issued, the process proceeds to step S128.

[0092] The driving trajectory correction unit 440 generates a driving trajectory TRA2 that corrects the driving trajectory TR1 so that the vehicle 10 stops at a predetermined stop position immediately before the predicted collision position CPA or before entering the intersection and waits for remote control. In step S128, the driving trajectory TRA2 is selected as the driving trajectory TR, and the process proceeds to step S130.

[0093] In step S130, the selected driving trajectory TR is transmitted to the driving device 60. The driving device 60 performs automatic driving of the vehicle 10 according to the driving trajectory TR.

[0094] Thus, according to the remote support method based on the remote support system 100 of the first embodiment, the parameter B, which prioritizes safety, is used in the first determination process, and the parameter A, which prioritizes driving efficiency, is used in the second determination process. With this configuration, it is possible to determine whether to issue a remote operation request while prioritizing safety, and it is possible to drive the vehicle 10 while prioritizing driving efficiency.

[0095] 1-6. Modifications

[0096] The remote support system 100 of the first embodiment may also adopt the following modified aspects.

[0097] There is no limitation on the functional configuration of the automatic driving control device 40. That is, a part or all of the functions of the automatic driving control device 40 may be mounted on the vehicle 10 or may be configured on the remote server 4 of the remote operation device 2.

[0098] Figure 6 In the illustrated routine, the first determination process of steps S110 to S116 and the second determination process of steps S120 to S130 are not limited to being executed in parallel, but may be executed sequentially. In this case, for example, the first determination process of steps S110 to S116 may be executed before the second determination process of steps S120 to S130.

[0099] In the automatic driving control device 40 of embodiment 1, a configuration is provided in which a common collision determination unit 430 performs collision determination processing using different parameters A and B. However, a configuration may also be provided in which a plurality of collision determination units perform collision determination processing using unique parameters. Figure 7 This is a diagram for explaining a modified example of the automatic driving control device 40 according to the first embodiment. Figure 7 The automatic driving control device 40 shown in FIG. 1 includes a collision determination unit A432 and a collision determination unit B434 instead of Figure 5 In addition to the collision determination unit 430, Figure 5 Same composition.

[0100] Collision determination unit A432 is a functional block for performing collision determination processing using inherent parameters equivalent to parameter A. In other words, collision determination unit A432 has the same function as the collision determination processing using parameter A in collision determination unit 430. Collision determination unit B434 is a functional block for performing collision determination processing using inherent parameters equivalent to parameter B. In other words, collision determination unit B434 has the same function as the collision determination processing using parameter B in collision determination unit 430. With this configuration, collision determination unit A432 and collision determination unit B434 can implement processing similar to that of collision determination unit 430 in Embodiment 1.

[0101] Figure 8 This is a diagram for explaining another modified example of the automatic driving control device 40 according to the first embodiment. Figure 8 The automatic driving control device 40 shown in FIG. 4 is provided with a collision judgment model A 436 and a collision judgment model B 438 instead of Figure 5 The driving track generation unit 412, the object movement prediction unit 414 and the collision determination unit 430 have the same Figure 5 Same composition.

[0102] Collision determination model A436 is a machine learning model used to perform a collision determination process equivalent to the collision determination process using parameter A. That is, collision determination model A436 has the same function as the collision determination process using parameter A in collision determination unit 430. For example, when assuming the generation of a driving trajectory, collision determination model A436 returns true as the predicted correct answer when deceleration is required, and performs model learning. Furthermore, collision determination model B438 is a functional block used to perform a collision determination process using parameter B. That is, collision determination model B438 has the same function as the collision determination process using parameter B in collision determination unit 430. For example, when assuming a remote operation request, collision determination model B438 returns true as the predicted correct answer when a remote operation request is required, and performs model learning. With this configuration, collision determination model A436 and collision determination model B438 can implement the same process as collision determination unit 430 in embodiment 1.

Claims

1. A remote support system configured to, when a vehicle configured to perform autonomous driving is predicted to have difficulty continuing autonomous driving, issue a remote operation request to a remote operator requesting remote operation of the vehicle, the remote support system comprising: a storage device for storing commands; as well as at least one processor connected to the storage device, The at least one processor is configured to obtain driving environment information of the vehicle, The at least one processor is configured to execute a collision determination process, wherein the collision determination process is a process of determining whether the vehicle will collide with a detection target based on a future driving trajectory of the vehicle generated based on the driving environment information. The at least one processor is configured to: The collision determination process is performed using a second evaluation criterion to obtain a second collision determination process result. If the second collision determination processing result indicates a collision, the driving trajectory is corrected to serve as the first driving trajectory; if the second collision determination processing result indicates no collision, the driving trajectory is used as the first driving trajectory. The at least one processor is configured to: In parallel with the generation of the first travel trajectory, a second travel trajectory is generated for causing the vehicle to stop at a predetermined stop position and wait for the remote operation. The at least one processor is configured to: The collision determination process is performed using a first evaluation criterion to obtain a first collision determination process result. If the first collision determination processing result is a collision, determining whether to send a remote operation request; if it is determined that the remote operation request needs to be sent, issuing the remote operation request; if the first collision determination processing result is no collision, determining not to send the remote operation request; The at least one processor is configured to: If it is determined that the remote operation request is not sent, the first driving trajectory is selected as the autonomous driving driving trajectory; if it is determined that the remote operation request is sent, the second driving trajectory is selected as the autonomous driving driving trajectory. The vehicle performs automatic driving according to the automatic driving trajectory, The first evaluation criterion is an evaluation criterion that places greater emphasis on the safety of the vehicle than the second evaluation criterion. The second evaluation criterion places greater emphasis on the driving efficiency of the vehicle than the first evaluation criterion.

2. The remote support system according to claim 1, wherein: The collision determination process includes a predicted trajectory generation process of generating a predicted future trajectory of an avoidance object based on the driving environment information. The at least one processor is configured to determine whether the vehicle will collide with a detection object based on the predicted trajectory and the driving trajectory.

3. The remote support system according to claim 2, wherein: The evaluation criteria include an evaluation criteria related to the accuracy of the avoidance object, The collision determination process includes acquiring a likelihood index value indicating the accuracy of the detected object detected based on the driving environment information, and setting the detected object as the avoidance target if the acquired likelihood index value is higher than a predetermined determination threshold value. In the collision determination process, the determination threshold of the first evaluation criterion is lower than the determination threshold of the second evaluation criterion.

4. The remote support system according to claim 2, wherein: The evaluation criteria include an evaluation criteria associated with the likelihood of the avoidance object complying with traffic regulations, In the predicted trajectory generation process, the predicted trajectory of the avoidance object is generated using the compliance probability. The first compliance probability is lower than the second compliance probability, The first compliance probability is the compliance probability of the first evaluation criterion, The second compliance probability is the compliance probability of the second evaluation criterion.

5. The remote support system according to claim 2, wherein: The evaluation criteria include an evaluation criterion related to the possibility of the avoidance object maintaining its current motion state. In the predicted trajectory generation process, the predicted trajectory of the avoidance object is generated using the maintenance possibility. The first maintenance probability is lower than the second maintenance probability, The first maintenance possibility is the maintenance possibility of the first evaluation criterion, The second maintenance possibility is the maintenance possibility of the second evaluation criterion.

6. The remote support system according to claim 2, wherein: The collision determination process includes an intersection determination process for determining whether the predicted track and the travel track intersect. The at least one processor is configured to determine whether the vehicle will collide with the detection object based on a determination result of the intersection determination process, The evaluation criteria include an evaluation criteria related to an error range allowed in the intersection determination process, In the intersection determination process, the error range of the first evaluation criterion is narrower than the error range of the second evaluation criterion.

7. A remote support method, characterized in that: include: When it is predicted that it is difficult for a vehicle configured to perform autonomous driving to continue autonomous driving, issuing a remote operation request to a remote operator to request remote operation of the vehicle; executing a collision determination process for determining whether the vehicle will collide with a detection target based on a future travel trajectory of the vehicle generated based on driving environment information; performing the collision determination process using a second evaluation criterion to obtain a second collision determination process result, and if the second collision determination process result indicates a collision, correcting the driving trajectory to use as the first driving trajectory; and if the second collision determination process result indicates no collision, using the driving trajectory as the first driving trajectory; In parallel with the generation of the first travel trajectory, a second travel trajectory is generated for causing the vehicle to stop at a predetermined stop position and wait for the remote operation; performing the collision determination process using a first evaluation criterion to obtain a first collision determination process result, determining whether to send a remote operation request if the first collision determination process result indicates a collision, issuing the remote operation request if it is determined that the remote operation request needs to be sent, and determining not to send the remote operation request if the first collision determination process result indicates no collision; If it is determined that the remote operation request is not sent, the first driving trajectory is selected as the autonomous driving driving trajectory; if it is determined that the remote operation request is sent, the second driving trajectory is selected as the autonomous driving driving trajectory; as well as The vehicle performs automatic driving according to the automatic driving trajectory, The first evaluation criterion is an evaluation criterion that places greater emphasis on the safety of the vehicle than the second evaluation criterion. The second evaluation criterion places greater emphasis on the driving efficiency of the vehicle than the first evaluation criterion.

8. A non-transitory storage medium storing instructions executable by one or more processors and causing the one or more processors to perform the following functions, wherein the non-transitory storage medium is characterized in that the functions include: When it is predicted that it is difficult for a vehicle configured to perform autonomous driving to continue autonomous driving, issuing a remote operation request to a remote operator to request remote operation of the vehicle; Acquiring driving environment information of the vehicle; executing a collision determination process for determining whether the vehicle will collide with a detection target based on a future travel trajectory of the vehicle generated based on the driving environment information; performing the collision determination process using a second evaluation criterion to obtain a second collision determination process result, and if the second collision determination process result indicates a collision, correcting the driving trajectory to use as the first driving trajectory; and if the second collision determination process result indicates no collision, using the driving trajectory as the first driving trajectory; In parallel with the generation of the first travel trajectory, a second travel trajectory is generated for causing the vehicle to stop at a predetermined stop position and wait for the remote operation; performing the collision determination process using a first evaluation criterion to obtain a first collision determination process result, determining whether to send a remote operation request if the first collision determination process result indicates a collision, issuing the remote operation request if it is determined that the remote operation request needs to be sent, and determining not to send the remote operation request if the first collision determination process result indicates no collision; If it is determined that the remote operation request is not sent, the first driving trajectory is selected as the autonomous driving driving trajectory; if it is determined that the remote operation request is sent, the second driving trajectory is selected as the autonomous driving driving trajectory; as well as The vehicle performs automatic driving according to the automatic driving trajectory, The first evaluation criterion is an evaluation criterion that places greater emphasis on the safety of the vehicle than the second evaluation criterion. The second evaluation criterion places greater emphasis on the driving efficiency of the vehicle than the first evaluation criterion.

Images