Autonomous driving method, device, computer-readable recording medium, and system

Abstract

An automatic driving device (110) includes a registration unit (118) that registers, for an automatic driving vehicle, location information for each location at which remote assistance has been implemented in the past, the location information including a position and an influence range defined by at least a distance from the position, and an implementation unit (114) that, in a case where a request for remote assistance is made from the automatic driving vehicle within the influence range included in the registered location information, implements the remote assistance and causes automatic driving of the automatic driving vehicle to be implemented.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an automatic driving method, an automatic driving device, an automatic driving program, and an automatic driving system. BACKGROUND

[0002] In the past, a remote monitoring technology that ensures safety during autonomous travel of an autonomous travel vehicle has been proposed (see Patent Literature 1). In the technology described in Patent Literature 1, the autonomous travel vehicle transmits a camera image of the periphery of the vehicle captured by a camera to a remote monitoring center. In addition, in a case where an obstacle is detected on the basis of information obtained from an autonomous sensor including the camera, the autonomous travel vehicle automatically stops. The remote monitoring center determines whether or not it is possible to restart travel of the autonomous travel vehicle on the basis of the received camera image in a case where the autonomous travel vehicle automatically stops. Furthermore, the remote monitoring center transmits a start signal to the autonomous travel vehicle in a case where it is determined that it is possible to restart travel of the autonomous travel vehicle. In a case where the start signal is received from the remote monitoring center, the autonomous travel vehicle restarts travel.

[0003] In addition, a regulation device that reduces the work burden of an operator in a state where safety is maintained has been proposed (see Patent Literature 2). The regulation device described in Patent Literature 2 registers position information and sensing data as location data and further records in association with remote control information in a case where remote control is implemented. The regulation device compares current location data acquired from an autonomous driving vehicle that is an object vehicle and the location data that has been registered in the regulation device, and determines whether or not there is similar location data in a case where remote control is implemented on the autonomous driving vehicle. In a case where location data similar to the current location data of the object vehicle is registered, the regulation device performs first remote control in which the operator does not intervene. In a case where there is no similar location data, the regulation device performs second remote control in which the operator intervenes.

[0004] Patent Literature 1: Japanese Patent Application Publication No. 2019-87015

[0005] Patent Literature 2: Japanese Patent Application Publication No. 2019-185280

[0006] As a result of detailed research by the inventor, it was found that, in the technology described in Patent Literature 1, since a monitor calls in each time an obstacle is detected, the monitor is called multiple times in the same important factor each time an obstacle is detected. For example, assuming a case where road construction is performed on a path on which the autonomous travel vehicle travels, there is a possibility that a preceding vehicle that stops due to the construction, a guide who urges a lane change, or the like is respectively detected as an obstacle, and the monitor is called each time these vehicles or the guide are detected.

[0007] In addition, the inventors have found that, in the technology described in Patent Literature 2, since it is determined whether to implement remote control using the position information included in the location data, there is a problem in that it is not possible to properly determine whether an operator is involved, and in the case where an operator is involved, it is not possible to properly determine the timing of calling the operator. As a result, there is a case where an unnecessary operator call is generated. SUMMARY

[0008] The present disclosure aims to suppress the generation of an unnecessary operator call and the implementation of inappropriate remote instruction in a scenario where remote assistance of an automated vehicle is required.

[0009] An automated driving method according to an aspect of the present disclosure is a method in which a computer executes the following processing: for an automated vehicle, registers location information for each location where remote assistance has been implemented in the past, the location information including a position and an influence range defined by at least a distance from the position; and in the case where a request for remote assistance is generated in an automated vehicle within the influence range included in the registered location information, preserves the implementation of the remote assistance and implements the automated driving of the automated vehicle.

[0010] In addition, an automated driving device according to an aspect of the present disclosure is configured to include: a registration unit that, for an automated vehicle, registers location information for each location where remote assistance has been implemented in the past, the location information including a position and an influence range defined by at least a distance from the position; and an implementation unit that, in the case where a request for remote assistance is generated in an automated vehicle within the influence range included in the registered location information, preserves the implementation of the remote assistance and implements the automated driving of the automated vehicle.

[0011] In addition, an automated driving program according to an aspect of the present disclosure is a program for causing a computer to execute the following processing, including the following processing: for an automated vehicle, registers location information for each location where remote assistance has been implemented in the past, the location information including a position and an influence range defined by at least a distance from the position; and in the case where a request for remote assistance is generated in an automated vehicle within the influence range included in the registered location information, preserves the implementation of the remote assistance and implements the automated driving of the automated vehicle.

[0012] In addition, an automated driving system according to an aspect of the present disclosure is an automated driving system including a vehicle-mounted device mounted on an automated driving vehicle and controlling automated driving of the automated driving vehicle, and a remote assistance device that performs remote assistance of the automated driving of the automated driving vehicle, the remote assistance device including a registration unit that registers, for the automated driving vehicle, location information including a location and an influence range defined by at least a distance from the location, for each location at which remote assistance has been performed in the past, and the remote assistance device or the vehicle-mounted device including an implementation unit that, in a case where an automated driving vehicle within the influence range included in the registered location information generates a request for remote assistance, causes the remote assistance to be performed and the automated driving of the automated driving vehicle to be implemented.

[0013] According to the automated driving method, device, program, and system of the present disclosure, in a scenario in which remote control of an automated driving vehicle is required, generation of an unnecessary operator call and implementation of an inappropriate remote instruction can be suppressed. BRIEF DESCRIPTION OF DRAWINGS

[0014] The above and other objects, features, and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0015] Figure 1 is a diagram for explaining a problem point in a case where remote assistance is performed based on registered location information,

[0016] Figure 2 is a diagram for explaining a problem point in a case where remote assistance is performed based on registered location information,

[0017] Figure 3 is a diagram showing an outline structure of an automated driving system,

[0018] Figure 4 is a block diagram showing a hardware configuration of a remote assistance device,

[0019] Figure 5 is a block diagram showing a hardware configuration of a vehicle-mounted device,

[0020] Figure 6 is a functional block diagram of the remote assistance device in the first and second embodiments,

[0021] Figure 7 is a diagram showing one example of a vehicle information DB,

[0022] Figure 8 is a diagram showing one example of a location information DB,

[0023] Figure 9 is a diagram for explaining a comparison of sensor characteristics and registration characteristics,

[0024] Figure 10 is a diagram for explaining comparison of a sensor feature and a registered feature,

[0025] Figure 11 is a diagram showing one example of an influence range DB,

[0026] Figure 12 is a diagram for explaining a determination method of an influence range,

[0027] Figure 13 is a diagram for explaining another example of a determination method of an influence range,

[0028] Figure 14 is a functional block diagram of the on-vehicle device in the first and second embodiments,

[0029] Figure 15 is a flowchart showing one example of the AD vehicle side automatic driving process in the first embodiment,

[0030] Figure 16 is a flowchart showing one example of the remote assistance device side automatic driving process in the first embodiment,

[0031] Figure 17 is a functional block diagram of the remote assistance device in the modified example of the first embodiment,

[0032] Figure 18 is a functional block diagram of the on-vehicle device in the modified example of the first embodiment,

[0033] Figure 19 is a flowchart showing one example of the AD vehicle side automatic driving process in the modified example of the first embodiment,

[0034] Figure 20 is a flowchart showing one example of the remote assistance device side automatic driving process in the modified example of the first embodiment,

[0035] Figure 21 is a diagram for explaining a state in which new remote assistance is required,

[0036] Figure 22 is a flowchart showing one example of the remote assistance device side automatic driving process in the second embodiment. DETAILED DESCRIPTION

[0037] First, before detailed description of each embodiment, a problem point in a case where a place where remote assistance is implemented is registered as place information as described in the technology described in Patent Document 2, and based on the registered place information, remote assistance is implemented, is described more specifically.

[0038] In the technology described in Patent Literature 2, it is determined whether the location information of the subject vehicle is similar to the registered location information, to determine the presence or absence of intervention by the operator. For example, it is determined whether the positions included in the location information are close to each other. In a case where the determination criterion for the similarity of the location information is strict, the likelihood that there is no registered location information similar to the location information of the subject vehicle increases, and it is determined that there is no registered obstacle or the like, and the probability of calling the operator increases.

[0039] A more specific example will be described. For example, as shown in Figure 1 , assume that location information that takes a road parked vehicle as an important factor is registered. Assume that the location information includes position information of a vehicle parked on the road, and remote control information for avoiding the vehicle to travel (no operator call). In this case, assume that in a case where traffic congestion, temporary stop, slow travel, or the like (hereinafter, referred to as "traffic congestion or the like") occurs due to the road parked vehicle, remote assistance is requested from the automated driving vehicle at a time when the traffic congestion or the like is detected due to stop or deceleration or the like of a preceding vehicle. However, in Figure 1 , the position at which remote assistance is requested is spaced apart from the position included in the registered location information, and thus it is determined that it is a request for remote assistance on a location where location information is not registered, and the operator is called.

[0040] In addition, similarly, in the example of Figure 1 , in a case where the determination criterion for the similarity of the location information is relaxed, it is determined that the location information of the subject vehicle is similar to the registered location information, and the automated driving of the subject vehicle is controlled based on the remote control information included in the location information. In this case, there is a possibility that inappropriate remote instruction is notified to the subject vehicle. For example, assume that remote instruction to overtake the vehicle parked on the road is notified to the subject vehicle as the remote control information. In this case, there is a case where the preceding vehicle and the subject vehicle start overtaking at the same time, and other traffic participants are troubled.

[0041] In addition, for example as shown in Figure 2 , assume that location information that takes road construction as an important factor is registered. Assume that the location information includes position information where road construction is performed, and information of remote assistance for the purpose of calling the operator. In remote assistance in this case, for example, the operator confirms a camera image that captures the situation around the subject vehicle, recognizes a hand flag of a guide at a road construction site, and in a case where a go signal is given, remotely controls the subject vehicle to travel along a travel path that avoids the construction site. In this case, assume that in a case where traffic congestion or the like occurs due to the road construction, and Figure 1Similar to the situation where remote assistance is requested from autonomous vehicles upon detecting traffic congestion, etc., the similarity criteria for determining location information are strictly enforced. Figure 1 In similar situations, if the request is deemed to be for remote assistance at a location without registered location information, an operator is called. Furthermore, if the criteria for determining location information similarity are relaxed, an operator is called even when the distance to the guide is still considerable. As a result, the called operator must wait until the vehicle reaches the guide in order to provide remote assistance, thus incurring a prolonged constraint on the operator.

[0042] Therefore, in each of the following embodiments, the location information includes the area of ​​influence (described in detail below) for registration. Furthermore, if a remote assistance request from an autonomous vehicle is made within the area of ​​influence, it is determined that the request arose from traffic congestion or other factors that are inherently important due to the registered location information (such as parking on the road or road construction). In this case, remote assistance requests such as instructions to change actions like overtaking and operator calls are temporarily withheld. Moreover, when the autonomous vehicle arrives at the location where remote assistance was originally required, remote assistance is provided to the autonomous vehicle.

[0043] Hereinafter, each embodiment will be described with reference to the accompanying drawings.

[0044] <First Implementation>

[0045] like Figure 3 As shown, the autonomous driving system 100 of the first embodiment includes a remote assistance device 110 installed in a control center or the like, or equipped in the cloud, an on-board device 130 mounted on a vehicle, and an operator terminal 150 operated by an operator 50. Furthermore, the number of on-board devices 130 and operator terminals 150 included in the autonomous driving system 100 is not limited to [specific number missing]. Figure 3 The example shown.

[0046] In the autonomous driving system 100, for example, a private car, a bus, a taxi, a vehicle for a car sharing, and the like are controlled by the in-vehicle device 130 to perform autonomous driving. Hereinafter, a vehicle on which the in-vehicle device 130 is mounted and which performs autonomous driving will be referred to as an AD (Autonomous Driving) vehicle 30. The in-vehicle device 130 is connected to the remote assistance device 110 via a network such as the Internet, respectively. In addition, the operator terminal 150 is connected to the remote assistance device 110 via the network, respectively. Thus, each in-vehicle device 130 and each operator terminal 150 are connected via the remote assistance device 110. Further, the operator terminal 150 can be configured in the same manner as the remote assistance device 110 in a control center and connected to the remote assistance device 110 via an intranet, or can be configured outside the control center and connected to the remote assistance device 110 via the Internet.

[0047] The remote assistance device 110 is implemented by an information processing device such as a personal computer or a server device, for example. In the present embodiment, the remote assistance device 110 is implemented by a server device. Figure 4 A hardware configuration of the remote assistance device 110 is shown. As shown in FIG. 1, the remote assistance device 110 has a CPU (Central Processing Unit) 12, a memory 14, a storage device 16, an input device 18, an output device 20, a storage medium reading device 22, and a communication I / F (Interface) 24. Each of the configurations is connected to each other in a communicable manner via a bus 26. Figure 4

[0048] The autonomous driving program for executing the remote assistance device side autonomous driving process described later is stored in the storage device 16. The CPU 12 is a central arithmetic processing unit that executes various programs or controls each configuration. That is, the CPU 12 reads out a program from the storage device 16 and executes the program using the memory 14 as a work area. The CPU 12 performs control of each configuration and various arithmetic processing according to the program stored in the storage device 16.

[0049] The memory 14 is configured by a RAM (Random Access Memory) and temporarily stores programs and data as a work area. The storage device 16 is configured by a ROM (Read Only Memory), a HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like, and stores various programs including an operating system and various data.

[0050] ​The input device 18 is, for example, a keyboard, a mouse, or the like, and is a device for making various inputs. The output device 20 is, for example, a display, a printer, or the like, and is a device for outputting various information. The output device 20 can also function as the input device 18 by employing a touch panel display.

[0051] The storage medium reading device 22 reads data stored in various storage media such as a CD (Compact Disc) ROM, a DVD (Digital Versatile Disc) ROM, a Blu-ray disc, a USB (Universal Serial Bus) memory, and the like, writes data to the storage media, and the like.

[0052] The communication I / F 24 is an interface for communication with other devices using standards such as Ethernet (registered trademark), FDDI, Wi-Fi (registered trademark), and the like.

[0053] The in-vehicle device 130 is realized by, for example, an ECU (Electronic Control Unit) or the like. In Figure 5 The hardware configuration of the in-vehicle device 130 is shown. As Figure 5 indicated, the in-vehicle device 130 is the same as the remote assistance device 110 and has a CPU 32, a memory 34, a storage device 36, an input / output I / F 38, a storage medium reading device 42, and a communication I / F 44. The storage device 36 stores an autonomous driving program for executing an AD vehicle side autonomous driving process described later. The respective configurations are connected to each other in a communicable manner via a bus 49. The GPS 46, the sensing device 48, and various vehicle configurations such as other ECUs are connected to the input / output I / F 38.

[0054] The GPS 46 measures the position of the in-vehicle device 130, that is, the position of the AD vehicle 30, and outputs position information at each measurement time. The position information is, for example, latitude and longitude. The sensing device 48 is, for example, a camera, a laser radar, or the like, and outputs sensing data that senses the state of the periphery of the vehicle. Information indicating the state of the AD vehicle 30 such as vehicle speed and azimuth angle is input from various vehicle configurations to the in-vehicle device 130.

[0055] The operator terminal 150 is realized by an information processing device such as a personal computer, a tablet terminal, or the like. The hardware configuration of the operator terminal 150 is substantially the same as the hardware configuration of the remote assistance device 110 shown in Figure 4 and thus the description is omitted.

[0056] Next, the functional configuration of the remote assistance device 110 of the first embodiment is described with reference to Figure 6 Figure 6 ​As shown, the remote assistance device 110 includes a vehicle status monitoring unit 112, an assistance method determination unit 114, a remote assistance implementation unit 116, and a location information registration unit 118. Through... Figure 4 The CPU 12 shown implements each functional unit. In addition, the remote assistance device 110 stores vehicle information DB (database) 120, location information DB 122, map DB 124, and range of influence DB 126 in a designated storage area.

[0057] The vehicle status monitoring unit 112 acquires vehicle information, remote assistance requests, and sensing data transmitted from the on-board unit 130. The vehicle information includes the identification information of the AD vehicle 30 (on-board unit 130), i.e., the vehicle ID; vehicle speed, azimuth angle, and other information detected by the various vehicle configurations of the AD vehicle 30; and the location information of the AD vehicle 30 measured by the GPS 46. When the vehicle status monitoring unit 112 acquires vehicle information, it appends the date the vehicle information was acquired to the acquired information and stores it, for example, in a [location not specified]. Figure 7 The vehicle information shown is DB120.

[0058] A remote assistance request is a message sent from the on-board unit 130 when remote assistance is needed in the AD vehicle 30 (details to follow). If the vehicle status monitoring unit 112 acquires the remote assistance request along with vehicle information, it transfers the acquired remote assistance request and vehicle information to the assistance method determination unit 114. Additionally, if the vehicle status monitoring unit 112 acquires sensing data, it transfers the acquired sensing data to the remote assistance implementation unit 116.

[0059] In location information DB122, for AD vehicle 30, location information is registered for each location where remote assistance was previously implemented, including the location where remote assistance was implemented and the range of influence defined by at least the distance from that location. Figure 8 This shows an example of location information DB122. Figure 8 In the example, each row (each record) is equivalent to location information for a given location. Each location information includes location identification information, i.e., "location ID," location information, i.e., "location," the "key factors" that enabled the remote assistance request, "range of influence (distance)," "range of influence (time)," "response methods," and "characteristics," among other information.

[0060] The influence range refers to a range in which there is a possibility that the automatic driving of the AD vehicle 30 according to the pre-set action plan is hindered due to traffic congestion and the like caused by the "important factor" itself. The "influence range (distance)" is information that specifies the influence range according to the distance from the "position", and the "influence range (time)" is information that specifies the influence range according to the time at which the "important factor" is generated. The detailed determination method of the influence range will be described later.

[0061] The "countermeasure" is a method of remote assistance from the remote assistance device 110, and in the present embodiment, there are a method of automatically remotely instructing the contents of the action change such as the overtaking, the lane change, and the like, and a method of calling the operator 50 to cope with it. The details of the countermeasure will be described later. In addition, in a case where the "important factor" is traffic congestion, for example, and the natural elimination of the important factor is required, the countermeasure based on the remote assistance is not performed, so there is also a case where the "countermeasure" is "none" (indicated as "-" in the following table). Figure 8

[0062] The "feature quantity" is a feature quantity extracted from the sensing data that senses the situation around the sensing location at the time when the remote assistance is implemented at the corresponding location. Hereinafter, the feature quantity included in the location information will also be referred to as the "registered feature". The details of the feature quantity will be described later.

[0063] The assistance method determination section 114 determines whether the remote assistance request is generated within the influence range included in any one of the location information registered in the location information DB 122, if the remote assistance request and the vehicle information are accepted. Specifically, the assistance method determination section 114 retrieves the location information having the "position" with the distance from the position of the AD vehicle 30 shown by the position information included in the vehicle information within a predetermined range, from the location information DB 122. In a case where a plurality of pieces of location information are retrieved, the assistance method determination section 114 can refer to the map DB 124, for example, and select the location information having the "position" that is in front of and closest to the AD vehicle 30 on the travel path. The map DB 124 stores map data including information of the traffic environment such as the number of lanes, signals, and the like. The assistance method determination section 114 can determine the direction of travel and the travel lane of the AD vehicle 30 by mapping the travel trajectory of the AD vehicle 30 indicated by the time series of the position information stored in the vehicle information DB 120 for each date to the map data. In addition, the assistance method determination section 114 can also acquire the vehicle information including information of the travel scheduled route and the travel lane from the AD vehicle 30.

[0064] ​Then, the assistance method decision section 114 decides whether a remote assistance request has been made within the influence range, based on the "influence range (distance)" and the "influence range (time)" included in the searched location information. More specifically, the assistance method decision section 114 decides that a remote assistance request has been made within the influence range, in a case where an "important factor" on the location shown by the searched location information has occurred based on the "influence range (time)", and the AD vehicle 30 is present within the range determined according to the "influence range (distance)".

[0065] The assistance method decision section 114 decides to implement remote assistance based on the location information, in a case where it is decided that a remote assistance request has been made within the influence range, and hands over the vehicle information and the corresponding location information to the remote assistance implementation section 116. In addition, the assistance method decision section 114 decides to implement remote assistance based on the operator 50, in a case where it is decided that the occurrence of a remote assistance request is not within the influence range, hands over the vehicle information to the remote assistance implementation section 116, and instructs the call of the operator 50.

[0066] The remote assistance implementation section 116 searches for an operator 50 in an idle state, if instructed to call the operator 50 from the assistance method decision section 114. In addition, the remote assistance implementation section 116 acquires the sensing data from the AD vehicle 30. Then, the remote assistance implementation section 116 displays a screen that images the acquired sensing data on the operator terminal 150 corresponding to the searched operator 50, and instructs remote assistance. The remote assistance implementation section 116 transmits remote control information generated based on the operation of the operator terminal 150 by the operator 50 to the on-vehicle device 130.

[0067] The remote assistance implementation section 116 decides whether a countermeasure method is registered for the location information, in a case where the vehicle information and the location information are accepted. Specifically, the remote assistance implementation section 116 decides that a countermeasure method is registered, in a case where the "countermeasure method" of the location information is other than "none", and decides that a countermeasure method is not registered, in a case where the "countermeasure method" is "none". In a case where a countermeasure method is not registered, the remote assistance implementation section 116 instructs the continuation of autonomous driving based on the action plan set in advance. As the instruction, either the follow-up of the preceding vehicle can be instructed explicitly, or the AD vehicle 30 can be notified that no action change is required.

[0068] Further, in the case where the response method is registered, the remote assistance implementation section 116 instructs the transmission of the sensing data to the in-vehicle device 130, and acquires the sensing data indicating the situation around the current AD vehicle 30. The remote assistance implementation section 116 extracts a feature quantity (hereinafter, referred to as "sensor feature") from the acquired sensing data, and determines whether the sensor feature coincides with the registered feature included in the accepted location information. For example, the remote assistance implementation section 116 can determine that the two feature quantities coincide in the case where the degree of similarity between the feature quantities is equal to or higher than a predetermined value. In the case where the two feature quantities do not coincide, the remote assistance implementation section 116 instructs the continuation of the automatic driving based on the pre-set action plan, and instructs the transmission of the sensing data, as in the case where the response method is not registered.

[0069] On the other hand, in the case where the two feature quantities coincide, the remote assistance implementation section 116 implements the remote assistance in accordance with the response method included in the location information. Specifically, in the case where the action change is registered as the response method, the remote assistance implementation section 116 generates the remote control information based on the content of the action change, and transmits the remote control information to the in-vehicle device 130. For example, the action implemented at the time of the action change, or the travel trajectory traveled at the time of the action change, and the like are generated as the remote control information. Further, in the case where the operator call is registered as the response method, the remote assistance implementation section 116 implements the remote assistance identical to the case where the operator 50 is instructed to call from the assistance method determination section 114.

[0070] For example, assume the case where the location information including the registered feature indicating the on-street parking (the dotted-elliptical portion of the left drawing) of FIG. 1 is registered, and the remote assistance request occurs at the position of the AD vehicle 30 shown in the right drawing of FIG. 1. Figure 9 Figure 9 For example, assume the case where the location information including the registered feature indicating the on-street parking (the dotted-elliptical portion of the left drawing) of FIG. 1 is registered, and the remote assistance request occurs at the position of the AD vehicle 30 shown in the right drawing of FIG. 1. Figure 9 Figure 9 For example, assume the case where the location information including the registered feature indicating the on-street parking (the dotted-elliptical portion of the left drawing) of FIG. 1 is registered, and the remote assistance request occurs at the position of the AD vehicle 30 shown in the right drawing of FIG. 1. Figure 9

[0071] Further, for example, assume the case where the location information including the registered feature indicating the road construction (the dotted-elliptical portion of the left drawing) of FIG. 2 is registered, and the remote assistance request occurs at the position of the AD vehicle 30 shown in the right drawing of FIG. 2. Figure 10 Figure 10 Further, for example, assume the case where the location information including the registered feature indicating the road construction (the dotted-elliptical portion of the left drawing) of FIG. 2 is registered, and the remote assistance request occurs at the position of the AD vehicle 30 shown in the right drawing of FIG. 2.​​​​Figure 10 The position of the AD car 30 shown in the left drawing generates a remote assistance request. At this time, since the registered feature and the sensor feature are not in agreement, the remote assistance implementation section 116 determines that the remote assistance request has been generated due to a traffic jam or the like caused by road construction, and instructs continuation of automatic driving based on the action plan. Then, if the AD car 30 is slowly driven by the automatic driving based on the action plan, and reaches the position shown in the right drawing, where the registered feature and the sensor feature are in agreement, the remote assistance implementation section 116 calls the operator 50 in accordance with the coping method included in the location information. The remote assistance implementation section 116 accepts the travel trajectory (solid arrow in the drawing) that avoids the road construction portion, which is generated by the operation of the operator terminal 150 by the operator 50. Figure 10 Figure 10

[0072] As described above, in the case where a remote assistance request is generated in the influence range of a traffic jam or the like caused by the original important factor of the remote assistance request, the remote assistance is temporarily reserved until the AD car 30 reaches the location where the remote assistance should originally be implemented. Thereby, it is possible to suppress generation of unnecessary calling of the operator 50, and implementation of inappropriate remote instruction.

[0073] The location information registration section 118 generates location information related to the implemented remote assistance, and registers it in the location information DB 122, in the case where it is determined that generation of the remote assistance request is not in the influence range, and the remote assistance based on the operator 50 is implemented. Specifically, the location information registration section 118 accepts, from the operator 50 via the operator terminal 150, information related to the position of the obstacle that is the important factor of the generation of the remote assistance request, and the content of the coping method such as action change.

[0074] More specifically, the operator 50 specifies the region of the obstacle that is the important factor on the camera image or the three-dimensional point group data based on the laser radar, or the like, of the surroundings of the AD car 30 displayed on the operator terminal 150. For example, in the case where the important factor is a road parked vehicle, the region on the camera image or the three-dimensional point group data of the road parked vehicle is specified. The important factor and the content of the coping method are directly input to the operator terminal 150 by the operator 50, or input from selection from a pull-down menu, or the like. The information specified and input by the operator terminal 150 is transmitted to the remote assistance device 110.

[0075] ​​The location information registration unit 118 determines the position of the obstacle in the world coordinate system based on the position of the region of the obstacle that becomes a factor on the camera image or the three-dimensional point group data accepted and the position of the AD vehicle 30, and registers the position as "position" of the location information (hereinafter, also referred to as "location position"). Alternatively, at the time when the remote assistance is implemented, the obstacle and the AD vehicle 30 exist in a close position, so the position of the AD vehicle 30 can also be the location position. Further, in a case where there is no specific obstacle that becomes a factor of the remote assistance request, such as a case where the factor is traffic congestion, it is also acceptable to register the position of the AD vehicle 30 as the location position. Furthermore, the location information registration unit 118 extracts a feature amount that becomes a registration feature from the region of the obstacle that becomes a factor on the camera image or the three-dimensional point group data.

[0076] In addition, the location information registration unit 118 determines the influence range based on the factor of the remote assistance request. For example, the location information registration unit 118 refers to the influence range DB 126 that defines a default value of the influence range for each factor, and determines the influence range to be registered as the location information. In Figure 11 An example of the influence range DB 126 is shown. In Figure 11 In the example, "influence range (distance)" is a distance from the location position. As shown in Figure 12 In a case where the factor is road construction or a parked vehicle on the road, the influence range is defined as a distance from the location position, that is, the obstacle that becomes a factor, to the downstream side in the vehicle travel direction. In addition, in a case where the position of the obstacle that becomes a factor and the like is predicted to change with time, such as a case where the factor is traffic congestion, the influence range is defined as a distance from the location position to the upstream side and the downstream side in the vehicle travel direction.

[0077] "Influence range (time)" is a predicted value of the duration from the generation of the factor to the elimination of the factor according to the factor. In Figure 11 In the example, in a case where the factor is road construction, construction information is acquired from an external organization or the like at the time of registration of the location information, and the date of the road construction is determined as the "influence range (time)" to be registered as the location information.

[0078] Further, the method of determining the influence range is not limited to the example using the influence range DB 126 described above. For example, the location information registration unit 118 can determine the influence range to be registered as the location information based on statistical information related to past traffic around the location. Referring to Figure 13more specifically, the maximum value of P(x) of the corresponding location is determined as the "influence range (time)", and the maximum value of P(y) is determined as the "influence range (distance)". Alternatively, the time and distance at which the simultaneous probability P(x, y) is the maximum can be determined as the "influence range (time)" and the "influence range (distance)". Furthermore, it is not limited to the case where the probability distribution is calculated, and the average value, median value, or the like can be calculated for the time and distance of the traffic congestion caused by the on-street parking at each location where the on-street parking has occurred in the past, and the statistical value can be determined as the "influence range (time)" and the "influence range (distance)".

[0079] Further, the location information registration unit 118 can also determine the influence range as described above based on at least one of the time period during which remote assistance is implemented and the road characteristics at the corresponding location. For example, the time period of rush hours is determined in advance, and the location information registration unit 118 makes the influence range determined as described above n (n > 1, for example, 1.5) times larger in a case where the time during which remote assistance is implemented is included in the time period. Further, the location information registration unit 118 acquires the road characteristics at the location where remote assistance is implemented from the map DB 124, and makes the influence range determined as described above n (n > 1, for example, 1.5) times larger in a case where the road at the corresponding location is a one-way single lane, for example. Thus, the influence range can be optimized based on at least one of the time period and the road characteristics.

[0080] Further, the location information registration unit 118 deletes the location information registered in the location information DB 122 for which the time specified by the "influence range (time)" has elapsed. For example, for the location information for which the "influence range (time)" is registered with a time of "thirty minutes" or the like, the location information registration unit 118 deletes the location information in a case where thirty minutes have elapsed from the time when the location information is registered in the location information DB 122. Further, for example, for the location information for which the "influence range (time)" is registered with a period of a specified date, the location information registration unit 118 deletes the location information after the end of the registered period. Furthermore, the location information registration unit 118 determines whether any of the AD vehicles 30 has passed through the location shown by the location information registered in the location information DB 122 without requiring remote assistance based on the travel trajectories of the respective AD vehicles 30 obtained from the vehicle information DB 120. In a case where any of the AD vehicles 30 has passed through the location without requiring remote assistance, the location information registration unit 118 deletes the location information of the location from the location information DB 122.

[0081] Next, the function configuration of the in-vehicle device 130 of the first embodiment will be described with reference to Figure 14 Fig. 2. As shown in Fig. 2, the in-vehicle device 130 includes a vehicle information acquisition section 132, a surrounding situation monitoring section 134, a transmission / reception section 136, and a travel control section 138. Each functional section is realized by the CPU 32 shown in Fig. 1. Figure 14 Figure 5 The vehicle information acquisition section 132 acquires vehicle information including information such as a vehicle speed and a direction angle detected by various vehicle configuration of the AD vehicle 30, and position information of the AD vehicle 30 measured by the GPS 46, and hands over the vehicle information to the transmission / reception section 136.

[0082] The surrounding situation monitoring section 134 acquires sensing data output from the sensing device 48 that senses a situation of the surroundings of the vehicle. As described above, the sensing device 48 is, for example, a camera or a laser radar, and the sensing data is, for example, an image of the surroundings of the AD vehicle 30 captured by the camera, or three-dimensional point cloud data indicating three-dimensional coordinates of other vehicles, buildings, and the like in the surroundings of the AD vehicle 30 measured by the laser radar. The surrounding situation monitoring section 134 determines whether a travel path based on a predetermined action plan is blocked based on the acquired sensing data.

[0083] Specifically, the surrounding situation monitoring section 134 determines the presence or absence of an obstacle that blocks the travel path based on the camera image or the three-dimensional point cloud data. The obstacle is a stationary object, or a moving object that moves at a speed equal to or lower than a predetermined speed. Thus, a preceding vehicle that temporarily stops or moves slowly due to traffic congestion or the like is also determined as the obstacle. The surrounding situation monitoring section 134 notifies the transmission / reception section 136 of the determination result of whether the travel path is blocked, and hands over the acquired sensing data to the transmission / reception section 136 and the travel control section 138.

[0084] Specifically, the surrounding situation monitoring section 134 determines the presence or absence of an obstacle that blocks the travel path based on the camera image or the three-dimensional point cloud data. The obstacle is a stationary object, or a moving object that moves at a speed equal to or lower than a predetermined speed. Thus, a preceding vehicle that temporarily stops or moves slowly due to traffic congestion or the like is also determined as the obstacle. The surrounding situation monitoring section 134 notifies the transmission / reception section 136 of the determination result of whether the travel path is blocked, and hands over the acquired sensing data to the transmission / reception section 136 and the travel control section 138.

[0085] ​The transmitting and receiving unit 136 appends the vehicle ID of AD vehicle 30 to the vehicle information received from the vehicle information acquisition unit 132 and sends it to the remote assistance device 110 at regular intervals. Additionally, the transmitting and receiving unit 136 receives remote assistance-related instructions from the remote assistance device 110. These instructions include remote control information such as response methods based on registered location information or operator 50's actions, instructions to continue autonomous driving based on a pre-set action plan, and instructions to transmit sensor data. If the transmitting and receiving unit 136 receives a sensor data transmission instruction from the remote assistance device 110, it transmits the sensor data received from the surrounding situation monitoring unit 134 to the remote assistance device 110. Furthermore, if the transmitting and receiving unit 136 receives a determination from the surrounding situation monitoring unit 134 indicating a blocked driving path, and has not received any remote assistance-related instructions from the remote assistance device 110, it sends a remote assistance request along with the vehicle information to the remote assistance device 110. In addition, when the transmitting and receiving unit 136 receives remote control information from the remote assistance device 110, it hands over the received remote control information to the driving control unit 138.

[0086] The driving control unit 138 performs action plan-based autonomous driving control when no remote assistance is requested, or when instructed by the remote assistance device 110 to continue action plan-based autonomous driving. Autonomous driving in this situation can be achieved, for example, using methods previously known related to Level 4 autonomous driving, based on a pre-set path, sensor data indicating the surrounding conditions of the AD vehicle 30, and information from the map DB124; therefore, detailed descriptions are omitted.

[0087] Furthermore, if the driving control unit 138 receives remote control information from the transmitting and receiving unit 136, it executes autonomous driving control based on the action plan reflecting the remote control information. Therefore, the driving control unit 138, based on sensor data, also considers the movement of surrounding vehicles, such as... Figure 9 or Figure 10 As shown by the solid arrow in the image, the control is for AD vehicle 30 to travel on a path that avoids obstacles, which are an important factor in the requirements of remote assistance.

[0088] Next, the operation of the autonomous driving system 100 of the first embodiment will be described. The action plan is set in the AD vehicle 30, and the in-vehicle device 130 starts the control of the autonomous driving based on the action plan. Also, the vehicle information acquisition section 132 acquires the vehicle information from the various vehicle configurations of the AD vehicle 30 and the GPS 46, and the transmission / reception section 136 periodically performs the process of transmitting the vehicle information to which the vehicle ID is added to the remote assistance device 110. In the remote assistance device 110, the vehicle state monitoring section 112 stores the received vehicle information in the vehicle information DB 120.

[0089] In addition, in parallel with the above process, the AD vehicle side autonomous driving process shown in Figure 15 is executed in the in-vehicle device 130. Also, if the remote assistance device 110 receives the remote assistance request transmitted from the in-vehicle device 130, the remote assistance device side autonomous driving process shown in Figure 16 is executed in the remote assistance device 110. Hereinafter, the AD vehicle side autonomous driving process and the remote assistance device side autonomous driving process will be described in detail. Note that the AD vehicle side autonomous driving process and the remote assistance device side autonomous driving process are one example of the autonomous driving method of the present disclosure.

[0090] First, one example of the AD vehicle side autonomous driving process will be described with reference to Figure 15

[0091] In step S112, the surrounding situation monitoring section 134 acquires the sensing data output from the sensing device 48. Then, the surrounding situation monitoring section 134 monitors the surrounding situation of the AD vehicle 30 by determining whether the travel path based on the action plan decided in advance is blocked based on the acquired sensing data.

[0092] Next, in step S114, the transmission / reception section 136 determines whether the determination result of the surrounding situation monitoring section 134 in the above step S112 indicates that the travel path is blocked. In the case where the travel path is not blocked, the process moves to step S116, and in the case where the travel path is blocked, the process moves to step S118.

[0093] In step S116, the travel control section 138 continues the control of the autonomous driving based on the action plan, and the process returns to step S112.

[0094] ​On the other hand, in step S118, the transmission / reception section 136 determines whether or not the remote control information has been received from the remote assistance device 110. In the case where the remote control information has been received, that is, in the case where the content of the action change based on the registered location information, or the content of the action change input by the operator changes the action plan, the process moves to step S120, and in the case where the remote control information has not been received, the process moves to step S122. In step S120, the travel control section 138 executes the control of the automatic driving based on the action plan reflecting the received remote control information, and the process returns to step S112.

[0095] In step S122, the transmission / reception section 136 determines whether or not the instruction to continue the automatic driving based on the action plan has been received from the remote assistance device 110. In the case where the instruction to continue the automatic driving based on the action plan has been received, the process moves to step S124, and in the case where it has not been received, the process moves to step S130. In step S124, the travel control section 138 continues the automatic driving based on the action plan.

[0096] Next, in step S126, the transmission / reception section 136 determines whether or not the transmission instruction of the sensing data has been received from the remote assistance device 110. In the case where the transmission instruction of the sensing data has been received, the process moves to step S128, the transmission / reception section 136 transmits the sensing data to the remote assistance device 110, and the process returns to step S112. In the case where the transmission instruction of the sensing data has not been received, step S128 is skipped, and the process returns to step S112.

[0097] In step S130, the transmission / reception section 136 transmits the remote assistance request to the remote assistance device 110 together with the vehicle information, and the process returns to step S112.

[0098] Next, with reference to Figure 16 , one example of the remote assistance device side automatic driving process will be described.

[0099] In step S162, the vehicle state monitoring section 112 acquires the remote assistance request transmitted from the in-vehicle device 130 together with the vehicle information. Next, in step S164, the assistance method decision section 114 retrieves the location information of the "location" having the distance from the position of the AD vehicle 30 shown by the position information included in the acquired vehicle information within a prescribed range from the location information DB 122. That is, the location information of the registered location in the vicinity of the AD vehicle 30 is retrieved.

[0100] Next, in step S166, the assistance method decision section 114 determines whether the remote assistance request has been made within the influence range of the registered location based on the "influence range (distance)" and the "influence range (time)" included in the retrieved location information. In the case of being within the influence range, the processing moves to step S172, and in the case of not being within the influence range, the processing moves to step S168. Further, in the above step S164, in the case where the location information DB 122 does not have the location information of the registered location in the vicinity of the AD vehicle 30, the processing also moves to step S168.

[0101] In step S168, the assistance method decision section 114 decides to implement the remote assistance based on the operator 50, the remote assistance implementation section 116 calls the operator 50, implements the remote assistance based on the operator 50, and transmits the remote control information to the on-vehicle device 130.

[0102] Next, in step S170, the location information registration section 118 accepts the information of the important factor in which the remote assistance request has been made, the position of the obstacle that becomes the important factor, and the contents of the countermeasure such as the action change from the operator 50 via the operator terminal 150. Then, the location information registration section 118 generates the location information based on the accepted information and registers it in the location information DB 122, and the remote assistance device side automatic driving processing ends.

[0103] On the other hand, in step S172, the remote assistance implementation section 116 determines whether the countermeasure is registered in the location information retrieved in the above step S164. In the case where the countermeasure is registered, the processing moves to step S174, and in the case where it is not registered, that is, in the case where the countermeasure is "none", the processing moves to step S182.

[0104] In step S174, the remote assistance implementation section 116 instructs the transmission of the sensing data to the on-vehicle device 130 and acquires the sensing data indicating the current situation around the AD vehicle 30. Next, in step S176, the remote assistance implementation section 116 determines whether the sensor feature extracted from the acquired sensing data coincides with the registered feature included in the location information retrieved in the above step S164. In the case where the two features coincide, the processing moves to step S180, and in the case where they do not coincide, the processing moves to step S178.

[0105] In step S178, the implementation of the remote assistance is temporarily held, the transmission and reception section 136 transmits the instruction to continue the automatic driving based on the pre-set action plan to the on-vehicle device 130, and the processing returns to step S174.

[0106] In step S180, the AD vehicle 30 approaches an obstacle that is a crucial factor in the remote assistance requirement, so the remote assistance implementation unit 116 implements remote assistance based on the response method contained in the location information retrieved in step S164 above. Then, the automatic driving processing on the remote assistance device side ends.

[0107] In step S182, while waiting for important factors such as traffic congestion to disappear naturally, the transmitting and receiving unit 136 sends an instruction to the vehicle-mounted device 130 to continue the automatic driving based on the pre-set action plan, and the automatic driving processing on the remote assistance device side ends.

[0108] As explained above, in the autonomous driving system of the first embodiment, for the AD vehicle, location information including the range of influence is pre-registered for each location where remote assistance has been implemented in the past. Furthermore, if a remote assistance request arises from an AD vehicle within the range of influence of the registered location information, the implementation of remote assistance is temporarily suspended, allowing autonomous driving based on the action plan to continue. This suppresses unnecessary operator calls.

[0109] Furthermore, in the autonomous driving system of the first embodiment, remote assistance is implemented when the registered features match the sensor features, that is, when the AD vehicle approaches an obstacle that is an important factor for requiring remote assistance. Thus, remote assistance can be implemented at appropriate timing, and inappropriate implementation of remote assistance can be suppressed.

[0110] <Modifications of the First Embodiment>

[0111] In the first embodiment, the determination of whether the location is within its influence range and the determination of the consistency between sensor features and registered features are described on the remote assistance device 110 side. However, as a variation, these processes can also be performed on the AD vehicle 30 side. Figure 17 The functional configuration of the remote assistance device 110A in the modified example is shown. Figure 18 The functional configuration of the vehicle-mounted device 130A in the modified example is shown. Furthermore, detailed descriptions of parts identical to the remote assistance device 110 and the vehicle-mounted device 130 in the first embodiment are omitted.

[0112] like Figure 17As shown, the remote assistance device 110A includes a vehicle state monitoring section 112A, a remote assistance implementation section 116A, and a location information registration section 118. The vehicle state monitoring section 112A retrieves location information in the vicinity of the position of the on-vehicle device 130A (AD vehicle 30) from the location information DB 122 based on vehicle information acquired from the on-vehicle device 130A, and transmits it to the on-vehicle device 130A. Further, the vehicle state monitoring section 112A can transmit all of the location information existing within a prescribed range from the AD vehicle 30, or can transmit all of the location information existing on a travel scheduled route in the case where the travel scheduled route can be acquired from the AD vehicle 30. The remote assistance implementation section 116A is responsible for processing relating to remote assistance of the operator 50 among the functions of the remote assistance implementation section 116 in the first embodiment.

[0113] As shown, the on-vehicle device 130A includes a vehicle information acquisition section 132, a surrounding situation monitoring section 134, a transmission / reception section 136A, an assistance method decision section 114A, an assistance implementation section 117A, a travel control section 138, and a location information DB 122A. The transmission / reception section 136A receives location information from the remote assistance device 110A, and updates the contents of the location information DB 122A based on the received location information. Figure 18

[0114] The assistance method decision section 114A determines whether a remote assistance request has been generated within an influence range included in the location information based on the location information stored in the location information DB 122A, in the same manner as the assistance method decision section 114 in the first embodiment. The assistance method decision section 114A decides to implement remote assistance based on the location information and hands over the location information to the assistance implementation section 117A in the case where it is determined that a remote assistance request has been generated within the influence range. Further, the assistance method decision section 114A decides to implement remote assistance based on the operator 50 and instructs a call of the operator 50 to the remote assistance device 110A via the transmission / reception section 136A in the case where it is determined that generation of a remote assistance request is not within the influence range.

[0115] The assistance implementation section 117A decides the action content to be implemented in the AD vehicle 30 by the same processing as the generation of remote control information based on the location information among the functions of the remote assistance implementation section 116 in the first embodiment, and hands it over to the travel control section 138.

[0116] ​Next, one example of the AD vehicle side automatic driving processing and the remote assistance device side automatic driving processing in the modified example will be described. Further, the same processing as the AD vehicle side automatic driving processing and the remote assistance device side automatic driving processing in the first embodiment will be omitted from detailed description. The remote assistance device 110A retrieves the point information existing in the vicinity of the AD vehicle 30 based on the vehicle information received from the on-vehicle device 130A and transmits it to the on-vehicle device 130A. Then, the on-vehicle device 130A updates the content of the point information DB 122A based on the point information received from the remote assistance device 110A.

[0117] In parallel with the above processing, the AD vehicle side automatic driving processing illustrated in FIG. 12 is executed in the on-vehicle device 130A. Figure 19 First, one example of the AD vehicle side automatic driving processing in the modified example will be described with reference to FIG. 13. Figure 19 Steps S112 to S116 are the same as the AD vehicle side automatic driving processing in the first embodiment, and in the case where the travel path is not blocked, the automatic driving based on the action plan is continued.

[0118] In step S114, in the case where it is determined by the surrounding situation monitoring section 134 that the travel path is blocked, the processing moves to step S132. In step S132, the assistance method decision section 114A retrieves the point information of the "point" having a distance from the position of the AD vehicle 30 shown by the position information included in the vehicle information within a prescribed range from the point information DB 122A.

[0119] Next, in step S134, the assistance method decision section 114A determines whether the travel path is blocked within the influence range of the registered point based on the "influence range (distance)" and the "influence range (time)" included in the retrieved point information. In the case where it is within the influence range, the processing moves to step S136, and in the case where it is not within the influence range, the processing moves to step S144. Further, in the case where there is no point information of the registered point in the vicinity of the AD vehicle 30, and the point information cannot be retrieved from the point information DB 122A in the above S132, the processing also moves to step S144.

[0120] In step S136, the assistance implementation section 117A determines whether the coping method is registered in the point information retrieved in the above step S132. In the case where the coping method is registered, the processing moves to step S140, and in the case where it is not registered, i.e., in the case where the coping method is "none", the processing moves to step S124.

[0121] In step S140, the auxiliary implementation unit 117A acquires the sensing data acquired by the surrounding situation monitoring unit 134, and determines whether the sensor feature extracted from the sensing data coincides with the registered feature included in the location information searched in step S132 described above. In the case where the two features coincide, the process moves to step S142, and in the case where they do not coincide, the process moves to step S138.

[0122] In step S138, implementation of the action change is temporarily held, the travel control unit 138 continues automatic driving based on the action plan set in advance, and the process returns to step S140. In step S142, the auxiliary implementation unit 117A determines whether the coping method included in the location information is a method of calling the operator 50 or a method of automatically implementing the action change. In the case of calling the operator, the process moves to step S144, and in the case of the action change, the process moves to step S146.

[0123] In step S144, the transmission / reception unit 136A transmits the sensing data acquired by the surrounding situation monitoring unit 134 to the remote assistance device 110A, and transmits a remote assistance request to the remote assistance device 110A to cause the operator 50 to implement remote assistance. In the case where the process moves from step S142 to step S146, in step S146, the auxiliary implementation unit 117A generates control information showing the content of the action change based on the coping method included in the location information. Then, the travel control unit 138 performs control of automatic driving based on the action plan reflecting the control information, and the process returns to step S112. Further, in the case where the process moves from step S144 to step S146, in step S146, the travel control unit 138 performs control of automatic driving based on the action plan reflecting the remote control information received from the remote assistance device 110A, and the process returns to step S112.

[0124] Next, one example of the remote assistance device side automatic driving process of the modified example will be described with reference to Figure 20 to the example of the remote assistance device side automatic driving process of the modified example.

[0125] In step S162, the vehicle state monitoring section 112A acquires the vehicle information transmitted from the in-vehicle device 130A. In addition, in a case where the remote assistance is requested by the AD vehicle 30, the remote assistance request is acquired together with the vehicle information. Next, in step S184, the vehicle state monitoring section 112A retrieves the location information of the "location" having a distance from the location of the AD vehicle 30 shown by the position information included in the acquired vehicle information within a prescribed range from the location information DB 122. That is, the location information of the registered location in the vicinity of the AD vehicle 30 is retrieved. Then, the vehicle state monitoring section 112A transmits the retrieved location information to the in-vehicle device 130A. By repeating steps S162 and S184, the remote assistance device 110A can transmit the location information existing in the vicinity of the AD vehicle 30 to the in-vehicle device 130A in advance.

[0126] Next, in step S186, the remote assistance implementation section 116A determines whether or not the operator 50 is instructed to be called from the in-vehicle device 130A. In a case where the operator 50 is called, the processing moves to step S188, and in a case where the operator 50 is not called, in other words, in a case where the remote assistance request is not received, the automatic driving processing on the remote assistance device side is ended. This is because there is a case where the action change can be made on the in-vehicle device 130A side based on the transmitted location information.

[0127] In step S188, the remote assistance implementation section 116A calls the operator 50, implements the remote assistance based on the operator 50, and transmits the remote control information to the in-vehicle device 130A. Next, in step S170, the location information registration section 118 registers the location information in the location information DB 122 in a case where the remote assistance based on the operator 50 is implemented without registering the location information, and ends the automatic driving processing on the remote assistance device side.

[0128] As explained above, in the modification of the first embodiment, as with the first embodiment, it is possible to suppress the generation of the unnecessary operator call and the implementation of the inappropriate remote assistance.

[0129] <Second Embodiment>

[0130] Next, the second embodiment will be explained. Further, in the automatic driving system of the second embodiment, the same parts as the automatic driving system 100 of the first embodiment are attached with the same reference numerals and the detailed explanation is omitted. In addition, the hardware configuration of the remote assistance device and the in-vehicle device of the second embodiment is the same as that of the remote assistance device 110 and the in-vehicle device 130 of the first embodiment shown in FIG. 1, so the explanation is omitted. Figure 4 and Figure 5 The hardware configuration of the remote assistance device 110 and the in-vehicle device 130 of the first embodiment shown in FIG. 1 is the same as that of the second embodiment, so the explanation is omitted.

[0131] like Figure 3 As shown, the autonomous driving system 200 of the second embodiment includes a remote assistance device 210, an on-board device 130 mounted on the AD vehicle 30, and an operator terminal 150 operated by an operator 50.

[0132] Next, refer to Figure 6 The functional configuration of the remote assistance device 210 in the second embodiment will be described. For example... Figure 6 As shown, the remote assistance device 210 includes a vehicle status monitoring unit 112, an assistance method determination unit 114, a remote assistance implementation unit 216, and a location information registration unit 218. Through... Figure 4 The CPU12 shown implements each functional unit.

[0133] The remote assistance implementation unit 216 is the same as the remote assistance implementation unit 116 in the first embodiment, implementing remote assistance based on location information or based on operator 50. In addition, while maintaining the state of remote assistance, i.e., instructing the continuation of autonomous driving based on a pre-set action plan, the remote assistance implementation unit 216 determines whether the AD vehicle 30 requires new remote assistance. If new remote assistance is required, the remote assistance implementation unit 216 implements remote assistance based on operator 50.

[0134] For example, such as Figure 21 As shown in the left figure, suppose that AD vehicle 30 gets stuck in traffic congestion caused by parking on the road, thus generating a remote assistance request. Furthermore, in this example, suppose that location information DB122 does not register location information that considers parking on the road as a significant factor, or that the location of AD vehicle 30 at the time the remote assistance request was generated is outside the influence range of the registered location information. In this case, operator 50 is called to perform remote assistance and register new location information in location information DB122. However, at this moment, operator 50 cannot ascertain the existence of the original significant factor, i.e., parking on the road, based on the surrounding conditions of AD vehicle 30, so instructs the continuation of autonomous driving based on the action plan and registers location information that considers traffic congestion as a significant factor.

[0135] Then, the lead vehicle overtakes the parked vehicles on the road, and AD car 30 follows the lead vehicle slowly forward, as... Figure 21 As shown in the central diagram, the vehicle approaches a parked vehicle on the road. However, AD vehicle 30 is unable to move because it is instructed to continue autonomous driving based on the action plan. In the second embodiment, this situation is determined to be a state requiring new remote assistance, and further remote assistance is implemented.

[0136] Specifically, the remote assistance implementation unit 216 determines whether the AD vehicle 30 needs new remote assistance in the influence range, based on at least one of the state of the AD vehicle 30 shown by the vehicle information stored in the vehicle information DB 120 and the sensed data acquired from the in-vehicle device 130. The remote assistance implementation unit 216 determines that the state in which the automatic driving based on the action plan cannot be continued, that is, the state in which new remote assistance is needed, in the case where the AD vehicle 30 is parked for a prescribed time or more. In addition, the remote assistance implementation unit 216 sequentially acquires sensed data and determines whether the degree of dissimilarity between the feature quantity based on the acquired sensed data and the feature quantity based on the sensed data acquired last time is equal to or more than a prescribed value. In the case where the degree of dissimilarity between the feature quantities is equal to or more than the prescribed value, it is indicated that the sensed data is drastically changed because the obstacle in front of the AD vehicle 30 is changed from the preceding vehicle to the vehicle parked on the road due to the preceding vehicle overtaking the vehicle parked on the road. In this case, the remote assistance implementation unit 216 can also determine the state in which new remote assistance is needed.

[0137] The remote assistance implementation unit 216, if it determines the state in which new remote assistance is needed, generates remote control information indicating a travel trajectory for overtaking the vehicle parked on the road (solid arrow in the right drawing of FIG. 10) by the operation of the operator 50, for example. Figure 21 Figure 21 The remote assistance implementation unit 216, if it determines the state in which new remote assistance is needed, generates remote control information indicating a travel trajectory for overtaking the vehicle parked on the road (solid arrow in the right drawing of FIG. 10) by the operation of the operator 50, for example.

[0138] The location information registration unit 218 updates the registered location information in the case where the remote assistance based on the operator 50 is implemented by the remote assistance implementation unit 216 determining that new remote assistance is needed. Specifically, the location information registration unit 218 updates the registered location information based on the position of the AD vehicle 30 at which the remote assistance was first required in the influence range, the position of the AD vehicle 30 at which the remote assistance was implemented, and the content of the implemented remote assistance.

[0139] For example, in the example of FIG. 10, in the stage of the left drawing of FIG. 10, the location information in which the "important factor" is traffic congestion, the "position" is the position of the AD vehicle 30 at which the remote assistance was first required, the "influence range" is the value of the influence range corresponding to the traffic congestion, and the "countermeasure" is "none" is registered. The location information registration unit 218 updates the "important factor" of this location information to a vehicle parked on the road, the "position" to the position of the vehicle parked on the road, the "influence range" to the position of the AD vehicle 30 at which the remote assistance was first required from the position of the location, and the "countermeasure" to an action change (overtaking). Figure 21 Figure 21 Next, the operation of the automatic driving system 200 of the second embodiment will be described. In the second embodiment, in the remote assistance device 210, the following is executed.

[0140] Next, the operation of the automatic driving system 200 of the second embodiment will be described. In the second embodiment, in the remote assistance device 210, the following is executed.​​Figure 22 The remote assistance device side automatic driving process is shown. Further, in the remote assistance device side automatic driving process of the second embodiment, the same process as the remote assistance device side automatic driving process of the first embodiment is added with the same reference numerals and detailed explanation is omitted. Figure 16 ) is added with the same reference numerals and detailed explanation is omitted.

[0141] If the continuation of the automatic driving based on the pre-set action plan is instructed in step S178 because the registered feature does not match the sensor feature, the process moves to step S262. In step S262, the remote assistance implementation section 216 refers to at least one of the vehicle information DB 120 and the sensing data to determine whether it is a state requiring new remote assistance. In the case of requiring new remote assistance, the process moves to step S266, and in the case of not requiring it, the process returns to step S174.

[0142] Further, if the continuation of the automatic driving based on the pre-set action plan is instructed in step S182 because the point information does not register the coping method, the process moves to step S264. In step S264, the remote assistance implementation section 216 determines whether it is a state requiring new remote assistance, as in the above step S262. In the case of requiring new remote assistance, the process moves to step S266, and in the case of not requiring it, the remote assistance device side automatic driving process ends.

[0143] In step S266, the remote assistance implementation section 216 calls the operator 50 to implement remote assistance. Next, in step S268, the point information registration section 218 updates the registered point information based on the position of the AD car 30 that first requested remote assistance within the influence range, the position of the AD car 30 at the time of implementing remote assistance, and the content of the implemented remote assistance. Then, the remote assistance device side automatic driving process ends.

[0144] As explained above, in the automatic driving system of the second embodiment, in the case where the continuation of the automatic driving based on the pre-set action plan is instructed for the remote assistance request, it is determined whether it is a state requiring new remote assistance. Also, in the case of requiring new remote assistance, operator-based remote assistance is implemented, and the point information is updated based on the implemented remote assistance. Thus, it is possible to make corrections in the case where the registration of the first point information results in a misjudgment.

[0145] <Second Embodiment Variation>

[0146] As in the variation of the first embodiment, in the second embodiment, as a variation, it is also possible to cause the vehicle-mounted device side to have the functions of part of the assistance method decision section 114 and the remote assistance implementation section 216 of the remote assistance device 210.

[0147] In this modification, in the AD vehicle side automatic driving processing illustrated in FIG. 12, after step S138, in the on-vehicle device, it is determined whether or not it is a state in which new remote assistance is required, based on at least one of the acquired vehicle information and the sensing data. Also, in the case where new remote assistance is required, it moves to step S144, and in the case where it is not required, it returns to step S140. Similarly, after step S124, in the on-vehicle device, it is determined whether or not it is a state in which new remote assistance is required, and in the case where new remote assistance is required, it moves to step S144, and in the case where it is not required, it returns to step S112. Figure 19

[0148] The AD vehicle side automatic driving processing and the remote assistance device side automatic driving processing performed by the CPU reading in the software (program) in each of the above embodiments can also be performed by various processors other than the CPU. As the processor in this case, a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) in which the circuit configuration can be changed after manufacture, and an ASIC (Application Specific Integrated Circuit) having a circuit configuration designed specifically for performing a certain processing, that is, a dedicated circuit, and the like can be exemplified. In addition, the automatic driving processing can be performed by one of these various processors, or by a combination of two or more processors of the same kind or different kinds (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, and the like). In addition, the hardware structure of these various processors is more specifically a circuit in which circuit elements such as semiconductor elements are combined.

[0149] In addition, although in each of the above embodiments, the manner in which the program is stored (installed) in advance in the storage section is described, it is not limited thereto. The program can also be provided in a form in which it is stored in a non-transitory tangible storage medium such as a CD-ROM, a DVD-ROM, a USB memory, and the like. In addition, the program can also be in a form in which it is downloaded from an external device via a network.

[0150] The present disclosure is described in accordance with the embodiments, but it should be understood that the present disclosure is not limited to the embodiments, the structure. The present disclosure also includes various modifications, modifications within the same scope. In addition to this, various combinations, modes, even other combinations, modes in which only one element, more or less elements are included are also included in the scope, the idea range of the present disclosure.​

Claims

1. An automatic driving method, wherein, The computer executes the following process: For an autonomous vehicle, registering, as location information, a feature amount of sensing data obtained by sensing a situation around a corresponding location, for each location for which remote assistance has been implemented in the past, the location information including a position and an influence range defined by at least a distance from the position; and In a case where a request for remote assistance is generated from an autonomous vehicle within the influence range included in the registered location information, reserving implementation of the remote assistance on the condition that a similarity between a feature amount based on sensing data sequentially acquired from the autonomous vehicle and a feature amount registered as the location information is less than a prescribed value, and causing autonomous driving of the autonomous vehicle to be implemented.

2. The autonomous driving method according to claim 1, wherein As the autonomous driving of the autonomous vehicle, a pre-set autonomous driving control is continued to be implemented.

3. The autonomous driving method according to claim 1, wherein The influence range is determined based on a value determined in advance according to an important factor of remote assistance or statistical information related to past traffic around the corresponding location.

4. The autonomous driving method according to claim 3, wherein The influence range is corrected based on at least one of a period during which remote assistance is implemented and a road characteristic on the corresponding location.

5. The autonomous driving method according to any one of claims 1 to 3, wherein The influence range is defined by a time at which the important factor of remote assistance occurs in addition to the distance.

6. The autonomous driving method according to claim 5, wherein In a case where the time elapses, the location information including the influence range is deleted.

7. The autonomous driving method according to any one of claims 1 to 3, wherein The computer further executes the following process: Sensing data around an autonomous vehicle for which implementation of the remote assistance is reserved is sequentially acquired, and in a case where a similarity between a feature amount based on the acquired sensing data and a feature amount registered for the corresponding location is equal to or greater than a prescribed value, the reserved remote assistance is implemented.

8. The autonomous driving method according to claim 7, wherein A response method to a request for remote assistance is further registered as the location information, Remote assistance of content corresponding to the response method registered in the location information is implemented as the remote assistance.

9. The autonomous driving method according to claim 8, wherein The response method is transmission of control information that instructs an autonomous vehicle to change action or a call to an operator.

10. The autonomous driving method according to claim 7, wherein The feature amount is a feature amount indicating an obstacle that blocks a pre-set travel path based on autonomous driving control with respect to the autonomous vehicle.

11. The autonomous driving method according to any one of claims 1 to 3, wherein The above computer also executes a process of implementing new remote assistance in a state where implementation of the above remote assistance is reserved, in a case where the vehicle stops for more than a prescribed time within the above influence range, or in a case where sensing data of the surroundings of the autonomous vehicle is sequentially acquired, and a degree of dissimilarity between a feature quantity based on the acquired sensing data and a feature quantity based on sensing data acquired last time is more than a prescribed value.

12. The autonomous driving method according to claim 11, wherein In a case where the above new remote assistance is implemented within the above influence range, the registered location information is updated based on a location where remote assistance was first implemented within the above influence range, a location where the above new remote assistance was implemented, and a content of remote assistance implemented by the above new remote assistance.

13. The autonomous driving method according to claim 11, wherein The above new remote assistance is implementation of a call by an operator.

14. The autonomous driving method according to any one of claims 1 to 3, wherein In a case where the autonomous vehicle passes through a location where location information is registered without requiring remote assistance, the above location information is deleted.

15. An automatic driving device, wherein, comprises: a registration unit that, for an autonomous vehicle, registers, as location information, a feature quantity of sensing data based on sensing a situation of the surroundings of a location where remote assistance was implemented in the past, for each location, the above location information including a location and at least an influence range prescribed by a distance from the above location; and a implementation unit that, in a case where a request for remote assistance is generated for the autonomous vehicle within an influence range included in the above registered location information, reserves implementation of the above remote assistance on the condition that a degree of similarity between a feature quantity based on sensing data sequentially acquired from the above autonomous vehicle and a feature quantity registered as the above location information is less than a prescribed value, and implements autonomous driving of the above autonomous vehicle.

16. A computer-readable recording medium storing an automatic driving program, wherein, a computer-executable program that causes a computer to execute a process of: for an autonomous vehicle, registering, as location information, a feature quantity of sensing data based on sensing a situation of the surroundings of a location where remote assistance was implemented in the past, for each location, the above location information including a location and at least an influence range prescribed by a distance from the above location; and in a case where a request for remote assistance is generated for the autonomous vehicle within an influence range included in the above registered location information, reserving implementation of the above remote assistance on the condition that a degree of similarity between a feature quantity based on sensing data sequentially acquired from the above autonomous vehicle and a feature quantity registered as the above location information is less than a prescribed value, and implementing autonomous driving of the above autonomous vehicle.

17. An autonomous driving system that includes an on-vehicle device mounted on an autonomous vehicle and that controls autonomous driving of the above autonomous vehicle, and a remote assistance device that implements remote assistance of autonomous driving of the above autonomous vehicle, wherein The above-described remote assistance device includes, for an autonomous vehicle, a registration unit that registers, as location information, a feature amount of sensing data based on sensing a situation around a corresponding location for each location for which remote assistance has been implemented in the past, the location information including a position and an influence range defined by at least a distance from the position, The above-described remote assistance device or the above-described on-vehicle device includes an implementation unit that, in a case where an autonomous vehicle within an influence range included in the registered location information generates a request for remote assistance, retains implementation of the remote assistance on a condition that a similarity between a feature amount based on sensing data sequentially acquired from the autonomous vehicle and a feature amount registered as the location information is less than a predetermined value, and causes autonomous driving of the autonomous vehicle to be implemented.

Images