Method and device for controlling a mobile body

Abstract

The present disclosure relates to a control method and a control device for a mobile body. A method of controlling a mobile body includes the steps of selecting a control mode for automatic driving of the mobile body, and transmitting information of the selected control mode to the mobile body. The control mode includes a first mode, a second mode in which a restriction related to travel safety is stricter than in the first mode, and a third mode in which the restriction is stricter than in the first mode and is looser than in the second mode. The step of selecting the control mode includes the step of selecting a control mode corresponding to a combination of a determination result of whether remote support for the mobile body is needed and a determination result of a state of a remote support side that copes with remote support for the mobile body, from among options including the first mode, the second mode, and the third mode.

Description

Technical Field

[0001] This disclosure relates to methods and apparatus for controlling mobile bodies that have autonomous driving capabilities and remote support capabilities. Background Technology

[0002] Japanese Patent Application Publication No. 2021-022319 discloses a vehicle control device that selectively executes two driving modes of a vehicle's driving mechanism. The two driving modes include an automatic driving mode and a remote driving mode. In conventional control devices, in the automatic driving mode selection, the control device controls the driving mechanism based on autonomously generated operating information. On the other hand, in the remote driving mode selection, conventional control devices control the driving mechanism based on remote driving information from the operator.

[0003] Conventional control devices also predict the range for selecting the autonomous driving mode (autonomous driving range) and the range for selecting the remote driving mode (remote driving range) based on path information related to the vehicle's autonomous driving. Furthermore, when the vehicle approaches the remote driving range, the conventional control device requests remote driving information from the operator. If remote driving information is provided in response to this request, the conventional control device switches the vehicle's driving mode from autonomous driving mode to remote driving mode.

[0004] In addition to Japanese Patent Application Publication No. 2021-022319, other documents that indicate the level of technology in the technical field related to this disclosure may include International Publication No. 2021 / 177052, Japanese Patent Application Publication No. 2019-191893, and Japanese Patent Application Publication No. 2017-174282.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2021-022319

[0008] Patent Document 2: International Publication No. 2021 / 177052

[0009] Patent Document 3: Japanese Patent Application Publication No. 2019-191893

[0010] Patent Document 4: Japanese Patent Application Publication No. 2017-174282

[0011] In the aforementioned conventional control devices, the automatic driving mode is selected when the vehicle's current location is within the automatic driving zone and far from the remote driving zone. However, it is assumed that even in such a situation, remote support provided by the operator, such as remote driving, is required. Furthermore, in this case, the technical problem becomes how to continue the vehicle's automatic driving during the period from when remote support is determined to be needed until the vehicle receives the information related to remote support provided in response.

[0012] In particular, it is considered that in the event of no immediate response to a request for remote support, the vehicle may temporarily stop in front of the location where the key factor for remote support is located, thus enabling autonomous driving. That is, in this case, the vehicle waits in front of the location of the key factor for remote support to receive information related to remote support. However, the resulting waiting time may cause anxiety for the vehicle's occupants, or it may affect traffic flow around the vehicle. Therefore, it can be said that there is room for technological development to prevent such problems from occurring. Summary of the Invention

[0013] One object of this disclosure is to provide a technique for appropriately responding to sudden requests for remote support that arise during the autonomous driving process of a mobile body with both autonomous driving and remote support capabilities.

[0014] The first point of this disclosure is a method for controlling a mobile body with autonomous driving capabilities and remote support capabilities, which has the following characteristics.

[0015] The method includes the following steps: selecting a control mode for autonomous driving of the mobile body; and sending information of the selected control mode to the mobile body.

[0016] The control modes include a first mode and a second mode, which has stricter constraints related to driving safety compared to the first mode.

[0017] The step of selecting the control mode includes the following steps: determining whether remote support for the mobile body is required; and if it is determined that remote support is not required, selecting the first mode as the control mode, and if it is determined that remote support is required, selecting the second mode as the control mode.

[0018] The second aspect of this disclosure is a method for controlling a mobile body with autonomous driving capabilities and remote support capabilities, which has the following characteristics.

[0019] The method includes the following steps: selecting a control mode for autonomous driving of the mobile body; and sending information of the selected control mode to the mobile body.

[0020] The control modes include a first mode and a second mode, which has stricter constraints related to driving safety compared to the first mode.

[0021] The step of selecting the control mode includes the following steps: determining the state of the remote support side in response to remote support for the mobile body; and when the state of the remote support side is determined to be a ready state in which preparation for responding to remote support is completed, selecting the first mode as the control mode; and when the state of the remote support side is determined to be a non-ready state in which preparation for responding to remote support is not completed, selecting the second mode as the control mode.

[0022] The third aspect of this disclosure is a method for controlling a mobile body with autonomous driving capabilities and remote support capabilities, which has the following characteristics.

[0023] The method includes the following steps: selecting a control mode for autonomous driving of the mobile body; and sending information of the selected control mode to the mobile body.

[0024] The control modes include a first mode, a second mode with stricter constraints related to driving safety compared to the first mode, and a third mode with stricter constraints compared to the first mode and more lenient constraints compared to the second mode.

[0025] The step of selecting the control mode includes the following steps: determining whether remote support for the mobile body is required; determining the status of the remote support side that responds to remote support for the mobile body; and selecting from options including the first mode, the second mode, and the third mode a control mode corresponding to the combination of the determination result of whether remote support is required and the determination result of the status of the remote support side.

[0026] The fourth aspect of this disclosure is a device for controlling a mobile body with both autonomous driving and remote support capabilities, which has the following characteristics.

[0027] The device includes a processor.

[0028] The processor is configured to perform the following processes: select a control mode for autonomous driving of the mobile body; and send information about the selected control mode to the mobile body.

[0029] The control modes include a first mode, a second mode with stricter constraints related to driving safety compared to the first mode, and a third mode with stricter constraints compared to the first mode and more lenient constraints compared to the second mode.

[0030] In the process of selecting the control mode, the processor is configured to perform the following processes: determine whether remote support for the mobile body is required; determine the state of the remote support side that responds to the remote support for the mobile body; and select from options including the first mode, the second mode, and the third mode a control mode corresponding to the combination of the determination result of whether remote support is required and the determination result of the state of the remote support side.

[0031] Invention Effects

[0032] According to this disclosure, the control mode for autonomous driving of the mobile body is selected based on at least one of a determination of whether remote support for the mobile body is needed and a determination of the state of the remote support side responding to such remote support. Furthermore, information about the selected control mode is sent to the mobile body.

[0033] In the first viewpoint, the control modes for autonomous driving of the mobile vehicle include a first mode and a second mode, which has stricter constraints related to driving safety compared to the first mode. Furthermore, in the first viewpoint, the first or second mode is selected based on a determination of whether remote support for the mobile vehicle is required. Therefore, according to the first viewpoint, autonomous driving of the mobile vehicle can be appropriately performed based on this determination.

[0034] Similar to the first viewpoint, in the second viewpoint, the control modes for autonomous driving of the mobile body include a first mode and a second mode. Furthermore, in the second viewpoint, the first mode or the second mode is selected based on the determination result of the state of the remote support side responding to remote support for the mobile body. Therefore, according to the second viewpoint, autonomous driving of the mobile body can be appropriately performed based on this determination result.

[0035] In the third and fourth viewpoints, the control modes include a first mode, a second mode, and a third mode. The third mode is a control mode with stricter constraints related to driving safety compared to the first mode, and less stringent constraints related to driving safety compared to the second mode. Furthermore, in the third viewpoint, a control mode is selected from the options including the first, second, and third modes, corresponding to a combination of a determination result regarding whether remote support for the mobile body is needed and a determination result regarding the state of the remote support side responding to that remote support. Therefore, according to the third and fourth viewpoints, autonomous driving of the mobile body can be appropriately performed based on the combination of the determination result and the determination result. Attached Figure Description

[0036] Figure 1 This diagram illustrates remote support for mobile entities.

[0037] Figure 2 This diagram illustrates the autonomous driving function of a moving vehicle.

[0038] Figure 3 This diagram illustrates an example of a vehicle control mode set in the implementation method.

[0039] Figure 4 This diagram illustrates an example of vehicle control mode selection based on the determination of the necessity of remote support for the moving body.

[0040] Figure 5 This diagram illustrates an example of vehicle control mode selection based on the determination of the state of the remote support side in response to remote support for mobile bodies.

[0041] Figure 6 This diagram illustrates an example of vehicle control mode selection based on the determination of the state of the remote support side in response to remote support for mobile bodies.

[0042] Figure 7 This diagram illustrates an example of vehicle control mode selection based on a combination of the determination of whether remote support for a moving body is necessary and the determination of the state of the remote support side in response to such remote support.

[0043] Figure 8 This is a block diagram illustrating an example of the structure of a management server.

[0044] Figure 9 This is a block diagram representing an example of the structure of a moving object.

[0045] Figure 10 This is a block diagram representing an example of the configuration of an operator's terminal.

[0046] Figure 11 It is a flowchart representing the process executed in the management server (processor) and is particularly associated with the implementation method.

[0047] Explanation of reference numerals in the attached figures:

[0048] 1: Management server; 2: Vehicle; 3: Operator terminal; 11, 21, 31: Data processing device; 13, 24, 34: Communication device; 14, 26, 35: Processor; 15, 27, 36: Memory; MD: Vehicle control mode; CND: Status of remote support side; MDE: Mode information; NDS: Necessity of remote support. Detailed Implementation

[0049] Hereinafter, the control method and control device of the moving body according to embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be noted that the control method of the embodiments is implemented through computer processing performed in the control device of the embodiments. Furthermore, in the various figures, the same or equivalent parts are labeled with the same reference numerals, and their descriptions are simplified or omitted.

[0050] 1. Remote support

[0051] Figure 1 This diagram illustrates remote support for mobile entities. Figure 1 The document describes the management server 1, the vehicle 2, and the operator's (OP) terminal (hereinafter also referred to as "operator terminal") 3.

[0052] Management server 1 is an example of a control device in this implementation. Management server 1 manages remote support services. Typically, management server 1 is managed by the provider of the remote support services. Management server 1 communicates with vehicle 2 and operator terminal 3.

[0053] Vehicle 2 is an example of a mobile body controlled by the control device of the embodiment. Vehicle 2 has the function of receiving remote support (remote support function). Vehicle 2 can be a vehicle owned by a provider of remote support services, or a vehicle owned by a legal person or individual who has entered into a contract with such provider to provide remote support services. In addition to the remote support function, vehicle 2 also has the function of autonomous driving (autonomous driving function). This autonomous driving function will be described later.

[0054] Operators (OPs) provide remote support for vehicle 2. Operators (OPs) can be employees of the provider offering remote support services, or employees or individuals of a legal entity that has entered into a business contract with that provider. The total number of operators (OPs) providing remote support services must be at least two. The total number of operators (OPs) responding to remote support requests for vehicle 2 must be at least one, and this responder (responsible person) is selected on management server 1.

[0055] Here, "at least one responder" means that there is at least one designated operator (OP) (i.e., a candidate responder) to respond to remote support for vehicle 2. During the execution of remote support for vehicle 2, at least one operator (OP) (i.e., the actual business responder) is selected from the candidates of at least one responder. The selection of this at least one actual business responder is also performed on management server 1.

[0056] As examples of remote support, identification support and judgment support are illustrated. For instance, consider the scenario where vehicle 2 is operating autonomously. When sunlight shines on a traffic light located in front of vehicle 2, the accuracy of identifying the traffic light's status decreases. Without identifying the light status, it becomes difficult to determine at what time and what action should be taken. In such a situation, support is provided for identifying the light status by the operator (OP) and / or for judging the actions of vehicle 2 based on the results of the operator's identification of the light status.

[0057] Remote support also includes remote driving. In remote driving, for example, the operator (OP) refers to camera images projected around vehicle 2, camera images projected inside vehicle 2, etc., to perform at least one of the following driving operations: steering, acceleration, and deceleration.

[0058] exist Figure 1 The text describes the exchange of data COM12 and COM21 between management server 1 and vehicle 2 (more precisely, the terminal mounted on vehicle 2), and data COM13 and COM31 between management server 1 and operator terminal 3. Data COM12 is sent from management server 1 to vehicle 2. Data COM21 is sent from vehicle 2 to management server 1. Data COM13 is sent from management server 1 to operator terminal 3. Data COM31 is sent from operator terminal 3 to management server 1.

[0059] Examples of data COM21 include data identifying vehicle 2, data showing camera images of the area surrounding vehicle 2, and data showing camera images inside the vehicle 2's passenger compartment. Examples of data COM21 also include various data when vehicle 2 requests remote support (e.g., data identifying vehicle 2, camera image data, data indicating the content of remote support, etc.). Examples of data COM13 include data corresponding to the selection result of the operator (OP) responding to remote support for vehicle 2. Examples of data COM31 and COM12 include data on identification support and judgment support performed by the operator (OP). In the case of remote driving of vehicle 2, data COM31 and COM12 include driving operation data.

[0060] 2. Autonomous driving function

[0061] Figure 2 This diagram illustrates the autonomous driving function of vehicle 2. Figure 2The image depicts a data processing device 21 for vehicle 2. The data processing device 21 is a computer that controls the behavior of vehicle 2. Typically, the data processing device 21 is mounted on vehicle 2. At least a portion of the functions of the data processing device 21 can also be implemented as the functions of an external device located outside vehicle 2. That is, at least a portion of the functions of the data processing device 21 can also be implemented separately by vehicle 2 and external devices.

[0062] The data processing unit 21 performs vehicle control, for example, for the autonomous driving of vehicle 2. In this vehicle control, for example, the steering, acceleration, and deceleration of vehicle 2 are controlled so that vehicle 2 follows a target trajectory TR. The target trajectory TR is the path that vehicle 2 should travel. The target trajectory TR is generated, for example, based on the travel route from the starting point of vehicle 2 to its destination. The travel route is calculated, for example, by a navigation system (not shown). The target trajectory TR can be generated either based on surrounding information of vehicle 2 or based on a combination of the travel route and surrounding information.

[0063] The target trajectory TR includes, for example, a set of target control values ​​TCi used in vehicle control for autonomous driving. As an example of target control values ​​TCi, vehicle 2 should traverse the target position [Xi, Yi]. Figure 2 In the example shown, the X direction is the forward direction of vehicle 2, and the Y direction is the top-down direction orthogonal to the X direction. However, the coordinate system (X, Y) is not limited to... Figure 2 The example shown illustrates this. Target positions [Xi, Yi] are set at specified intervals (e.g., 1m to 2m) on the target orbit TR. For example, the target positions [Xi, Yi] are set to a point several seconds to tens of seconds after the current time.

[0064] The target control value TCi can also be a combination of the target position [Xi, Yi] and the target velocity [VXi, VYi] of vehicle 2 at the target position [Xi, Yi]. Alternatively, the target time at the target position [Xi, Yi] can be used instead of the target velocity [VXi, VYi]. Furthermore, when using the target time, the target control value TCi can also include the orientation of vehicle 2 at the target time. Additionally, besides the target position [Xi, Yi] and the target velocity [VXi, VYi], the target control value TCi can also include at least one of the target yaw angle of vehicle 2 at the target position [Xi, Yi] and the target acceleration of vehicle 2 at the target position [Xi, Yi].

[0065] To enable vehicle 2 to follow the target trajectory TR, in the vehicle control for autonomous driving, the deviation (e.g., position deviation, speed deviation, timing deviation, yaw angle deviation, etc.) between the target vehicle state corresponding to the target control value TCi and the current vehicle state of vehicle 2 is calculated. In this vehicle control, control command values ​​for the driving mechanism (actuators) of vehicle 2 are also calculated in a manner that reduces this deviation. That is, the control command values ​​are commands used to control the current vehicle state to the target vehicle state. The driving mechanism includes a steering mechanism, a drive mechanism, and a braking mechanism. Then, the driving mechanism is controlled based on the control command values.

[0066] 3. Vehicle Control Mode (MD)

[0067] In this implementation, a vehicle control mode (MD) for autonomous driving is set. The vehicle control mode (MD) is set in management server 1. Information about the vehicle control mode (MD) is sent from management server 1 to vehicle 2 each time the vehicle control mode (MD) is set. Vehicle 2 refers to this information in the vehicle control mode (MD) for autonomous driving vehicle control. Hereinafter, the information about the vehicle control mode (MD) sent from management server 1 to vehicle 2 will also be referred to as "mode information (MDE)". The mode information (MDE) includes... Figure 1 The data shown is in COM12.

[0068] Figure 3 This diagram illustrates an example of the vehicle control mode MD set in the implementation method. Figure 3 In the example shown, five vehicle control modes MD, including Aggressive mode (AGG mode), Moderate mode (MDR mode), Semi-conservation mode (SCN mode), Conservation mode (CNS mode), and Safe Stop mode, are set as vehicle control mode MD.

[0069] The four vehicle control modes (MD) other than the safety stop mode differ in the stringency of their safety-related constraints. These constraints are most stringent in the safety-priority mode, decreasing in the order of quasi-safety-priority mode, standard mode, and efficiency-priority mode. It should be noted that the safety stop mode is a vehicle control mode (MD) that is not subject to safety-related constraints. When the safety stop mode is set, vehicle control is performed to bring vehicle 2 to a stop on the shoulder or side of the road.

[0070] Regarding the “strictness of constraints related to driving safety”, for example, multiple candidates of driving routes calculated by the navigation system can be evaluated by looking at at least one of the items exemplified below.

[0071] For example, consider a scenario where multiple candidate routes pass through both general roads and highways. In this case, it is expected that the higher the proportion of the distance traveled via general roads, the more information the vehicle needs to recognize for driving safety. Furthermore, it is expected that the higher the proportion of the distance traveled via highways, the fewer the total number of lane changes and the total number of changes in direction (right and left turns) required to traverse the route. Therefore, candidate routes with a high proportion of the distance traveled via general roads can be evaluated as candidates with "strict constraints related to driving safety," while candidate routes with a low proportion of the distance traveled via general roads can be evaluated as candidates with "relaxed constraints related to driving safety."

[0072] In another example, consider multiple candidate routes that traverse both urban and suburban areas. In this case, it's predictable that the higher the proportion of distance traversing urban areas, the more information the vehicle needs to identify for driving safety. Therefore, candidate routes with a high proportion of distance traversing urban areas can be evaluated as having "stricter constraints related to driving safety." It should be noted that the distinction between urban and suburban areas can be based on map information (population density information).

[0073] In another example, consider a scenario where multiple candidate routes include information on traffic volume (e.g., vehicle and pedestrian traffic per unit time) in the areas the route traverses. In this case, it is expected that higher traffic volume necessitates a greater amount of information that the vehicle must identify for safe driving. Therefore, a candidate route with high traffic volume can be evaluated as one with "strict constraints related to driving safety." It should be noted that the area traversed by the route can be defined, for example, by measuring the area along the route on a specified area scale (e.g., 50m). 2 ~100m 2 The traffic volume of a route can be represented by connecting unit areas when dividing the map (using a scale). Furthermore, the traffic volume of the area traversed by the route can be calculated by adding the traffic volumes of unit areas connected along the route.

[0074] Based on the aforementioned evaluation criteria, such as the proportion of distance traveled from general roads, the proportion of distance traveled from urban areas, and the traffic volume of the areas traversed by the route, multiple candidate routes can be graded from the perspective of "the strictness of constraints related to driving safety." In the implementation method, based on the level of this grade, a process is performed to associate these candidates with the four vehicle control modes (MDs) mentioned above (that is, the four vehicle control modes MD excluding the safe stop mode).

[0075] Other examples of evaluation of the "strictness of constraints related to driving safety" include various limits, thresholds, and variables used in vehicle control for autonomous driving.

[0076] Examples of various limit values ​​used in vehicle control include upper or lower limits for target control values ​​such as speed, acceleration (or deceleration), abruptness, and steering angular velocity. Differentiations can be set in the "strictness of constraints related to driving safety" by setting the upper or lower limits of target control values ​​at multiple levels.

[0077] Examples of various thresholds used in vehicle control include inter-vehicle distance (or inter-vehicle time) when following a preceding vehicle, lateral margin distance when avoiding a collision with an obstacle, speed when passing beside an obstacle, and allowable deviation from the currently traveling lane when passing beside an obstacle. The distance used to determine whether lane-changing control can be performed (e.g., the distance between two vehicles when making a lane change to merge into a lane alongside another vehicle) is also an example of a threshold. Differences can be set in the "strictness of constraints related to driving safety" by setting such thresholds at multiple levels.

[0078] Examples of variables used in vehicle control include those used for calculating the risk of collision with obstacles, the risk of a moving object running out of the vehicle's blind spot, etc. Examples of such variables include those input to a model that predicts the future position of an obstacle or moving object (assumed acceleration, tolerance, etc.). Variables used in vehicle control also include the reliability of the identification results of objects recognized by the vehicle and their states (e.g., traffic lights and their light colors). Differences can be set in the "strictness of constraints related to driving safety" by setting such variables at multiple levels.

[0079] In the implementation, the various restrictions, thresholds, and variables mentioned above are pre-classified into at least four levels based on the "strictness of constraints related to driving safety." Furthermore, these classified restrictions, thresholds, and variables are associated with the four vehicle control modes (MD) mentioned above.

[0080] 4. Example of selecting Vehicle Control Mode (MD)

[0081] In the implementation, the selection of the above five vehicle control modes MD (that is, all vehicle control modes MD including the safety stop mode) is based on at least one of the necessity NDS for remote support of vehicle 2 and the state CND of the remote support side responding to the remote support.

[0082] 4-1. The Necessity of Remote Support for NDS

[0083] The presence or absence of a necessary NDS is determined in management server 1. The information used to determine the presence or absence of a necessary NDS is provided by vehicle 2, management server 1, or any predictor.

[0084] For example, vehicle 2 identifies objects included in camera images reflecting its surroundings. Furthermore, vehicle 2 detects the movement of the identified objects. For instance, if identifying an object in the surrounding images is difficult, vehicle 2 requests the management server 1 to identify that object. It should be noted that when the management server 1 receives this request, it determines that a necessary NDS (Network Detection System) is required. In another example, if object identification is performed but unnatural movement of the object is detected, vehicle 2 requests the management server 1 to confirm the object's status. When the management server 1 receives this request, it also determines that a necessary NDS is required.

[0085] Vehicle 2 can also identify the status of the occupant (driver) as shown in the camera images inside the vehicle's passenger compartment. If an abnormal driver status is detected, Vehicle 2 can request confirmation of that abnormal status from the management server 1. Upon receiving this request, the management server 1 determines that a necessary NDS (Network Detection System) is required. Vehicle 2 can also detect the movement of objects around it based on information from recognition sensors other than cameras. If unnatural movement of an object is detected, Vehicle 2 can request confirmation of that object's status from the management server 1. Upon receiving this request, the management server 1 also determines that a necessary NDS is required.

[0086] Management server 1 can appropriately acquire camera images of vehicle 2 (e.g., surrounding images, interior images), and information other than camera images (e.g., information from recognition sensors other than the camera, information from internal sensors of vehicle 2). Therefore, when equipped with object recognition and driver state recognition functions of vehicle 2, management server 1 can identify objects included in the camera images of vehicle 2, detect the movement of the identified objects, and identify the driver's state of vehicle 2. Then, if unnatural movement of an object is detected, an abnormal state of the driver is identified, or an abnormal state of vehicle 2 is detected, management server 1 determines that a necessary NDS is required.

[0087] Any predictor may be formed, for example, within the management server 1. Any predictor may also be formed outside the management server 1. For example, the predictor may use machine learning to predict the occurrence of remote support for vehicle 2 using time-series data of camera images from vehicle 2, time-series data of camera images from other vehicles, time-series data of images from infrastructure cameras located at intersections, etc., or remote support service provision history data (e.g., time period and location). Then, if the occurrence of remote support for vehicle 2 is predicted, the predictor generates prediction information and sends it to the management server 1. Upon receiving this prediction information, the management server 1 determines that a necessary NDS (Non-Distributed Support Module) is required.

[0088] Management server 1 determines the presence or absence of a necessary NDS based on information provided from vehicle 2, management server 1, or any predictor. Then, based on the result of this determination, management server 1 selects one of the five vehicle control modes (MD) mentioned above.

[0089] Figure 4 This diagram illustrates an example of selecting the vehicle control mode (MD) based on the presence or absence determination result of the necessity NDS. Figure 4 In the first example shown, the AGG mode is selected when the NDS is determined to be unnecessary ("No"), and the MDR mode is selected when the NDS is determined to be necessary ("Yes"). In this example, the AGG mode is equivalent to the "first mode" of this disclosure, and the MDR mode is equivalent to the "second mode" of this disclosure.

[0090] Here, the difference between the first and second modes lies in the "strictness of safety-related constraints." That is, the second mode has stricter constraints related to driving safety compared to the first mode. Furthermore, the relationship between AGG mode and MDR mode is the same as that between the first and second modes.

[0091] exist Figure 4In the second example shown, the MDR mode is selected when the NDS is determined to be unnecessary ("No"), and the SCN mode is selected when the NDS is determined to be necessary ("Yes"). In this example, the MDR mode is equivalent to the "first mode" of this disclosure, and the SCN mode is equivalent to the "second mode" of this disclosure. Furthermore, the relationship between the MDR mode and the SCN mode is the same as that between the first mode and the second mode.

[0092] 4-2. Status of Remote Support Side (CND)

[0093] The remote support side refers to the entire system, including management server 1, operator (OP), and operator terminal 3, that provides remote support for vehicle 2. The state of the remote support side (CND) differs significantly between a ready state (RD state) where preparation for responding to remote support is complete, and a non-ready state (URD state) where preparation for responding to remote support is incomplete. The non-ready state can be further subdivided into an available state (AV state) where preparation for switching to a ready state is complete, and an unavailable state (UAV state) where preparation for this switch is incomplete.

[0094] The determination of which of the three states CND is based, for example, on the selection status of a predetermined operator (OP) (i.e., a candidate responder) to respond to remote support for vehicle 2. Specifically, if a candidate responder is selected and an approval signal from that candidate is detected, the state CND is determined to be equivalent to a ready state. If a candidate responder is selected but no approval signal from that candidate is detected, the state CND is determined to be equivalent to a non-ready state (available state). If a candidate responder is not selected and there are insufficient candidates, the state CND is determined to be equivalent to a non-ready state (unavailable state).

[0095] In another example, the state CND is determined based on the status of the operator (OP) selected from the candidates of responders during remote support execution. Specifically, if the selection of the actual business responder is completed and an approval signal from that actual business responder is detected, the state CND is determined to be equivalent to a ready state. If the actual business responder notifies the management server 1 whether the mastery of the status of vehicle 2 is complete, the state CND is determined to be equivalent to a non-ready state (available state) until a signal indicating that the mastery is complete is detected from a notification device such as a switch or button. If the actual business responder notifies the management server 1 whether the mastery of the status of vehicle 2 is possible, the state CND is determined to be equivalent to a non-ready state (unavailable state) when a signal indicating that the mastery is impossible is detected from a notification device such as a switch or button.

[0096] In another example, the state CND is determined based on the communication status between management server 1 and vehicle 2, and the communication status between management server 1 and operator OP's terminal. Specifically, if a communication delay occurs between management server 1 and vehicle 2, or between management server 1 and operator terminal 3, the state CND is determined to be equivalent to a non-ready state (available or unavailable state) based on the level of the communication delay.

[0097] In another example, the state CND is determined based on the state of the system providing remote support for vehicle 2. Specifically, if the system is in a normal state, the state CND is determined to be equivalent to a ready state. If the system is in an abnormal state, the state CND is determined to be equivalent to a non-ready state (available or unavailable state) depending on the level of the abnormality.

[0098] Based on the selected status of the responder candidates, the status of the actual business responder, the communication status between management server 1 and vehicle 2 or between management server 1 and operator terminal 3, or the status of the system providing remote support for vehicle 2, management server 1 determines which of the three states the status CND belongs to. Then, based on the determination result, management server 1 selects one of the five vehicle control modes MD mentioned above.

[0099] Figure 5 and Figure 6 This diagram illustrates an example of selecting the vehicle control mode (MD) based on the determination result of the state (CND). Figure 5 In the first example shown, the MDR mode is selected when the state CND is determined to be in the RD state, and the SCN mode is selected when the state CND is determined to be in the URD state. In this example, the MDR mode is equivalent to the "first mode" of this disclosure, and the SCN mode is equivalent to the "second mode" of this disclosure. The relationship between the MDR mode and the SCN mode is the same as that between the first mode and the second mode, as already described.

[0100] exist Figure 5 In the second example shown, the SCN mode is selected when the state CND is determined to be in the RD state, and the CNS mode is selected when the state CND is determined to be in the URD state. In this example, the SCN mode is equivalent to the "first mode" of this disclosure, and the CNS mode is equivalent to the "second mode" of this disclosure. Furthermore, the relationship between the SCN mode and the CNS mode is the same as that between the first mode and the second mode.

[0101] exist Figure 6In the first example shown, the MDR mode is selected when the state CND is determined to be in the RD state, the SCN mode is selected when the state CND is determined to be in the URD state (AV state), and the CNS mode is selected when the state CND is determined to be in the URD state (UAV state). In this example, the MDR mode is equivalent to the "first mode" of this disclosure, the SCN mode is equivalent to the "third mode" of this disclosure, and the CNS mode is equivalent to the "second mode" of this disclosure.

[0102] Here, the difference between the first, second, and third modes lies in the "strictness of safety-related constraints." That is, for the second mode, the constraints related to driving safety are stricter than those for the first mode. Furthermore, for the third mode, the constraints related to driving safety are stricter than those for the first mode, but less stringent than those for the second mode. Moreover, the relationship between the MDR, CNS, and SCN modes is the same as that between the first, second, and third modes.

[0103] exist Figure 6 In the second example shown, the SCN mode is selected when the state CND is determined to be in the RD state; the CNS mode is selected when the state CND is determined to be in the URD state (AV state); and the safe stop mode is selected when the state CND is determined to be in the URD state (UAV state). In this example, the SCN mode is equivalent to the "first mode" of this disclosure, the CNS mode is equivalent to the "second mode" of this disclosure, and the safe stop mode is equivalent to the "fourth mode" of this disclosure. The relationship between the SCN mode and the CNS mode is the same as that between the first mode and the second mode, as already described.

[0104] 4-3. The combination of necessity NDS and status CND

[0105] exist Figure 4 In the example shown, NDS selected Vehicle Control Mode (MD) based on the necessity of remote support. Figure 5 and Figure 6 In the example shown, the vehicle control mode MD was selected based on the state of the remote support side (CND). Figure 7 In the example shown, the vehicle control mode MD is selected based on the combination of the remote support necessity NDS and the remote support side status CND.

[0106] exist Figure 7In the first example shown, when the NDS is determined to be unnecessary ("No"), the MDR mode is selected when the state CN is determined to be in the RD state, the SCN mode is selected when the state CN is determined to be in the URD state (AV state), and the CNS mode is selected when the state CN is determined to be in the URD state (UAV state). That is to say, this first example applies the following when the NDS is determined to be unnecessary ("No"). Figure 6 The first example.

[0107] exist Figure 7 In the second example shown, when it is determined that the NDS is necessary ("Yes"), the SCN mode is selected when the state CN is determined to be in the RD state, the CNS mode is selected when the state CN is determined to be in the URD state (AV state), and the safety stop mode is selected when the state CN is determined to be in the URD state (UAV state). That is to say, this second example applies the following when the NDS is determined to be necessary ("Yes"). Figure 6 The second example.

[0108] Thus, according to the implementation method, a vehicle control mode MD is set based on at least one of the necessity NDS for remote support of vehicle 2 and the state CND of the remote support side responding to the remote support, and vehicle control for autonomous driving of vehicle 2 is performed according to the vehicle control mode MD. Therefore, for example, when the necessity NDS is switched from no ("no") to yes ("yes"), autonomous driving can be appropriately performed until the start of remote support. When the necessity NDS is switched from yes ("yes") to no ("no"), vehicle control can be appropriately performed when autonomous driving restarts.

[0109] Furthermore, according to the implementation method, autonomous driving can be appropriately performed during the period up to the start of remote support, depending on the state of the remote support side (CND) when the Necessary NDS is switched from "no" to "yes". Furthermore, vehicle control can be appropriately performed when autonomous driving restarts, depending on the state of the remote support side (CND) when the Necessary NDS is switched from "yes" to "no". This is expected to improve the stability of autonomous driving execution when switching of the Necessary NDS occurs frequently. Therefore, according to the implementation method, it is expected to provide reassurance to the occupants of vehicle 2, or minimize the impact on traffic flow around vehicle 2.

[0110] 5. System Configuration Example

[0111] Reference Figures 8-10 An example of the structure of a remote support system is described. Figure 8 This is a block diagram representing an example of the configuration of management server 1. Figure 9 This is a block diagram representing a configuration example of vehicle 2. Figure 10 This is a block diagram illustrating an example of the configuration of operator terminal 3.

[0112] 5-1. Example of the configuration of management server 1

[0113] exist Figure 8 In the example shown, management server 1 includes a data processing unit 11, a database 12, and a communication unit 13. It should be noted that the database 12 and the communication unit 13 are connected to the data processing unit 11 via a defined network.

[0114] The data processing device 11 includes at least one processor 14 and at least one memory 15. The processor 14 includes a CPU (Central Processing Unit). The memory 15 is a volatile memory such as DDR memory, used for executing various programs used by the processor 14 and temporarily storing various data. Among the various data used by the processor 14 is data stored in the database 12.

[0115] As an example of data stored in database 12, a data DNDS used to determine the presence or absence of a necessary NDS is shown. The data DNDS includes, for example, data of camera images from vehicle 2 (e.g., surrounding images, interior images), time-series data of those camera images, data other than those of the camera images, and time-series data other than those of the camera images. The data DNDS also includes prediction information received from any of the aforementioned predictors. If the management server 1 functions as such a predictor, the data DNDS also includes time-series data of camera images from other vehicles, time-series data of images from infrastructure cameras, and provision history data of remote support services.

[0116] As another example of data stored in database 12, data DCND for determining the status CND for remote support can also be cited. Data DCND includes, for example, data on the selected status of a predetermined operator OP (i.e., a candidate responder) responding to remote support for vehicle 2, and data related to the presence or absence of an approval signal received from that candidate. In another example, data DCND also includes data on the status of the operator OP (i.e., the actual business responder) selected from the candidates responders during the execution of remote support, and data related to the presence or absence of an approval signal received from that actual business responder. In the case where the actual business responder notifies the management server 1 whether the status of vehicle 2 has been assessed, data DCND also includes data related to the presence or absence of a signal indicating that assessment has been completed, and data related to the presence or absence of a signal indicating that assessment is impossible.

[0117] In yet another example, the data DCND includes data on the communication status between management server 1 and vehicle 2, and data on the communication status between management server 1 and the operator OP's terminal. In yet another example, data on the status of the system providing remote support for vehicle 2 is included in the data DCND.

[0118] Communication device 13 connects to external devices via a communication network. The communication network is not particularly limited and can use wired or wireless networks. Examples of communication networks include the Internet, WWW (World Wide Web), telephone lines, LAN (Local Area Network), SAN (Storage Area Network), and DTN (Delay Tolerant Network). Examples of wireless communication include WiFi (Wireless Fidelity) and Bluetooth (registered trademark). Wireless communication can be direct communication with the management server 1 (Ad Hoc communication) or indirect communication via an access point. The communication destinations of communication device 13 include vehicle 2 and operator terminal 3.

[0119] 5-2. Example of vehicle 2's composition

[0120] exist Figure 9 In the example shown, vehicle 2 includes a data processing unit 21, a camera 22, a sensor group 23, a communication unit 24, and a driving unit 25. It should be noted that the camera 22, driving unit 25, and other elements are connected to the data processing unit 21 via a defined network.

[0121] The data processing device 21 is a computer used to process various data acquired by the vehicle 2. The data processing device 21 includes at least one processor 26 and at least one memory 27. The processor 26 includes a CPU. The memory 27 is used to run various programs employed by the processor 26 and to temporarily store various data. The various data acquired by the data processing device 21 are stored in the memory 27.

[0122] The processor 26 performs the following processing by executing a program stored in the memory 27: classifying multiple candidates for driving routes calculated sequentially by the navigation system from the perspective of "the strictness of constraints related to driving safety". Furthermore, when the processor 26 receives mode information MDE from the management server 1, it performs the following processing by executing a program stored in the memory 27: selecting a candidate from the classified driving route candidates that corresponds to the mode information MDE.

[0123] Furthermore, the processor 26 executes a program stored in the memory 27 to set various limits, thresholds, and variables used in vehicle control for autonomous driving of the vehicle 2. Upon receiving mode information MDE from the management server 1, the processor 26 sets various limits, thresholds, and variables corresponding to that mode information MDE. It should be noted that these limits, thresholds, and variables are pre-classified into at least four levels based on the "strictness of constraints related to driving safety," as already described.

[0124] Furthermore, the processor 26 performs remote support control of the vehicle 2 by executing a program stored in the memory 27. For example, the vehicle 2 determines whether remote support is needed by executing the program stored in the memory 27. If remote support is determined to be needed, the processor 26 generates a remote support request message and sends it to the management server 1 via the communication device 24.

[0125] Camera 22 captures images of the area around vehicle 2 and inside the vehicle's interior. Sensor group 23 includes identification sensors (external sensors) other than the camera. Examples of such identification sensors include LiDAR (Laser Imaging Detection and Ranging) and radar. Sensor group 23 also includes internal sensors of vehicle 2. Examples of such internal sensors include a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor. Sensor group 23 also includes a sensor (GPS sensor) for acquiring the vehicle 2's position information.

[0126] Communication device 24 performs wireless communication with a base station of a wireless network. Examples of communication standards for this wireless communication include 4G, LTE, or 5G mobile communication standards. The connection destination of communication device 24 includes at least management server 1. In communication with management server 1, communication device 24 transmits various data received from camera 22 and sensor group 23 to management server 1. If a remote support request is generated, communication device 24 transmits the request information data to management server 1. Furthermore, communication device 24 receives mode information (MDE) data from management server 1.

[0127] The running gear 25 includes, for example, a drive unit, a steering unit, and a braking unit. The drive unit drives the tires of the vehicle 2. The steering unit steers the tires of the vehicle 2. The braking unit applies braking force to the vehicle 2. Acceleration of the vehicle 2 is controlled by the drive unit. Deceleration of the vehicle 2 is controlled by the braking unit. If the drive unit is a motor, regenerative braking, controlled by the motor, can also be used to brake the vehicle 2. Steering of the vehicle 2 is controlled by the steering unit.

[0128] 5-3. Example of operator terminal 3 configuration

[0129] exist Figure 10 In the example shown, the operator terminal 3 includes a data processing device 31, a display 32, an input device 33, and a communication device 34. It should be noted that the display 32, input device 33, and other elements are connected to the data processing device 31 via a defined network.

[0130] The data processing device 31 is a computer used to process various data acquired by the vehicle 2. The data processing device 31 includes at least one processor 35 and at least one memory 36. The processor 35 includes a CPU. The memory 36 is used to run various programs employed by the processor 35 and to temporarily store various data. Various data acquired by the data processing device 31, and various data generated based on input signals from the input device 33, are stored in the memory 36. Examples of the former include camera image data of the vehicle 2 received from the management server 1. Examples of the latter include identification support and judgment support data. In the case of remote driving of the vehicle 2, the latter data also includes driving operation data.

[0131] Display 32 is a device for outputting camera images for providing remote support to vehicle 2. There is at least one display 32. Ideally, there are at least two displays 32 to output camera images of the area surrounding vehicle 2 and camera images of the interior of vehicle 2, respectively.

[0132] The input device 33 is a device operated by an operator (OP). The input device 33 includes, for example, an input section that accepts input from the operator (OP); and a control circuit that generates and outputs data based on the input to identify and determine support. Examples of input sections include touch panels, mice, keyboards, buttons, and switches. Examples of inputs from the operator (OP) include cursor movement operations output to the display 32 and button selection operations output to the display 32.

[0133] In the case of remote driving of vehicle 2, input device 33 may also include a driving input device. Examples of such driving input devices include a steering wheel, gear shift lever, accelerator pedal, and brake pedal.

[0134] The communication device 34 is connected to an external device via a communication network. The communication network is not particularly limited and can be either wired or wireless. The communication destination of the communication device 34 includes at least the management server 1. During communication with the management server 1, the communication device 34 sends various data related to determining the status (CND) of the remote support side to the management server 1. Examples of this various data include approval signals from predetermined operators (OPs) responding to remote support for vehicle 2, and data regarding the status of the operator selected from candidates for support during remote support execution. In the case of providing remote support for vehicle 2, the communication device 34 sends data such as support identification and support determination to the management server 1. Furthermore, the communication device 34 receives data such as camera images of vehicle 2 from the management server 1.

[0135] 6. Data Processing Example

[0136] Figure 11 This is a flowchart illustrating the process executed in management server 1 (processor 14) and particularly relevant to the implementation method. It should be noted that, to avoid repetitive explanations, in... Figure 11 In, based on Figure 7 The necessity of the combination of NDS and state CND is explained in the example of selecting the vehicle control mode MD. Figure 11 The routines shown are executed repeatedly, for example, according to a specified control cycle.

[0137] exist Figure 11 In the illustrated routine, firstly, it is determined whether there is a need for remote support NDS for vehicle 2 (step S1). The determination of whether there is a need for NDS is based on the data NDS stored in database 12. If the determination result of step S1 is affirmative (that is, there is a need for NDS ("yes")), the processing after step S2 is performed. If the determination result of step S1 is negative (that is, there is no need for NDS ("no")), the processing after step S7 is performed.

[0138] In step S2, it is determined whether the status CND of the remote support side is in the RD state. The determination of the status CND is based on the data DCND stored in database 12. It should be noted that the status of being ready to respond to remote support for vehicle 2 is the RD state, as already described.

[0139] If the determination result in step S2 is positive (that is, if the state CND is in the RD state), then the SCN mode is selected as the vehicle control mode MD (step S3). If the determination result in step S2 is negative (that is, if the state CND is in the URD state), then proceed with the processing after step S4. It should be noted that the state in which preparations for responding to remote support for vehicle 2 are not yet complete is the URD state, as already described.

[0140] In step S4, it is determined whether the status CND on the remote support side is in the AV state. The determination of the status CND has already been performed in step S2. Step S4 uses the result of step S2. It should be noted that the state where preparation for switching to the RD state is complete is the AV state, as already described.

[0141] If the determination result in step S4 is positive (that is, if the state CND is in AV state), then CNS mode is selected as the vehicle control mode MD (step S5). If the determination result in step S4 is negative (that is, if the state CND is in UAV state), then safety stop mode is selected as the vehicle control mode MD (step S6). It should be noted that the state in which preparation for switching to RD state is not completed is UAV state, as already described.

[0142] In step S7, it is determined whether the remote support side's CND status is RD. That is, the process in step S7 is the same as that in step S2. If the determination in step S7 is positive (i.e., the CND status is RD), the MDR mode is selected as the vehicle control mode MD (step S8). If the determination in step S7 is negative (i.e., the CND status is URD), the process proceeds to step S9 and beyond.

[0143] In step S9, it is determined whether the remote support side's state CND is in AV state. That is, the process in step S9 is the same as that in step S4. If the determination result in step S9 is positive (i.e., the state CND is in AV state), the SCN mode is selected as the vehicle control mode MD (step S10). If the determination result in step S9 is negative (i.e., the state CND is in UAV state), the CNS mode is selected as the vehicle control mode MD (step S11).

[0144] The following steps S3, S5, S6, S8, S10 or S11 will send the information of the vehicle control mode MD selected in these steps (that is, mode information MDE) to vehicle 2 (step S12).

[0145] 7. Effects

[0146] According to the implementation method described above, a vehicle control mode MD is set based on at least one of the necessity NDS for remote support of vehicle 2 and the state CND of the remote support side responding to the remote support, and vehicle control for autonomous driving of vehicle 2 is performed according to the information in the vehicle control mode MD. Therefore, for example, when the necessity NDS is switched from no ("no") to yes ("yes"), autonomous driving can be appropriately performed until the start of remote support for vehicle 2. Vehicle control can also be appropriately performed when autonomous driving restarts when the necessity NDS is switched from yes ("yes") to no ("no").

[0147] Furthermore, according to the implementation method, autonomous driving can be appropriately performed during the period up to the start of remote support, depending on the state of the remote support side (CND) when the Necessary NDS is switched from "no" ("no") to "yes" ("yes"). Furthermore, vehicle control can be appropriately performed when autonomous driving restarts, depending on the state of the remote support side (CND) when the Necessary NDS is switched from "yes" ("yes") to "no" ("no"). This is expected to improve the stability of autonomous driving execution when switching of the Necessary NDS occurs frequently. Therefore, according to the implementation method, it is expected to provide reassurance to the occupants of vehicle 2, or minimize the impact on traffic flow around vehicle 2.

Claims

1. A method for controlling a mobile body, characterized in that, Includes the following steps: Select the control mode for the autonomous driving of the mobile body; as well as The selected control mode information is sent to the mobile body. The control modes include a first mode, a second mode with stricter constraints related to driving safety compared to the first mode, and a third mode with stricter constraints compared to the first mode and more lenient constraints compared to the second mode. The control mode also includes a fourth mode that stops the moving body on the shoulder or roadside strip. The steps for selecting the control mode include the following: Determine whether remote support for the mobile body is required; Determine the status of the remote support side in response to remote support for the mobile body; and Select a control mode from the options including the first mode, the second mode, the third mode, and the fourth mode that corresponds to the combination of the determination result of whether remote support is needed and the determination result of the status of the remote support side. The state of the remote support side includes a ready state where preparations for responding to remote support are complete, and a non-ready state where preparations for responding to remote support are not complete. The non-ready state includes an available state where preparation for switching to the ready state is complete, and an unavailable state where preparation for switching is not complete. The step of determining the state of the remote support side includes the following steps: if the state of the remote support side is determined to be the non-ready state, determine whether the state of the remote support side is the available state or the unavailable state. The following processing is performed during the step of selecting the control mode: If it is determined that remote support is needed and the state of the remote support side is in the ready state, the third mode is selected as the control mode corresponding to the combination. If it is determined that remote support is needed and the remote support side is in a non-ready state but in an available state, then the second mode is selected as the control mode corresponding to the combination. If it is determined that remote support is needed and the state of the remote support side is in the non-ready state and the unavailable state, the fourth mode is selected as the control mode corresponding to the combination.

2. The control method for a moving body according to claim 1, characterized in that, The following processing is performed during the step of selecting the control mode: If it is determined that the remote support is not needed and the state of the remote support side is determined to be the ready state, the first mode is selected as the control mode corresponding to the combination. as well as If it is determined that the remote support is not needed and the state of the remote support side is determined to be the non-ready state, the third mode is selected as the control mode corresponding to the combination.

3. The control method for a moving body according to claim 1, characterized in that, The following processing is performed during the step of selecting the control mode: If it is determined that the remote support is not needed and the state of the remote support side is the non-ready state and the available state, the third mode is selected as the control mode corresponding to the combination. as well as If it is determined that the remote support is not needed and the state of the remote support side is the non-ready state and the unavailable state, the second mode is selected as the control mode corresponding to the combination.

4. A control device for a mobile body, characterized in that, a device for controlling a mobile body having automatic driving function and remote support function, wherein... Equipped with a processor The processor is configured to perform the following processes: Select the control mode for autonomous driving of the mobile body; and The selected control mode information is sent to the mobile body. The control modes include a first mode, a second mode with stricter constraints related to driving safety compared to the first mode, and a third mode with stricter constraints compared to the first mode and more lenient constraints compared to the second mode. The control mode also includes a fourth mode that stops the moving body on the shoulder or roadside strip. In the process of selecting the control mode, the processor is configured to perform the following processes: Determine whether remote support for the mobile body is required; Determine the status of the remote support side that responds to remote support for the mobile body; as well as Select a control mode from the options including the first mode, the second mode, the third mode, and the fourth mode that corresponds to the combination of the determination result of whether remote support is needed and the determination result of the status of the remote support side. The state of the remote support side includes a ready state where preparations for responding to remote support are complete, and a non-ready state where preparations for responding to remote support are not complete. The non-ready state includes an available state where preparation for switching to the ready state is complete, and an unavailable state where preparation for switching is not complete. The step of determining the state of the remote support side includes the following steps: if the state of the remote support side is determined to be the non-ready state, determine whether the state of the remote support side is the available state or the unavailable state. The following processing is performed during the step of selecting the control mode: If it is determined that remote support is needed and the state of the remote support side is in the ready state, the third mode is selected as the control mode corresponding to the combination. If it is determined that remote support is needed and the remote support side is in a non-ready state but in an available state, then the second mode is selected as the control mode corresponding to the combination. If it is determined that remote support is needed and the state of the remote support side is in the non-ready state and the unavailable state, the fourth mode is selected as the control mode corresponding to the combination.

Images