Control device, control system, control method and control program
The control system dynamically adjusts control policies based on vehicle state analysis, reducing the need for multiple monitors and enhancing operational efficiency in self-driving vehicles.
Patent Information
- Application Number
- JP2022509890
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-03-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-03-25
AI Technical Summary
Existing systems for remotely controlling self-driving vehicles require multiple monitors, leading to increased operational delays and inefficiencies due to the need for remote monitors to manage multiple vehicles simultaneously.
A control system and method that includes a vehicle with sensors and a control center capable of analyzing vehicle states and dynamically adjusting control policies, allowing for automatic or remote control based on the vehicle's state, reducing the need for multiple monitors.
The system enables efficient operation of self-driving vehicles by accurately responding to vehicle states, minimizing the need for remote monitors and preventing delays, while allowing for smooth transitions between automatic and remote control modes.
Smart Images

Figure 0007589736000001 
Figure 0007589736000002 
Figure 0007589736000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a control device, a control system, and a control method. [Background technology]
[0002] In autonomous vehicles of level 4 and above, it is assumed that there will be no driver inside the vehicle. Therefore, it is desirable to build a monitoring system that will allow a remote monitor to be stationed to watch over the vehicle and to deal with any abnormalities remotely.
[0003] For example, Patent Document 1 describes a vehicle remote operation support system that includes a vehicle capable of running in three modes, an automatic driving mode, a manual driving mode, and a remote driving mode, and a control center that can remotely operate the vehicle. In the vehicle remote operation support system of Patent Document 1, when the vehicle determines that automatic driving is difficult, the vehicle requests remote operation from an operator at the control center.
[0004] Patent Document 2 describes a remote control device in which an operator intervenes to remotely control an autonomous vehicle. The remote control device of Patent Document 2 describes that the remote control device remotely controls the autonomous vehicle based on the driving trajectory linked to similar data in addition to the vehicle information of the autonomous vehicle.
[0005] Patent Document 3 describes an automatic driving control system that controls automatic driving of a vehicle from outside the vehicle. The automatic driving control system of Patent Document 3 describes that when a sensor unit of the vehicle detects the occurrence of a predetermined situation, the automatic driving control corresponding to the predetermined situation is executed in priority to the external control.
[0006] Patent Document 4 describes a management device that manages multiple autonomous vehicles and multiple remote drivers, the number of which is less than the number of the autonomous vehicles. The management device in Patent Document 4 describes that a standby remote driver is assigned to the remote driver of an autonomous vehicle that needs remote driving. [Prior art documents] [Patent documents]
[0007] [Patent Document 1] JP 2019-191982 A [Patent Document 2] JP 2019-185280 A [Patent Document 3] JP 2019-106674 A [Patent Document 4] JP 2018-142265 A Summary of the Invention [Problem to be solved by the invention]
[0008] In Patent Documents 1 to 4, a vehicle to be remotely controlled is remotely controlled by an operator (remote monitor) or the like on the control side. The remote monitor needs to keep a close eye on the vehicle he or she is in charge of. Therefore, when multiple autonomous vehicles are operated, the number of remote monitors who remotely monitor the vehicles must be increased. Furthermore, while remotely operating an autonomous vehicle, the remote monitor is forced to suspend the operation of other autonomous vehicles. This causes a disruption to the operation of autonomous vehicles.
[0009] The object of the present disclosure has been made to solve such problems, and is to provide a control device, a control system, and a control method that can smooth the operation of autonomous vehicles. [Means for solving the problem]
[0010] The control device of the present disclosure includes an analysis means for analyzing the state of a vehicle having an autonomous driving function based on information regarding the vehicle, an identification means for identifying a control policy, which is a predetermined control to be transmitted to the vehicle, based on the analyzed state of the vehicle, and a remote control means for controlling the driving of the vehicle based on the identified control policy.
[0011] In addition, the control system disclosed herein comprises a vehicle capable of autonomous driving and a control center that remotely controls the vehicle, wherein the vehicle has a sensor that detects information related to the vehicle and a vehicle control means that controls the autonomous driving based on the information detected by the sensor, and the control center has an analysis means that analyzes a state of the vehicle based on the information received from the vehicle, an identification means that identifies a control policy, which is a predetermined control, to be sent to the vehicle based on the analyzed state of the vehicle, and a remote control means that controls the driving of the vehicle based on the identified control policy.
[0012] In addition, the control method disclosed herein analyzes the state of the vehicle based on information regarding a vehicle capable of autonomous driving, identifies a control policy, which is a predetermined control to be sent to the vehicle, based on the analyzed state of the vehicle, and controls the driving of the vehicle based on the identified control policy. Effect of the Invention
[0013] According to the present disclosure, it is possible to provide a vehicle control system, a vehicle control center, and a vehicle control method that can facilitate smooth operation of autonomous vehicles. [Brief description of the drawings]
[0014] [Figure 1] 1 is a configuration diagram illustrating a control system according to a first embodiment. [Diagram 2] 1 is a configuration diagram illustrating a control device according to a first embodiment. [Diagram 3] 4 is a flowchart illustrating an operation of the vehicle according to the first embodiment. [Figure 4] 3 is a flowchart illustrating a control method of the control device according to the first embodiment. FIG. [Diagram 5] 3 is a sequence diagram illustrating a control method of the control system according to the first embodiment. FIG. [Figure 6] FIG. 11 is a configuration diagram illustrating a control system according to a second embodiment. [Figure 7]FIG. 11 is a configuration diagram illustrating a control device according to a second embodiment. [Figure 8] FIG. 11 is a flowchart illustrating the operation of a vehicle according to a second embodiment. [Figure 9] FIG. 11 is a flowchart illustrating a control method of the control device according to the second embodiment. [Figure 10] 11 is a sequence diagram illustrating a control method of the control system according to the second embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Hereinafter, the embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. In addition, in each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.
[0016] (Embodiment 1) A control system according to a first embodiment will be described. First, an overview of the control system will be described. Then, a vehicle and a control device constituting the control system will be described. After that, a control method using the control system will be described.
[0017] <Control System> First, a description will be given of a control system according to embodiment 1. Fig. 1 is a configuration diagram illustrating a control system according to embodiment 1. As shown in Fig. 1, a control system 100 includes a vehicle 110 and a control center 120c.
[0018] The control system 100 is a system that controls a vehicle 110 having an automatic driving function. In the control system 100 of this embodiment, the control center 120c receives sensor information, information on the state of the vehicle 110, and the like from the vehicle 110 via a network NW. The control center 120c analyzes the state of the vehicle 110 based on the received information on the vehicle 110, and controls the vehicle 110 according to the analysis result. For example, the control center 120c remotely controls the vehicle 110. Alternatively, the control center 120c causes the vehicle 110 to automatically control the vehicle 110 by transmitting an automatic driving control policy.
[0019] The control center 120c includes a control device 120. The vehicle 110 and the control device 120 that constitute the control system 100 will be described below.
[0020] <Vehicle> The vehicle 110 has an automatic driving function that enables automatic driving. The vehicle 110 having the automatic driving function is, for example, an automatic driving vehicle such as a private car, a taxi, a bus, or a truck. Note that the vehicle 110 having the automatic driving function is not limited to an automatic driving vehicle that runs on a road, and may be a train that runs on a railroad. The vehicle 110 has a sensor 111 and a vehicle control unit 112.
[0021] The sensor 111 detects information related to the vehicle 110. The sensor 111 is, for example, a camera, a speedometer, a steering angle indicator, a GPS (Global Positioning System) receiver, etc. The camera captures images of the front, surroundings, or interior of the vehicle 110. The speedometer measures the speed of the vehicle 110. The steering angle indicator detects the heading direction of the vehicle 110. The GPS receiver detects the position of the vehicle 110. Note that the sensor 111 is not limited to a camera, a speedometer, a steering angle indicator, a GPS receiver, etc., as long as it detects information related to the vehicle 110.
[0022] The sensor 111 outputs information about the detected vehicle 110 to the vehicle control unit 112. The sensor 111 also transmits the information about the detected vehicle 110 to the control center 120c via the network NW. The information about the detected vehicle 110 includes, for example, sensor information such as image information ahead of the vehicle 110, image information inside the vehicle, speed, course direction, and position. The information about the detected vehicle 110 also includes vehicle state information such as the running state of the vehicle 110 and the surrounding conditions of the vehicle 110 derived from the sensor information.
[0023] The vehicle control unit 112 controls the vehicle 110. Therefore, the vehicle control unit 112 has a function as a vehicle control means. The vehicle control unit 112 is connected to an ECU (Electronic Control Unit) and controls the vehicle 110 by controlling the ECU. The vehicle control unit 112 may be configured as hardware including a microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an interface unit (I / F), and the like. The CPU performs analysis processing and control processing of the state of the vehicle 110 based on information about the vehicle 110. The ROM stores analysis programs and control programs executed by the CPU. The RAM stores various data such as information about the vehicle 110. The interface unit (I / F) inputs and outputs signals to and from the outside, such as a network NW. The CPU, ROM, RAM, and interface unit are connected to each other via a data bus or the like.
[0024] The vehicle control unit 112 controls the autonomous driving based on information about the vehicle 110 detected by the sensor 111. For example, when the sensor 111 detects an obstacle ahead of the vehicle 110, the vehicle control unit 112 controls the vehicle 110 to stop. In addition, when the sensor 111 detects that the speed of the vehicle 110 is equal to or greater than the speed limit, the vehicle control unit 112 controls the vehicle 110 to decelerate.
[0025] The vehicle control unit 112 controls the vehicle 110 based on a control policy output from the control center 120c. The control policy is a predetermined control transmitted to the vehicle 110. The control policy includes an automatic control policy and a remote control policy. The automatic control policy controls the vehicle 110 to perform automatic driving. The remote control policy controls the vehicle 110 to be remotely driven. For example, when the vehicle control unit 112 receives an automatic control policy from the control center 120c to cause the vehicle 110 to perform automatic driving, the vehicle control unit 112 controls the automatic driving of the vehicle 110 based on information related to the vehicle 110.
[0026] On the other hand, when the vehicle control unit 112 receives a remote control policy for remotely controlling the vehicle 110 from the control center 120c, the vehicle control unit 112 controls the vehicle 110 based on the remote control from the control center 120c. The control policy may include rules related to autonomous driving in addition to the automatic control policy and the remote control policy. The rules related to autonomous driving include, for example, a driving control policy such as compliance with the speed limit.
[0027] <Control device> Next, the control device 120 will be described. Fig. 2 is a configuration diagram illustrating the control device 120 according to the embodiment 1. As shown in Fig. 2, the control device 120 provided in the control center 120c is a standalone device, and may be incorporated into the control system 100 as in Fig. 1.
[0028] The control device 120 may be configured with hardware including a general-purpose computer configured with a microcomputer including a CPU, RAM, an interface unit, etc., as well as a GPU, FPGA, and a HDD, SSD, etc. as storage. The CPU, GPU, FPGA, etc. perform analysis processing and control processing, etc. based on information related to the vehicle 110. The ROM, HDD, SSD, etc. store analysis programs and control programs executed by the CPU, etc. The RAM, HDD, SSD, etc. store various data such as information related to the vehicle 110. The interface unit (I / F) inputs and outputs signals to and from the outside, such as the network NW. The CPU, GPU, FPGA, ROM, RAM, HDD, SSD, and interface unit are connected to each other via a data bus, connection wiring, etc.
[0029] 1 and 2, a control device 120 controls a vehicle 110 having an automatic driving function capable of automatic driving. The control device 120 includes an analysis unit 121, an identification unit 122, and a remote control unit 123. The analysis unit 121, the identification unit 122, and the remote control unit 123 function as an analysis means, an identification means, and a remote control means, respectively.
[0030] The analysis unit 121 analyzes the state of the vehicle 110 based on information about the vehicle 110 having an automatic driving function. The information about the vehicle 110 is, for example, information detected by a sensor 111 provided in the vehicle 110. The analysis unit 121 receives the information detected by the sensor 111 via the network NW.
[0031] The analysis unit 121 analyzes the state of the vehicle 110 based on the information received from the vehicle 110. The state of the vehicle 110 includes, for example, a state in which the vehicle 110 is at high risk and a state in which the vehicle 110 is at high urgency. The state of the vehicle 110 also includes a state in which automatic control is possible and a state in which remote control is desirable. The analysis unit 121 may set the high risk and high urgency states in advance.
[0032] For example, among the sensors 111, a camera installed in front of the vehicle 110 acquires image information of the area ahead of the vehicle 110. Thus, the information relating to the vehicle 110 includes image information of the area ahead of the vehicle 110. When image information of the area ahead of the vehicle 110 is transmitted as the information relating to the vehicle 110, the analysis unit 121 stereoscopically recognizes the traffic environment, such as roads, intersections, traffic signs, and traffic lights, ahead of the vehicle 110 in three dimensions, based on the image information of the area ahead of the vehicle 110.
[0033] The analysis unit 121 also recognizes surrounding vehicles and people based on image information of the area ahead of the vehicle 110. Furthermore, the analysis unit 121 calculates in real time the time and position until the vehicle 110 intersects with the surrounding vehicles and people at an intersection based on the image information of the area ahead of the vehicle 110. The analysis unit 121 then analyzes the degree of risk of collision with the recognized surrounding vehicles or people in a traffic environment including the recognized roads, intersections, etc., as the state of the vehicle 110. For example, the analysis unit 121 may convert the state of the degree of risk of the vehicle 110 into a numerical value and analyze it.
[0034] Furthermore, when there are many people present within a range geometrically determined based on the traveling direction of the vehicle 110 based on image information ahead of the vehicle 110, the analysis unit 121 analyzes the risk level of the vehicle 110 as the state of the vehicle 110. For example, the analysis unit 121 may analyze that the risk level is high when the number of people present within a predetermined range geometrically determined is equal to or exceeds a threshold value.
[0035] Among the sensors 111, a camera installed inside the vehicle 110 acquires video information of the interior of the vehicle 110. Thus, the information about the vehicle 110 includes video information about the interior of the vehicle 110. When video information about the interior of the vehicle 110 is transmitted as information about the vehicle 110, the analysis unit 121 recognizes the actions and states of the passengers inside the vehicle based on the video information about the interior of the vehicle 110. Then, the analysis unit 121 analyzes the urgency of the recognized actions and states of the passengers inside the vehicle as the state of the vehicle 110. For example, the analysis unit 121 analyzes a state in which a passenger has fallen or is unsteady as a state of high urgency. The analysis unit 121 may analyze the state of urgency of the vehicle 110 by quantifying the number of passengers who have fallen or the frequency of unsteady movement.
[0036] Furthermore, when vehicle 110 is a bus and the information about vehicle 110 includes video information about the interior of the bus, analysis unit 121 may analyze passengers inside the bus and their postures as the state of vehicle 110. For example, analysis unit 121 may analyze the state of urgency of vehicle 110 by quantifying the number of people using canes inside the bus.
[0037] In this manner, the analysis unit 121 may analyze the predetermined degree of danger and urgency of the vehicle 110 as the state of the vehicle 110.
[0038] The identification unit 122 identifies a control policy based on the analyzed state of the vehicle 110. The control policy is a predetermined control that is transmitted to the vehicle 110. For example, the control policy includes an automatic control policy that causes the vehicle 110 to perform automatic driving. In addition, the control policy includes a remote control policy that controls the vehicle 110 to be remotely driven.
[0039] For example, when the analyzed state of the vehicle 110 corresponds to a predetermined danger level or higher, or a predetermined urgency level or higher, the identification unit 122 identifies the control policy as the remote control policy. Specifically, when the recognized danger level of collision with a surrounding vehicle or person is a predetermined threshold level or higher, the identification unit 122 identifies the control policy as the remote control policy. Also, when the recognized urgency level of the movement and state of the passenger in the vehicle is a predetermined threshold level or higher, the identification unit 122 identifies the control policy as the remote control policy. Also, when a passenger in the bus is a person who is likely to fall over or has a posture that is likely to fall over, the identification unit 122 may identify the control policy as the remote control policy when it is considered that remote control is necessary, such as when the vehicle 110 has stopped due to a factor other than the above-mentioned situation.
[0040] On the other hand, when the analyzed state of the vehicle 110 corresponds to a state lower than the predetermined danger level or the predetermined urgency level, the identifying unit 122 identifies the control policy as the automatic control policy.
[0041] The remote control unit 123 remotely controls the driving of the vehicle 110 based on the identified control policy. For example, when the identified control policy is an automatic control policy, the remote control unit 123 controls the vehicle 110 to drive automatically. On the other hand, when the identified control policy is a remote control policy, the remote control unit 123 controls the vehicle 110 to be remotely controlled.
[0042] Next, a control method will be described as an operation of the control system 100 according to the embodiment 1. The control method will be described separately for the operation of the vehicle 110, the control method of the control device 120, and the control method of the control system 100.
[0043] <Vehicle operation> FIG. 3 is a flow chart illustrating an operation of the vehicle 110 according to the first embodiment. As shown in step S111 of FIG. 3, information about the vehicle 110 is detected. For example, the sensor 111 provided in the vehicle 110 detects information about the vehicle 110 capable of autonomous driving. The information about the vehicle 110 is, for example, image information in front of the vehicle 110, image information inside the vehicle, speed, course direction, position, and the like. If the vehicle 110 has not received a control policy, it waits as it is. If the current control policy is an automatic control policy, the vehicle 110 continues autonomous driving based on the detected information. If the current control policy is a remote control policy, the vehicle 110 is remotely controlled by the control center 120c.
[0044] Next, as shown in step S112, the detected information is transmitted to the control center 120c. Specifically, the sensor 111 transmits information related to the detected vehicle 110 to the control center 120c (control device 120).
[0045] Next, as shown in step S113, a control policy is received. For example, the vehicle control unit 112 receives the control policy from the control device 120. If the control policy has not arrived from the control device 120 in step S113, the process may proceed to step S114. In other words, the current control policy may be maintained. In this way, the control policy may be received asynchronously in step S113.
[0046] Next, as shown in step S114, the driving of the vehicle 110 is controlled based on the control policy. For example, if the control policy received from the management device 120 is an automatic control policy, the vehicle control unit 112 performs automatic driving. On the other hand, if the control policy received from the management device 120 is a remote control policy, the vehicle control unit 112 follows the remote control by the management device 120. If the vehicle control unit 112 receives a new control policy that is different from the previous control policy, it follows the new control policy.
[0047] <Control method of the control device> Next, a control method of the control device 120 will be described. Fig. 4 is a flow chart illustrating the control method of the control device 120 according to the first embodiment. As shown in step S121 of Fig. 4, the control device 120 receives information related to the vehicle 110. For example, the analysis unit 121 in the control device 120 receives information related to the vehicle 110 detected by the sensor 111 from the vehicle 110 via the network NW.
[0048] Next, as shown in step S122, the state of the vehicle 110 is analyzed based on the received information. For example, the analysis unit 121 analyzes the degree of danger or urgency of the vehicle 110 based on the received information related to the vehicle 110.
[0049] Next, as shown in step S123, a control policy for vehicle 110 is identified based on the analyzed state of vehicle 110. For example, identification unit 122 identifies the control policy for vehicle 110 as an automatic control policy, a remote control policy, or the like based on the analyzed risk level or urgency level of vehicle 110. When the state of vehicle 110, such as the risk level or urgency level, has changed, identification unit 122 identifies a new control policy.
[0050] Next, as shown in step S124, the driving of the vehicle 110 is controlled based on the control policy. For example, when the control policy identified by the identification unit 122 is an automatic control policy, the remote control unit 123 controls the vehicle 110 to perform automatic driving. On the other hand, when the control policy identified by the identification unit 122 is a remote control policy, the remote control unit 123 remotely controls the vehicle 110.
[0051] <Control method of the control system> Next, a control method of the control system 100 will be described. FIG. 5 is a sequence diagram illustrating a control method of the control system 100 according to the first embodiment. As shown in step S131 of FIG. 5, the vehicle 110 detects information related to the vehicle 110. For example, the vehicle 110 detects information related to the vehicle 110 capable of autonomous driving by the sensor 111. If the vehicle 110 has not received a control policy, the vehicle 110 waits as it is. If the current control policy is an automatic control policy, the vehicle 110 continues autonomous driving based on the detected information. If the current control policy is a remote control policy, the vehicle 110 is remotely controlled by the control center 120c.
[0052] Next, as shown in step S132, the vehicle 110 transmits information about the detected vehicle 110 to the control center 120c. For example, the vehicle 110 transmits the information to the control device 120 via the network NW. As a result, as shown in step S133, the control device 120 receives the information about the vehicle 110. For example, the control device 120 receives the information about the vehicle 110 via the network NW.
[0053] Next, as shown in step S134, the control device 120 analyzes the state of the vehicle 110 based on the received information. For example, the control device 120 analyzes the degree of danger or urgency of the vehicle 110 based on the received information related to the vehicle 110.
[0054] Next, as shown in step S135, the control device 120 specifies a control policy for the vehicle 110 based on the analyzed state of the vehicle 110. For example, the control device 120 specifies the control policy for the vehicle 110 as an automatic control policy, a remote control policy, or the like based on the analyzed risk level or urgency level of the vehicle 110. When the state of the vehicle 110, such as the risk level or urgency level, changes, the control device 120 specifies a new control policy.
[0055] Next, as shown in step S136, the control device 120 transmits the control policy to the vehicle 110. For example, the control device 120 transmits the control policy to the vehicle 110 via the network NW. As a result, as shown in step S137, the vehicle 110 receives the control policy from the control device 120 via the network NW.
[0056] Next, as shown in step S138, the vehicle 110 controls the driving of the vehicle 110 based on the control policy. For example, if the control policy identified by the management device 120 is an automatic control policy, the vehicle 110 performs automatic driving. On the other hand, if the control policy identified by the management device 120 is a remote control policy, the vehicle 110 follows the remote control by the management device 120. If the vehicle 110 receives a new control policy that is different from the control policy previously received, the vehicle 110 follows the new control policy. In this way, the vehicle 110 controls the driving of the vehicle 110 based on the control policy.
[0057] Next, the effects of this embodiment will be described. In this embodiment, the control device 120 analyzes the state of the vehicle 110 based on information related to the vehicle 110, and identifies a control policy for the vehicle 110 based on the analyzed state of the vehicle 110. Therefore, since the control of the vehicle 110 is changed depending on the state of the vehicle 110, it is possible to appropriately deal with the state of the vehicle 110, and to facilitate smooth operation of the autonomous vehicle.
[0058] For example, when the risk or urgency of the vehicle 110 rises from low to medium, the speed control upper limit value of the automatic control policy is switched from medium to low. Furthermore, when the risk or urgency of the vehicle 110 is high, the automatic control is switched to remote control. Therefore, since automatic control does not require remote control, the number of remote monitors can be reduced. Also, since the remote monitor can keep a close eye on the vehicle 110 with a high risk or urgency, it is possible to prevent the operation of the autonomous vehicle from being disrupted.
[0059] (Embodiment 2) Next, a control system according to a second embodiment will be described. In the control system of this embodiment, at least one of the vehicle and the control device is provided with a learning device. Fig. 6 is a configuration diagram illustrating the control system according to the second embodiment. Fig. 7 is a configuration diagram illustrating the control device according to the second embodiment.
[0060] <Control System> 6, the control system 200 of this embodiment includes a vehicle 210 and a control center 220c. Each component of the control system 200 will be described below.
[0061] <Vehicle> The vehicle 210 includes a sensor 211 and a vehicle control unit 212, and the vehicle control unit 212 functions as a control means for controlling the vehicle 210. The vehicle control unit 212 of this embodiment includes a learning device 213. The learning device 213 functions as a learning means.
[0062] The learning device 213 is, for example, an AI (Artificial Intelligence). The learning device 213 learns rules for controlling the autonomous driving of the vehicle 210 based on information about the vehicle 210. For convenience, the rules for controlling the autonomous driving of the vehicle 210 are referred to as autonomous driving rules in this specification. The autonomous driving rules may include driving control such as compliance with speed limits.
[0063] The learning device 213 learns the automatic driving rule by learning the association between the information about the vehicle 210 detected by the sensor 211 and the automatic driving control of the vehicle 210 based on the information. Then, the vehicle control unit 212 analyzes the state of the vehicle 210 controlled by the automatic driving rule based on the information about the vehicle 210. The vehicle 210 transmits the detected information about the vehicle 210 and the analyzed state of the vehicle 210 to the control center 120c. Other configurations of the vehicle 210 are the same as the configuration of the vehicle 110 in the first embodiment.
[0064] <Control device> 7, a control center 220c is provided with a control device 220. The control device 220 is a standalone device, and may be incorporated into the control system 200 as shown in FIG.
[0065] 6 and 7, the control device 220 of this embodiment includes an analysis unit 221, an identification unit 222, and a remote control unit 223. The analysis unit 221, the identification unit 222, and the remote control unit 223 have the functions of analysis means, identification means, and remote control means, respectively, similar to the first embodiment. The analysis unit 221 of this embodiment includes a learning device 224. The learning device 224 has the function of learning means.
[0066] The learning device 224 is, for example, an AI. The learning device 224 learns rules for analyzing the state of the vehicle 210 based on information about the vehicle 210. In this specification, the rules for analyzing the state of the vehicle 210 are conveniently referred to as analysis rules. The analysis rules may include a recognition priority between a person and an obstacle in a camera image, an image processing resolution, an estimated pattern of behavior prediction from image information, and the like. Furthermore, the information about the vehicle 210 used for learning by the learning device 224 may be information about a plurality of vehicles 210.
[0067] The learning device 224 learns the analysis rule by learning the association between the information about the vehicle 210 detected by the sensor 211 and the analysis result of the state of the vehicle 210 based on the information. The learning device 224 may learn the association between the information about each vehicle 210 detected by each sensor 211 of the multiple vehicles 210 and the analysis result of the state of each vehicle 210 based on the information of each vehicle. Then, the identification unit 222 identifies the control policy based on the state of the vehicle 210 analyzed by the analysis rule. When the learning device 213 provided in the vehicle 210 is learning the automatic driving rule, the identification unit 222 identifies the control policy based on the analysis rule and the state of the vehicle 210 analyzed by the automatic driving rule. The other configuration of the control device 220 is the same as the configuration of the control device 120 of the first embodiment.
[0068] Next, a control method will be described as an operation of the control system 200 according to the embodiment 2. The control method will be described separately for the operation of the vehicle 210, the control method of the control device 220, and the control method of the control system 200.
[0069] <Vehicle operation> Fig. 8 is a flow chart illustrating an example of the operation of the vehicle 210 according to the embodiment 2. As shown in step S211 of Fig. 8, information related to the vehicle 210 is detected. For example, the sensor 211 detects information related to the vehicle 210 capable of autonomous driving.
[0070] Next, as shown in step S212, by using the learning device 213 that has learned the autonomous driving rules, the state of the vehicle 210 controlled by the autonomous driving rules is analyzed based on information about the vehicle 210. For example, the vehicle control unit 212 analyzes that the vehicle 210 is in a high-risk state or a low-risk state according to the autonomous driving rules, based on image information in front of the vehicle 210.
[0071] Next, as shown in step S213, the detected information and the state of the vehicle 210 controlled by the automatic driving rule are transmitted to the control center 220c. Specifically, the sensor 211 transmits image information of the area ahead of the vehicle 210 to the control device 220. Furthermore, the vehicle control unit 212 transmits to the control device 220 that the vehicle 210 is in a high-risk state or a low-risk state based on the image information of the area ahead of the vehicle 210. Here, when the image information of the sensor 211 and the state of the risk analyzed by the vehicle control unit 212 are transmitted separately, information for matching them with each other at the control center may be added.
[0072] Next, as shown in step S214, the control policy is received. For example, the vehicle control unit 212 receives the control policy from the control device 220. If the control policy has not arrived from the control device 220 in step S214, the process may proceed to step S215. In this way, the control policy may be received asynchronously in step S214.
[0073] Next, as shown in step S215, the driving of the vehicle 210 is controlled based on the control policy. For example, if the control policy received from the management device 220 is an automatic control policy, the vehicle control unit 212 performs automatic driving. On the other hand, if the control policy received from the management device 220 is a remote control policy, the vehicle control unit 212 follows the remote control by the management device 220.
[0074] <Control method of the control device> Next, a control method of the control device 220 will be described. FIG. 9 is a flow chart illustrating a control method of the control device 220 according to the second embodiment. As shown in step S221 of FIG. 9, information about the vehicle 210 and the state of the vehicle 210 controlled by the autonomous driving rule are received from the vehicle 210. For example, the analysis unit 221 in the control device 220 receives information about the vehicle 210 detected by the sensor 211 via the network NW. In addition, the identification unit 222 receives the state of the vehicle 210 controlled by the autonomous driving rule learned by the learning device 213 of the vehicle 210.
[0075] Next, as shown in step S222, the state of the vehicle 210 is analyzed based on the received information by using the learning device 224 that has learned the analysis rule. For example, the analysis unit 221 analyzes, based on the received image information of the area ahead of the vehicle 210, that the vehicle 210 is in a high-risk state or a low-risk state by using the analysis rule of the learning device 224.
[0076] Next, as shown in step S223, a control policy for the vehicle 210 is identified based on the state of the vehicle 210 analyzed by the analysis rule and the automatic driving rule. For example, the identification unit 222 identifies the control policy for the vehicle 210 as an automatic control policy or a remote control policy based on the analyzed risk level or urgency level of the vehicle 210.
[0077] Next, as shown in step S224, the driving of the vehicle 210 is controlled based on the control policy. For example, when the control policy identified by the identification unit 222 is an automatic control policy, the remote control unit 223 controls the vehicle 210 to perform automatic driving. On the other hand, when the control policy identified by the identification unit 222 is a remote control policy, the remote control unit 223 remotely controls the vehicle 210.
[0078] <Control method of the control system> Next, a description will be given of a control method of the control system 200. Fig. 10 is a sequence diagram illustrating the control method of the control system 200 according to the second embodiment.
[0079] As shown in step S231 of FIG. 10, the vehicle 210 detects information relating to the vehicle 210 by the sensor 211.
[0080] Next, as shown in step S232, the vehicle 210 uses the learner 213 that has learned the autonomous driving rules to analyze the state of the vehicle 210 controlled by the autonomous driving rules, based on information about the vehicle 210. For example, the vehicle 210 is analyzed to be in a high-risk state or a low-risk state.
[0081] Next, as shown in step S233, the vehicle 210 transmits the detected information and the state of the vehicle 210 controlled by the autonomous driving rule to the control center 220c. As a result, as shown in step S234, the control device 220 receives information about the vehicle 210 and the state of the vehicle 210 controlled by the autonomous driving rule.
[0082] Next, as shown in step S235, the state of the vehicle 210 is analyzed according to the analysis rule based on the received information. For example, by using the learning device 224 that has learned the analysis rule, the analysis unit 221 analyzes whether the vehicle 210 is in a high-risk state or a low-risk state based on the received information.
[0083] Next, as shown in step S236, the control policy is identified. For example, the identification unit 222 identifies the control policy of the vehicle 210 as an automatic control policy or a remote control policy based on the risk or urgency of the vehicle 210 analyzed by the analysis rule and the automatic driving rule.
[0084] Next, as shown in step S237, the control device 220 transmits the control policy to the vehicle 210. As a result, the vehicle 210 receives the control policy from the control device 220 as shown in step S238.
[0085] Next, as shown in step S239, the vehicle 210 controls the driving of the vehicle 210 based on the control policy. For example, when the control policy specified by the management device 220 is an automatic control policy, the management device 220 controls the vehicle 210 to perform automatic driving. Thus, the vehicle 210 performs automatic driving. On the other hand, when the control policy specified by the management device 220 is a remote control policy, the management device 220 remotely controls the vehicle 210. Thus, the vehicle 210 follows the remote control by the management device 220. In this way, the management device 220 controls the driving of the vehicle 210 based on the control policy.
[0086] According to the control system 200 of this embodiment, the vehicle control unit 212 includes a learner 213 that learns the automatic driving rules, and therefore it is possible to improve the accuracy and decision speed of the automatic control of the vehicle 210. Furthermore, the analysis unit 221 includes a learner 224 that learns the analysis rules, and therefore it is possible to improve the accuracy and analysis speed of analyzing the state of the vehicle 210. Furthermore, the identification unit 222 identifies a control policy based on the state of the vehicle 210 analyzed by the analysis rules and the automatic driving rules, and therefore it is possible to improve the suitability of the control policy.
[0087] By performing pattern learning by AI-based learners 213 and 224 on both the vehicle 210 side and the control device 220 side, it is possible to improve the control accuracy and decision speed in both the automatic control and remote control of the vehicle 210. This makes it possible to smooth the operation of the vehicle 210. Other operations and effects of the vehicle 210, the control device 220, and the control system 200 are included in the description of the first embodiment.
[0088] Although the first and second embodiments have been described above, the present invention is not limited to the first and second embodiments, and can be modified as appropriate without departing from the spirit of the present invention. For example, an embodiment that combines the configurations of the first and second embodiments is also included in the scope of the technical idea. In addition, a control program that causes a computer to execute the control method of the first and second embodiments is also included in the technical scope of the first and second embodiments.
[0089] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.
[0090] (Appendix 1) An analysis means for analyzing a state of a vehicle having an automatic driving function based on information related to the vehicle; A determination means for determining a control policy, which is a predetermined control to be transmitted to the vehicle, based on the analyzed state of the vehicle; a remote control means for controlling the operation of the vehicle based on the identified control policy; A control device equipped with
[0091] (Appendix 2) the analysis means includes a control-side learning means that learns an analysis rule for analyzing a state of the vehicle based on the information, The identification means identifies the control policy based on the state of the vehicle analyzed by the analysis rule. 2. The control device described in appendix 1.
[0092] (Appendix 3) a vehicle-side learning means provided in the vehicle learns an automatic driving rule for controlling the automatic driving based on the information; The identification means identifies the control policy based on the state of the vehicle analyzed by the analysis rule and the autonomous driving rule. 2. The control device described in appendix 2.
[0093] (Appendix 4) the control policy includes an automatic control policy for causing the vehicle to perform automatic driving and a remote control policy for remotely controlling the vehicle, The remote control means controls the vehicle to automatically drive when the identified control policy is the automatic control policy, and controls the vehicle to remotely control when the identified control policy is the remote control policy. 4. The control device according to claim 1,
[0094] (Appendix 5) the information is image information of a scene ahead of the vehicle, the analysis means analyzes a risk of collision with a recognized surrounding vehicle or person in a traffic environment including the recognized road and intersection as a state of the vehicle; the identifying means identifies the control policy as a remote control policy when the degree of risk is equal to or greater than a threshold. 6. The control device described in appendix 4.
[0095] (Appendix 6) the information is video information of the interior of the vehicle, the analyzing means analyzes the recognized motion of the passenger in the vehicle and the urgency of the state of the passenger as a state of the vehicle; the identifying means identifies the control policy as a remote control policy when the urgency is equal to or greater than a threshold value; 6. The control device described in appendix 4.
[0096] (Appendix 7) the vehicle is a bus, the information is video information of the interior of the vehicle, The analysis means analyzes passengers in the vehicle and their postures as a state of the vehicle, the identifying means identifies the control policy as a remote control policy when the passenger is a person prone to falling or when the posture is a posture prone to falling. 6. The control device described in appendix 4.
[0097] (Appendix 8) A vehicle having an autonomous driving function; A control center that remotely controls the vehicle; Equipped with The vehicle is A sensor that detects information about the vehicle; A vehicle control means for controlling the autonomous driving based on the information detected by the sensor; having The control center includes: an analysis means for analyzing a state of the vehicle based on the information received from the vehicle via a network; A determination means for determining a control policy, which is a predetermined control to be transmitted to the vehicle, based on the analyzed state of the vehicle; a remote control means for controlling the operation of the vehicle based on the identified control policy; having Control system.
[0098] (Appendix 9) the analysis means includes a control-side learning means that learns an analysis rule for analyzing a state of the vehicle based on the information, The identification means identifies the control policy based on the state of the vehicle analyzed by the analysis rule. 7. The control system described in appended note 8.
[0099] (Appendix 10) the vehicle control means includes a vehicle-side learning means that learns an automatic driving rule for controlling the automatic driving based on the information, The identification means identifies the control policy based on the state of the vehicle analyzed by the analysis rule and the autonomous driving rule. 9. The control system described in Appendix 9.
[0100] (Appendix 11) the control policy includes an automatic control policy for causing the vehicle to perform automatic driving and a remote control policy for remotely controlling the vehicle, The remote control means controls the vehicle to automatically drive when the identified control policy is the automatic control policy, and controls the vehicle to remotely control when the identified control policy is the remote control policy. 11. The control system according to any one of claims 8 to 10.
[0101] (Appendix 12) the information is image information of a scene ahead of the vehicle, the analysis means analyzes a risk of collision with a recognized surrounding vehicle or person in a traffic environment including the recognized road and intersection as a state of the vehicle; the identifying means identifies the control policy as a remote control policy when the degree of risk is equal to or greater than a threshold. 12. The control system described in claim 11.
[0102] (Appendix 13) the information is video information of the interior of the vehicle, The analysis means analyzes the behavior of passengers in the vehicle and the urgency of the passengers' conditions as the state of the vehicle, the identifying means identifies the control policy as a remote control policy when the urgency is equal to or greater than a threshold value; 12. The control system described in claim 11.
[0103] (Appendix 14) the vehicle is a bus, the information is video information of the interior of the vehicle, The analysis means analyzes passengers in the vehicle and their postures as a state of the vehicle, the identifying means identifies the control policy as a remote control policy when the passenger is a person prone to falling or when the posture is a posture prone to falling. 12. The control system described in claim 11.
[0104] (Appendix 15) Analyzing a state of a vehicle having an automatic driving function based on information about the vehicle; Identifying a control policy, which is a predetermined control to be transmitted to the vehicle, based on the analyzed state of the vehicle; Controlling operation of the vehicle based on the identified control policy. Control method.
[0105] (Appendix 16) When analyzing the state of the vehicle, learning an analysis rule for analyzing a state of the vehicle based on the information; When identifying the control policy, identifying the control policy based on the state of the vehicle analyzed by the analysis rule; The control method described in Appendix 15.
[0106] (Appendix 17) A vehicle-side learning means provided in the vehicle learns an automatic driving rule for controlling the automatic driving based on the information; When identifying the control policy, Identifying the control policy based on the state of the vehicle analyzed by the analysis rule and the autonomous driving rule. The control method described in Appendix 16.
[0107] (Appendix 18) the control policy includes an automatic control policy for causing the vehicle to perform automatic driving and a remote control policy for remotely controlling the vehicle, When controlling the operation of the vehicle, When the identified control policy is the automatic control policy, the vehicle is controlled to be automatically driven, and when the identified control policy is the remote control policy, the vehicle is controlled to be remotely controlled. A control method according to any one of appendix 15 to 17.
[0108] (Appendix 19) the information is image information of a scene ahead of the vehicle, When analyzing the state of the vehicle, Analyzing a risk of collision with a recognized surrounding vehicle or person in a traffic environment including the recognized road and intersection as a state of the vehicle; When identifying the control policy, If the risk level is equal to or greater than a threshold, the control policy is identified as a remote control policy. The control method described in Appendix 18.
[0109] (Appendix 20) the information is video information of the interior of the vehicle, When analyzing the state of the vehicle, Analyzing the state of the vehicle based on the behavior of the passengers in the vehicle and the urgency of the state of the passengers; When identifying the control policy, If the urgency is equal to or greater than a threshold, the control policy is identified as a remote control policy. The control method described in Appendix 18.
[0110] (Appendix 21) the vehicle is a bus, the information is video information of the interior of the vehicle, When analyzing the state of the vehicle, Analyzing passengers in the vehicle and their postures as a state of the vehicle; When identifying the control policy, specifying the control policy as a remote control policy when the passenger is a person prone to falling or when the posture is a posture prone to falling; The control method described in Appendix 18.
[0111] (Appendix 22) Analyzing a state of a vehicle having an automatic driving function based on information about the vehicle; Identifying a control policy, which is a predetermined control to be transmitted to the vehicle, based on the analyzed state of the vehicle; Controlling operation of the vehicle based on the identified control policy. A non-transitory computer-readable medium on which a control program for causing a computer to execute the following is stored.
[0112] (Appendix 23) When analyzing the state of the vehicle, learning an analysis rule for analyzing a state of the vehicle based on the information; When the control policy is specified, determining the control policy based on the state of the vehicle analyzed by the analysis rule; A non-transitory computer-readable medium having stored thereon a control program according to claim 22 that causes a computer to execute the control program.
[0113] (Appendix 24) A vehicle-side learning means provided in the vehicle learns an automatic driving rule for controlling the automatic driving based on the information; When the control policy is specified, identifying the control policy based on the state of the vehicle analyzed by the analysis rule and the autonomous driving rule; A non-transitory computer-readable medium having stored thereon a control program according to claim 23 that causes a computer to execute the control program.
[0114] (Appendix 25) the control policy includes an automatic control policy for causing the vehicle to perform automatic driving and a remote control policy for remotely controlling the vehicle, When controlling the operation of the vehicle, When the identified control policy is the automatic control policy, the vehicle is controlled to be automatically driven, and when the identified control policy is the remote control policy, the vehicle is controlled to be remotely controlled. A non-transitory computer-readable medium storing a control program according to any one of appendices 22 to 24, which causes a computer to execute the control program.
[0115] (Appendix 26) the information is image information of a scene ahead of the vehicle, When analyzing the state of the vehicle, Analyzing a risk of collision with a recognized surrounding vehicle or person in a traffic environment including the recognized road and intersection as a state of the vehicle; When the control policy is specified, When the risk level is equal to or greater than a threshold, the control policy is identified as a remote control policy. A non-transitory computer-readable medium having stored thereon a control program according to claim 25 that causes a computer to execute the control program.
[0116] (Appendix 27) the information is video information of the interior of the vehicle, When analyzing the state of the vehicle, Analyzing the behavior of passengers in the vehicle and the urgency of the passengers' conditions as the state of the vehicle; When the control policy is specified, When the urgency is equal to or greater than a threshold, the control policy is identified as a remote control policy. A non-transitory computer-readable medium having stored thereon a control program according to claim 25 that causes a computer to execute the control program.
[0117] (Appendix 28) the vehicle is a bus, the information is video information of the interior of the vehicle, When analyzing the state of the vehicle, Analyzing passengers in the vehicle and their postures as a state of the vehicle; When the control policy is specified, When the passenger is a person prone to falling or when the posture is a posture prone to falling, the control policy is specified as a remote control policy. A non-transitory computer-readable medium having stored thereon a control program according to claim 25 that causes a computer to execute the control program.
[0118] In the above example, the program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electric signals, optical signals, and electromagnetic waves. The transitory computer readable medium can provide the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path. [Explanation of symbols]
[0119] 100, 200 Control System 110, 210 vehicles 111, 211 Sensor 112, 212 Vehicle control unit 120, 220 control device 120c, 220c Control Center 121, 221 Analysis Department 122, 222 Specific part 123, 223 Remote control unit 213, 224 Learner NW Network
Claims
1. an analysis means for analyzing a first state of the vehicle according to an analysis rule for analyzing a state of the vehicle based on information about the vehicle having an automatic driving function; An identification means for identifying a control policy regarding control of the vehicle by using a second state of the vehicle analyzed by an automatic driving rule that controls automatic driving of the vehicle based on the information and the first state of the vehicle; a remote control means for controlling the operation of the vehicle based on the identified control policy; Equipped with the control policy includes an automatic control policy for causing the vehicle to perform automatic driving and a remote control policy for remotely controlling the vehicle, The remote control means controls the vehicle to automatically drive when the identified control policy is the automatic control policy, and controls the vehicle to remotely control when the identified control policy is the remote control policy. Control device.
2. the analysis means includes a control-side learning means for learning the analysis rule; The control device according to claim 1.
3. The automated driving rule is learned by a vehicle-side learning means provided in the vehicle.
3. The control device according to claim 1 or 2.
4. the information is image information of a scene ahead of the vehicle, the analysis means analyzes a risk of collision with a recognized surrounding vehicle or person in a traffic environment including the recognized road and intersection as a state of the vehicle; the identifying means identifies the control policy as a remote control policy when the degree of risk is equal to or greater than a threshold. The control device according to any one of claims 1 to 3.
5. the information is video information of the interior of the vehicle, the analyzing means analyzes the recognized motion of the passenger in the vehicle and the urgency of the state of the passenger as a state of the vehicle; the identifying means identifies the control policy as a remote control policy when the urgency is equal to or greater than a threshold value; The control device according to any one of claims 1 to 3.
6. the vehicle is a bus, the information is video information of the interior of the vehicle, The analysis means analyzes passengers in the vehicle and their postures as a state of the vehicle, the identifying means identifies the control policy as a remote control policy when the passenger is a person prone to falling or when the posture is a posture prone to falling. The control device according to any one of claims 1 to 3.
7. A vehicle having an autonomous driving function; A control center that remotely controls the vehicle; Equipped with The vehicle is means for analyzing a second state of the vehicle according to an automatic driving rule for controlling automatic driving of the vehicle based on information about the vehicle; A vehicle control means for controlling the automatic driving of the vehicle based on the information and the automatic driving rule; having The control center includes: an analysis means for analyzing a first state of the vehicle according to an analysis rule for analyzing a state of the vehicle based on the information received from the vehicle; a determination means for determining a control policy regarding control of the vehicle by using the second state received from the vehicle and the first state; a remote control means for controlling the operation of the vehicle based on the identified control policy; having Control system.
8. Analyzing a first state of the vehicle according to an analysis rule for analyzing a state of the vehicle based on information about the vehicle having an autonomous driving function; Identifying a control policy regarding control of the vehicle using a second state of the vehicle analyzed by an automatic driving rule that controls automatic driving of the vehicle based on the information in the vehicle and the first state; Controlling operation of the vehicle based on the identified control policy; the control policy includes an automatic control policy for causing the vehicle to perform automatic driving and a remote control policy for remotely controlling the vehicle, When controlling the driving of the vehicle based on the specified control policy, if the specified control policy is the automatic control policy, the vehicle is controlled to be automatically driven, and if the specified control policy is the remote control policy, the vehicle is controlled to be remotely controlled. Control method.
9. analyzing a first state of the vehicle according to an analysis rule for analyzing a state of the vehicle based on information about the vehicle having an autonomous driving function; Identifying a control policy regarding control of the vehicle using a second state of the vehicle analyzed by an automatic driving rule that controls automatic driving of the vehicle based on the information and the first state of the vehicle; Controlling operation of the vehicle based on the identified control policy. A control program for causing a computer to execute the above steps, the control policy includes an automatic control policy for causing the vehicle to perform automatic driving and a remote control policy for remotely controlling the vehicle, When controlling the driving of the vehicle based on the specified control policy, if the specified control policy is the automatic control policy, the vehicle is controlled to be automatically driven, and if the specified control policy is the remote control policy, the vehicle is controlled to be remotely controlled. Control program.
Citation Information
Patent Citations
Support system for rescuing vehicle occupant
JP2016057673A
Management system, management method, and management program
JP2018142265A
Self-driving car and control program
JP2018155577A
Automatic driving control system, automatic driving control method, and vehicle
JP2019106674A
Control method and control device for automatic drive vehicle, and computer readable storage medium
JP2019182425A
Cited By
Control device, control method, and control program
WO2026181145A1
