Safety rule generation device, moving body, autonomous driving system, safety rule generation method, and program

The safety rule generation device uses a hybrid control flow graph to integrate and synchronize control flows for multiple vehicles, addressing the challenge of complex scenarios by automatically generating safety rules that ensure safe operation in autonomous driving systems.

WO2024247847A1PCT designated stage expired Publication Date: 2024-12-05INTER UNIV RES INST RES ORG OF INFORMATION & SYST
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/018853
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2024-05-22
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional safety rule formulation technologies for autonomous driving struggle to handle complex scenarios involving multiple interacting vehicles, as they require manual assignment of safety rules to numerous conditional branches, making it impractical to cover all conditions.

Method used

A safety rule generation device that integrates control flows using a hybrid control flow graph (hCFG) to model state transitions of multiple vehicles, synchronizing interactions and assigning logical expression constraints to ensure safety, thereby generating safety rules that can handle complex scenarios automatically.

Benefits of technology

Enables the efficient formulation of safety rules for complex scenarios, ensuring safety and reducing the complexity of manually assigning rules, allowing autonomous vehicles to operate safely in diverse conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024018853_05122024_PF_FP_ABST
    Figure JP2024018853_05122024_PF_FP_ABST
Patent Text Reader

Abstract

This safety rule generation device comprises: a control flow input unit that receives the input of a plurality of control flows indicating the respective state transitions of a plurality of moving bodies operated under a prescribed scenario; a control flow integration unit that, on the basis of transition conditions assigned to the control flows, generates an integrated control flow in which the plurality of control flows are integrated; a constraint assignment unit that assigns, to each of the states included in the integrated control flow, a logical constraint whereby safety is guaranteed; and a safety rule generation unit that generates a safety rule on the basis of the logical constraint.
Need to check novelty before this filing date? Find Prior Art

Description

Safety rule generation device, mobile body, automated driving system, safety rule generation method and program

[0001] The present invention relates to a safety rule generation device, a mobile body, an automated driving system, a safety rule generation method, and a program.

[0002] There are technologies for formulating safety rules for automated driving. Safety rule formulation technologies are technologies for formulating safety rules that can be mathematically proven to be safe. For example, Non-Patent Document 1 discloses a mathematical model called Responsibility-Sensitive Safety (RSS). Also, for example, Non-Patent Document 2 discloses Goal-Aware RSS (GA-RSS), which is an extension of Responsibility-Sensitive Safety and ensures the achievement of a specified goal.

[0003] Shai Shalev-Shwartz, Shaked Shammah, and Amnon Shashua, "On a formal model of safe and scalable self-driving cars," CoRR, abs / 1708.06374, 2017. I. Hasuo, C. Eberhart, J. Haydon, J. Dubut, R. Bohrer, T. Kobayashi, S. Pruekprasert, X. Zhang, EA Pallas, A. Yamada, K. Suenaga, F. Ishikawa, K. Kamijo, Y. Shinya, and T. Suetomi, "Goal-aware RSS for complex scenarios via program logic," IEEE Transactions on Intelligent Vehicles, 2023, CoRR abs / 2207.02387.

[0004] However, with conventional technology, it has been difficult to formulate safety rules that can handle complex scenarios. For example, with conventional technology, complex scenarios have to be divided into simple sub-scenarios, and safety rules have to be assigned to each sub-scenario manually. In complex scenarios in which multiple vehicles interact, the number of conditional branches becomes enormous, making it unrealistic to manually formulate safety rules that cover all conditions.

[0005] In view of the above technical challenges, one aspect of the present invention aims to easily formulate safety rules that can handle complex scenarios.

[0006] In order to solve the above problems, a safety rule generation device according to one aspect of the present invention comprises a control flow input unit configured to accept input of multiple control flows indicating the state transitions of each of multiple moving bodies operating in a specified scenario, a control flow integration unit configured to generate an integrated control flow that integrates the multiple control flows based on transition conditions assigned to the control flows, a constraint assignment unit configured to assign logical expression constraints with guaranteed safety to each state included in the integrated control flow, and a safety rule generation unit configured to generate safety rules based on the logical expression constraints.

[0007] According to one aspect of the present invention, safety rules that can handle complex scenarios can be easily formulated.

[0008] FIG. 1 is a diagram showing an example of an intersection scenario. FIG. 2 is a diagram showing an example of a control flow graph. FIG. 3 is a diagram showing an example of a hybrid control flow graph. FIG. 4 is a diagram showing an example of a hybrid control flow graph. FIG. 5 is a block diagram showing an example of the overall configuration of an autonomous driving system. FIG. 6 is a block diagram showing an example of the hardware configuration of an autonomous driving system. FIG. 7 is a block diagram showing an example of the functional configuration of an autonomous driving system. FIG. 8 is a flowchart showing an example of an autonomous driving method. FIG. 9 is a diagram showing an example of experimental results.

[0009] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0010] One embodiment of the present invention is an autonomous driving system for safely moving an autonomously driven mobile object. In the autonomous driving system of this embodiment, a safety rule generation device generates safety rules that can handle complex scenarios in which multiple vehicles interact, and the mobile object autonomously performs driving operations in accordance with the safety rules.

[0011] Examples of the moving body in this embodiment include an autonomous driving vehicle, a route bus with a fixed driving route, an unmanned aerial vehicle such as a drone, an autonomous traveling robot such as a transport robot used at a manufacturing site or a logistics site, or a household robot used at home or an office, etc. Hereinafter, in this embodiment, an autonomous driving system that safely moves an autonomous driving vehicle will be described as an example.

[0012] <Autonomous Driving Safety Rules> Ensuring the safety of autonomous driving is an important issue for industry and society. In recent years, the "formulation of logical safety rules" has been attracting particular attention. The goal of "formulation of logical safety rules" is to formulate safety rules for each of the various driving scenarios that guarantee "safety as long as these rules are followed (and safety can be mathematically and logically proven)."

[0013] Logical safety rules are important for the following reasons: First, they are useful for improving the safety of autonomous vehicles. Second, they make it possible to explain the safety of autonomous vehicles to society in an easy-to-understand manner. Third, by dividing the scope of responsibility, they reduce the risk of manufacturers incurring unexpected or excessive manufacturer liability, and promote the development of autonomous driving as an industry.

[0014] A conventional technology for formulating safety rules for automated driving is the Responsibility Sensing Safety (RSS) theory, which is disclosed in Non-Patent Document 1. RSS is widely recognized as a safety rule for automated driving. For example, RSS is used in many academic studies, and its incorporation into international standards is being considered.

[0015] RSS is constructed with the following logical structure. That is, the logic is that "if the preconditions are met, the safety conditions can be met by executing the control strategy (proper response)." A control strategy is a method for controlling an autonomous vehicle. An example of a control strategy is a driving operation such as turning the steering wheel, applying the brakes, or accelerating by stepping on the accelerator.

[0016] The conventional RSS disclosed in Non-Patent Document 1 cannot handle complex scenarios. For example, the conventional RSS can only handle a simple scenario in which an autonomous vehicle avoids a collision with another vehicle traveling in the same lane. In other words, the conventional RSS only guarantees the safety condition of collision avoidance. Hereinafter, the conventional RSS will be referred to as a "Collision Avoiding RSS (CA-RSS)."

[0017] Non-Patent Document 2 proposes a Goal Aware RSS (GA-RSS), which is an extension of the Collision Avoidance RSS and guarantees the achievement of a predetermined goal. The Goal Aware RSS is constructed with the following logical structure: "If the preconditions are satisfied, the execution of a control strategy will enable the achievement of predetermined postconditions while also satisfying safety conditions."

[0018] The goal-achievement RSS can guarantee that a goal, such as "repeatedly change lanes while avoiding collisions and stopping at a specific position on the shoulder of the road," will be achieved safely. Therefore, the goal-achievement RSS can formulate safety rules that can handle complex scenarios.

[0019] In the Objective Achievement RSS, safety rules that guarantee safety are formulated using the following procedure. First, scenarios are created that correspond to the movements of the vehicle itself and other vehicles in the vicinity. Next, safety rules are set for each of the created scenarios. Finally, the safety of each set safety rule is logically proven.

[0020] For example, consider a scenario in which the vehicle turns right at an intersection. Hereinafter, this scenario will be referred to as an “intersection scenario.” Figure 1 is a diagram for explaining an example of an intersection scenario.

[0021] As shown in Figure 1, the intersection scenario is a scenario in which a subject vehicle (SV) turns right at an intersection. Figure 1 shows that a principal other vehicle (POV) is approaching in the oncoming lane. The subject vehicle (SV) needs to turn right at the intersection while avoiding a collision with the oncoming POV.

[0022] In an intersection scenario, first, the behavior of the oncoming vehicle POV and the behavior of the host vehicle SV are combined to create a scenario. At this time, the following cases must be considered. First, whether the oncoming vehicle POV passes through the intersection before the host vehicle SV or after the host vehicle SV. Second, when the host vehicle SV or the oncoming vehicle POV brakes to stop, whether they stop before a collision zone (CZ) in the intersection, stop within the collision zone, or stop after passing through the collision zone.

[0023] In the Goal Achievement RSS, after creating a scenario like the one above, it is necessary to comprehensively determine all possible cases and set safety rules for each. Therefore, the Goal Achievement RSS has the problem that it is not realistic to manually formulate safety rules for complex scenarios with a huge number of case cases.

[0024] The present embodiment aims to easily formulate safety rules that can handle complex scenarios with a huge number of case distinctions. According to one aspect, the present embodiment makes it possible to efficiently develop autonomous vehicles that can handle various scenarios, including complex scenarios.

[0025] <Hybrid Control Flow Graph (hCFG)> In this embodiment, to formulate safety rules that can handle complex scenarios in which multiple vehicles interact, the behavior of each vehicle is modeled using a hybrid control flow graph (hCFG). The hCFG is an extension of the control flow graph (CFG) using flow dynamics. The CFG is a graphical representation of all possible paths that a program may take when it is executed.

[0026] 2 is a diagram illustrating an example of a control flow graph. As shown in FIG. 2, the CFG includes multiple locations l, l′. l,1 , ..., l' l,nl , where location l and each location l′ l,1 , ..., l' l,nl There is a directed edge e between l,1 , ..., e l,nl where location l represents the basic location. l,i (i=1,...,n l ) is the distance from location l to other location l l,i means to jump to

[0027] In the hCFG, each directed edge e l,i Guard A l,i =Guard(e l,i ) is assigned. Guard A is a mechanism that has the same meaning as a conditional expression or conditional branch in a programming language. Guard A is an expression that must evaluate to true in order for processing to continue in a branch. If guard A evaluates to false, processing proceeds to another branch that evaluates to true.

[0028] Guard A is a guard A at location l. l,iIf it is not satisfied, the process stops at location l and any guard A l,i If this is satisfied, then the location l to the ith location l l,i means to jump to

[0029] hCFG is defined by formula (1).

[0030]

[0031] where L is a finite set of locations. Σ is a finite set of event names. E ⊆ L × Σ × L is a finite set of directed edges representing discrete jumps. V is a finite set of real-valued variables. Flow is a function that associates a system of ordinary differential equations (ODEs) x = f with each location l ∈ L. Guard is a function that associates a guard A with each directed edge e ∈ E. Assign is a function that associates a command of the form x:= f with each directed edge e ∈ E and a label Init indicating initialization. l Init ∈L is the initial location. Final∈L is the set of final locations.

[0032] 3 and 4 are diagrams showing examples of hybrid control flow graphs. Of these, Fig. 3 is a hybrid control flow graph showing state transitions of the host vehicle SV in an intersection scenario. Also, Fig. 4 is a hybrid control flow graph showing state transitions of an oncoming vehicle POV in an intersection scenario.

[0033] 3 and 4 are based on the assumption that the host vehicle SV is an autonomous vehicle. Therefore, for example, data such as the speed and position of the oncoming vehicle POV when turning right at an intersection can be acquired by the host vehicle SV. In other words, the host vehicle SV is configured to recognize the state and behavior of vehicles around the host vehicle SV. Regarding prediction of the control strategy for the oncoming vehicle POV, a method is employed that limits the possible behavior of the oncoming vehicle POV under the assumption that the oncoming vehicle POV is driving in accordance with the laws of physics, traffic rules, or driving practices, just as when a human drives a car.

[0034] As shown in Figures 3 and 4, in this embodiment, the state transition of each vehicle is modeled for each state type. Examples of state types include the position of the vehicle, the velocity of the vehicle, and the response time of the vehicle. For example, in the G SVPos is a hCFG indicating the state transition of the position of the host vehicle SV. POVVel is the hCFG that indicates the state transition of the speed of the oncoming vehicle POV.

[0035] 3 and 4, rectangular boxes represent locations l, and arrows connecting locations l represent directed edges e. Labels and logical expressions shown near directed edges e represent guards A. Labels shown on guards A are event names. Event names are shared by each hCFG, and locations l with the same event name for guards A indicate that vehicles interact with each other.

[0036] When the behavior of an autonomous vehicle is modeled using an hCFG, location l corresponds to a state of the autonomous vehicle. Furthermore, directed edge e corresponds to a state transition of the autonomous vehicle. Furthermore, guard A corresponds to a transition condition for a state transition to occur.

[0037] <Synchronized product of hCFG> When there are multiple hCFGs, they can be integrated into an hCFG network. Hereinafter, the operation of integrating hCFG networks will be referred to as "synchronized product."

[0038] The hCFG network is defined by equation (2).

[0039]

[0040] However, G (1) , ..., G (k) are open hCFGs. Open hCFGs are defined similarly to hCFGs, except that Flow(l) does not cover all real-valued variables and the final location is not fixed.

[0041] G included in the hCFG network (1), ..., G (k) has the following characteristics: (1) , ..., L (k) are separated. They share the same variable set V. They satisfy the predetermined compatibility conditions. The predetermined compatibility conditions include the following two compatibility conditions: The first compatibility condition is that each location L (1) , ..., L (k) The second compatibility condition is that the flow dynamics of each edge e (1) , ..., e (k) The allocation commands do not conflict.

[0042] The synchronous product G of the hCFG network N N,Final is the hCFG defined as follows: N,Final Location (1) , ..., l (k) where l (i) ∈L (i) Synchronous product G N,Final The set of effective edges E N,Final is G (1) , ..., G (k) The union E of directed edges with event names (1) ∪...∪E (k) Synchronous product G N,Final The variable set V of G (1) , ..., G (k) The synchronous product G N,Final is an event a∈E (i) occurs at location l (i) From location l' (i) otherwise, it transitions to location l (i) A directed edge (l (1) , ..., l (k) ) → a (l' (1) , ..., l' (k) ) That is, all hCFGs that share event a synchronize on event a. N,Final The flow dynamics of each location l (1) , ..., l (k)The above compatibility conditions ensure that this union is well-defined and covers all variables in the variable set V. (1) , ..., l (k) ) → a (l' (1) , ..., l' (k) ) guards each directed edge l (i) → a l' (i) Guard A associated with (i) It is a combination of the following. N,Final The assignment command for a directed edge of is the union of the directed edges of each hCFG. Due to the compatibility conditions above, assignments cannot be overwritten. The initial assignment command is defined similarly. N,Final The initial location of Init Location (1) Init , ..., l (k) Init It is a tuple consisting of: N,Final The set of final locations of L is specified separately. (1) ×...×L (k) is a set of locations defined by

[0043] <Hoare Annotation> In this embodiment, to guarantee the safety of a safety rule, a Hoare annotation γ is assigned to each location included in the synchronous product of the hCFG that models the scenario. The Hoare annotation γ is an example of a logical expression constraint.

[0044] The Hoare annotation γ is derived from the Hoare quadruples used for safety verification in the Goal Achievement RSS (see Non-Patent Document 2). The Hoare annotation γ includes the precondition, control strategy, and postcondition that constitute the Hoare quadruple.

[0045] The preconditions, control strategies, and postconditions included in the Hoare annotation γ are defined to satisfy a predetermined safety condition. Specifically, the Hoare annotation γ is defined such that the precondition is true, the postcondition is true while the control strategy is being executed, and the predetermined safety condition is true while the control strategy is being executed.

[0046] The Hoare annotation γ is the synchronous product of the hCFG from the final location Final to the initial location l Init The Hoare annotation γ that can be assigned to the final location Final is searched for, and if an assignable Hoare annotation γ exists, a Hoare annotation γ that can be assigned to the previous location is searched for. This is repeated until the initial location l Init If the Hoare annotation γ can be assigned to each of the above, safety rules can be formulated to complete the scenario while satisfying the specified safety conditions.

[0047] <Overall Configuration of Autonomous Driving System> The overall configuration of the autonomous driving system in this embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of the overall configuration of the autonomous driving system in this embodiment.

[0048] 5, the autonomous driving system 1 in this embodiment includes a safety rule generation device 10, a mobile body 20, and a user terminal 30. The safety rule generation device 10, the mobile body 20, and the user terminal 30 are connected to each other so as to be able to communicate data with each other via a communication network N1 such as a local area network (LAN) or the Internet.

[0049] The safety rule generation device 10 is an information processing device such as a personal computer, workstation, or server that generates safety rules in response to a request from a user terminal 30. The safety rule generation device 10 receives control flow information from the user terminal 30. The control flow information is information that describes the state transitions of multiple vehicles operating in a scenario for which safety rules are to be generated. In this embodiment, the control flow information is described in hCFG. The safety rule generation device 10 generates safety rules based on the control flow information and outputs the safety rule information. The safety rule information is information that indicates the safety rules for the scenario.

[0050] The mobile body 20 is a mobile body having an automatic driving function. The mobile body 20 stores in advance safety rule information output by the safety rule generation device 10. The mobile body 20 autonomously performs driving operations in accordance with the safety rules indicated in the safety rule information. In this embodiment, an example of the mobile body 20 is an automatically driven vehicle.

[0051] The user terminal 30 is an information processing terminal such as a personal computer, tablet terminal, or smartphone operated by a user of the autonomous driving system 1. The user terminal 30 accepts input of a scenario for which a safety rule is to be generated in response to a user's operation, and transmits control flow information created based on the scenario to the safety rule generation device 10. The user terminal 30 may be incorporated into an on-board device mounted on the mobile object 20. An example of an on-board device is a car navigation system.

[0052] 5 is merely an example, and various system configurations are possible depending on the application and purpose. For example, the safety rule generation device 10 may be implemented using multiple computers, or may be implemented as a cloud computing service. Furthermore, for example, the autonomous driving system 1 may be implemented by installing, in the mobile object 20, an in-vehicle device that combines the functions that the mobile object 20 and the user terminal 30 should each have.

[0053] <Hardware Configuration of Autonomous Driving System> The hardware configuration of each device included in the autonomous driving system 1 in this embodiment will be described with reference to FIG. 6 .

[0054] <Hardware Configuration of Computer> The safety rule generation device 10, the in-vehicle device mounted on the moving body 20, and the user terminal 30 in this embodiment are realized by, for example, a computer. Fig. 6 is a block diagram showing an example of the hardware configuration of the computer in this embodiment.

[0055] 6 , a computer 500 in this embodiment includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a HDD (Hard Disk Drive) 504, an input device 505, a display device 506, a communication I / F (Interface) 507, and an external I / F 508. The CPU 501, the ROM 502, and the RAM 503 form a so-called computer. The hardware components of the computer 500 are connected to each other via a bus line 509. The input device 505 and the display device 506 may be connected to the external I / F 508 for use.

[0056] The CPU 501 is a computing device that reads programs and data from a storage device such as the ROM 502 or the HDD 504 onto the RAM 503 and executes processing to realize overall control and functions of the computer 500 .

[0057] The ROM 502 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 502 functions as a main storage device that stores various programs, data, etc. required for the CPU 501 to execute various programs installed in the HDD 504. Specifically, the ROM 502 stores boot programs such as a Basic Input / Output System (BIOS) and an Extensible Firmware Interface (EFI) that are executed when the computer 500 starts up, as well as data such as OS (Operating System) settings and network settings.

[0058] The RAM 503 is an example of a volatile semiconductor memory (storage device) in which programs and data are erased when the power is turned off. The RAM 503 is, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM). The RAM 503 provides a working area in which various programs installed in the HDD 504 are expanded when executed by the CPU 501.

[0059] The HDD 504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in the HDD 504 include an OS, which is basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that the computer 500 may use a storage device that uses flash memory as a storage medium (e.g., an SSD (Solid State Drive)) instead of the HDD 504.

[0060] The input device 505 includes a touch panel, operation keys and buttons, a keyboard and mouse, a microphone for inputting sound data such as voice, and the like, which are used by the user to input various signals.

[0061] The display device 506 is composed of a display such as a liquid crystal display or organic electroluminescence (EL) display for displaying a screen, a speaker for outputting sound data such as voice, and the like.

[0062] The communication I / F 507 is an interface that connects to a communication network and enables the computer 500 to perform data communication.

[0063] The external I / F 508 is an interface with external devices, such as a drive device 510.

[0064] The drive device 510 is a device for loading a recording medium 511. The recording medium 511 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 511 may also include semiconductor memories that record information electrically, such as ROMs and flash memories. This allows the computer 500 to read from and / or write to the recording medium 511 via the external I / F 508.

[0065] The various programs to be installed in the HDD 504 are installed, for example, by setting the distributed recording medium 511 in a drive device 510 connected to the external I / F 508 and reading the various programs recorded on the recording medium 511 by the drive device 510. Alternatively, the various programs to be installed in the HDD 504 may be installed by being downloaded via the communication I / F 507 from a network different from the communication network.

[0066] <Functional Configuration of Autonomous Driving System> The functional configuration of the autonomous driving system in this embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of the functional configuration of the autonomous driving system 1 in this embodiment.

[0067] <User Terminal> As shown in FIG. 7, the user terminal 30 in this embodiment includes a scenario input unit 301, a control flow creation unit 302, and a control flow transmission unit 303.

[0068] The scenario input unit 301, the control flow creation unit 302, and the control flow transmission unit 303 are realized by, for example, processing that is executed by the CPU 501 according to a program loaded from the HDD 504 onto the RAM 503 shown in FIG.

[0069] The scenario input unit 301 receives input of a scenario for which safety rules are to be generated in response to a user operation. In this embodiment, a scenario in which multiple vehicles including the moving body 20 operate is the target for which safety rules are to be generated.

[0070] The control flow creation unit 302 creates a control flow based on a scenario input to the scenario input unit 301 in response to a user operation. The control flow creation unit 302 models the operation of each vehicle operating in the scenario using an hCFG. Therefore, the control flow created by the control flow creation unit 302 indicates the state transition of each vehicle operating in the scenario.

[0071] The control flow creation unit 302 creates a control flow for each type of state of each vehicle. For example, in a scenario for an autonomous vehicle, the types of states include the vehicle position, vehicle speed, and reaction speed. The types of states are not limited to these, and may include any state related to the safety of the moving body 20.

[0072] The control flow sending unit 303 sends control flow information indicating the control flow created by the control flow creating unit 302 to the safety rule generation device 10. The control flow information includes a plurality of control flows corresponding to the states of each vehicle operating in the scenario.

[0073] <Safety Rule Generation Device> As shown in FIG. 7 , the safety rule generation device 10 in this embodiment includes a control flow input unit 101, a control flow integration unit 102, a constraint assignment unit 103, a safety rule generation unit 104, and a safety rule output unit 105.

[0074] The control flow input unit 101, the control flow integration unit 102, the constraint assignment unit 103, the safety rule generation unit 104, and the safety rule output unit 105 are realized by the processing executed by the CPU 501 of a program loaded from the HDD 504 onto the RAM 503 shown in FIG. 6.

[0075] The control flow input unit 101 accepts input of control flow information received from the user terminal 30 .

[0076] The control flow integration unit 102 integrates multiple control flows included in the control flow information received by the control flow input unit 101. Specifically, the control flow integration unit 102 calculates the synchronous product of the multiple control flows included in the control flow information. As a result, the control flow integration unit 102 obtains an integrated control flow that integrates the multiple control flows.

[0077] In other words, the control flow integrator 102 integrates the multiple control flows based on the transition conditions under which state transitions occur in each control flow. Specifically, the control flow integrator 102 integrates the multiple control flows by assigning transition conditions such that interactions between the vehicles are synchronized. The interactions may include causal interactions or temporal interactions.

[0078] The control flow integrator 102 synchronizes interactions between vehicles based on event names described in the control flow. That is, if there are directed edges containing the same event name, the control flow integrator 102 assigns guards to each directed edge so that the events occur at the same time.

[0079] The constraint assignment unit 103 assigns a logical expression constraint to each state included in the integrated control flow generated by the control flow integration unit 102. The constraint assignment unit 103 calculates the logical expression constraint so that a predetermined safety condition is satisfied in each state. The constraint assignment unit 103 assigns a logical expression constraint to each state by going back from the final state of the integrated control flow to the initial state. In this embodiment, the logical expression constraint is a Hoare annotation γ.

[0080] When the constraint assignment unit 103 assigns logical expression constraints to all states of the integrated control flow, the safety rule generation unit 104 generates safety rules based on the logical expression constraints assigned to the integrated control flow. The safety rule generation unit 104 combines the preconditions and control strategies included in the logical expression constraints assigned to each state in the order shown in the integrated control flow. This generates safety rules in which the preconditions and control strategies are combined.

[0081] The safety rule output unit 105 generates safety rule information indicating the safety rules generated by the safety rule generation unit 104. The safety rule output unit 105 transmits the generated safety rule information to the mobile object 20.

[0082] <<Mobile Body>> As shown in FIG. 7 , the mobile body 20 in this embodiment includes a safety rule receiving unit 201 , a safety rule storage unit 202 , and a driving control unit 203 .

[0083] The safety rule receiving unit 201 and the operation control unit 203 are realized, for example, by a program loaded from the HDD 504 shown in Fig. 6 onto the RAM 503, which is executed by the CPU 501. The safety rule storage unit 202 is realized, for example, by using the HDD 504 shown in Fig. 6.

[0084] The safety rule receiving unit 201 receives input of safety rule information. The safety rule receiving unit 201 stores the received safety rule information in the safety rule storage unit 202. The safety rule receiving unit 201 may receive the safety rule information by receiving it from the safety rule generation device 10. The safety rule receiving unit 201 may receive the safety rule information from the input device 505 or the like in response to a user operation.

[0085] Safety rule information is stored in the safety rule storage unit 202. The safety rule information includes the safety rules generated by the safety rule generation device 10.

[0086] The driving control unit 203 executes driving operations to accomplish a predetermined scenario based on the safety rule information read from the safety rule storage unit 202. This realizes automatic driving in which the moving body 20 accomplishes a predetermined scenario while satisfying predetermined safety conditions.

[0087] <Processing Procedure of Autonomous Driving System> The processing procedure of the autonomous driving method executed by the autonomous driving system 1 in this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the processing procedure of the autonomous driving method in this embodiment.

[0088] In step S1, the scenario input unit 301 of the user terminal 30 accepts input of a scenario for which safety rules are to be generated in response to a user operation. The scenario input unit 301 sends scenario information indicating the input scenario to the control flow creation unit 302.

[0089] In step S2, the control flow creation unit 302 of the user terminal 30 receives scenario information from the scenario input unit 301. Next, in response to user operations, the control flow creation unit 302 creates a control flow indicating the state transitions of each vehicle operating according to the scenario indicated in the scenario information. The control flow creation unit 302 creates a control flow for each type of state for each vehicle. Then, the control flow creation unit 302 sends the created multiple control flows to the control flow transmission unit 303.

[0090] Specifically, the control flow creation unit 302 models the state transitions of each vehicle using an hCFG. First, for each vehicle, the control flow creation unit 302 generates locations that indicate the state of the vehicle and connects the locations with directed edges that indicate state transitions. Next, the control flow creation unit 302 assigns guards that indicate transition conditions to the directed edges. This generates a control flow modeled using an hCFG.

[0091] In step S3, the control flow transmission unit 303 of the user terminal 30 receives the control flows from the control flow creation unit 302. Next, the control flow transmission unit 303 generates control flow information including the control flows. Then, the control flow transmission unit 303 transmits the control flow information to the safety rule generation device 10.

[0092] In step S4, the control flow input unit 101 of the safety rule generation device 10 receives the control flow information from the user terminal 30. Next, the control flow input unit 101 accepts the input of the received control flow information. Subsequently, the control flow input unit 101 sends the input control flow information to the control flow integration unit 102.

[0093] In step S5, the control flow integration unit 102 of the safety rule generation device 10 receives control flow information from the control flow input unit 101. Next, the control flow integration unit 102 integrates multiple control flows included in the control flow information. Subsequently, the control flow integration unit 102 sends the integrated control flow obtained by integrating the multiple control flows to the constraint assignment unit 103.

[0094] Specifically, the control flow integration unit 102 integrates multiple control flows by assigning transition conditions so that causal interactions or temporal interactions between each vehicle are synchronized. The control flow integration unit 102 references the event names of guards assigned to directed edges of each control flow (hCFG), and if there are directed edges including the same event name, assigns a guard to each directed edge so that these events occur at the same timing.

[0095] In step S6, the constraint assignment unit 103 of the safety rule generation device 10 receives the integrated control flow from the control flow integration unit 102. Next, the constraint assignment unit 103 assigns logical expression constraints to each state included in the integrated control flow. Specifically, the constraint assignment unit 103 assigns logical expression constraints to each state included in the integrated control flow (hCFG) from the final location Final to the initial location l Init We assign a Hoare annotation γ to each location l by tracing the directed edges e towards .

[0096] More specifically, the constraint assignment unit 103 calculates a precondition that satisfies a predetermined safety condition, using a location to be assigned (referred to as a "target location") as a postcondition and a directed edge connected to the target location as a control strategy. Next, the constraint assignment unit 103 determines whether the location immediately before the directed edge satisfies the precondition.

[0097] If the previous location satisfies the precondition, the constraint assigner 103 assigns a Hoare annotation γ, including a precondition, a control strategy, and a postcondition, to the target location. If the previous location does not satisfy the precondition, the constraint assigner 103 determines that the Hoare annotation γ cannot be assigned to the target location. Thereafter, the constraint assigner 103 repeatedly assigns the Hoare annotation γ to the previous location as the target location.

[0098] In step S7, the constraint assignment unit 103 of the safety rule generation device 10 determines whether or not logical expression constraints have been assigned to all states included in the integrated control flow in step S6. Specifically, the constraint assignment unit 103 assigns the initial location l of the integrated control flow (hCFG) to the logical expression constraints. Init Determine whether or not has been assigned the Hoare annotation γ.

[0099] If the logical expression constraints can be assigned to the integrated control flow (YES), the constraint assignment unit 103 sends the integrated control flow to which the logical expression constraints have been assigned to the safety rule generation unit 104, and the process proceeds to step S8. On the other hand, if the logical expression constraints cannot be assigned to the integrated control flow (NO), the constraint assignment unit 103 ends the process without generating a safety rule. If the safety rule cannot be generated, the user can reconsider the scenario and execute the process again from step S1.

[0100] In step S7, a case where a logical expression constraint cannot be assigned to the integrated control flow corresponds to a case where a logical expression constraint (=Hoare annotation γ) cannot be assigned to each state of the control flow integrating the state transitions of each vehicle so as to satisfy the safety conditions. In other words, a case where a logical expression constraint cannot be assigned to the integrated control flow means that the control strategy indicated in the integrated control flow cannot be proven to achieve the safe goal. In this case, safety rules must be regenerated taking into account other control strategies or other scenarios. Since the Hoare annotation γ includes preconditions, control strategies, and postconditions, if a logical expression constraint cannot be assigned, for example, the scenario of the Hoare annotation γ may be reconsidered and the process may be repeated from step S1.

[0101] In step S8, the safety rule generation unit 104 of the safety rule generation device 10 receives the integrated control flow from the constraint assignment unit 103. Next, the safety rule generation unit 104 generates safety rules based on the logical expression constraints assigned to the integrated control flow. Specifically, the safety rule generation unit 104 combines the preconditions and control strategies of the logical expression constraints assigned to each state in the order shown in the integrated control flow. The safety rule generation unit 104 sends the safety rules combined with the preconditions and control strategies to the safety rule output unit 105.

[0102] In step S9, the safety rule output unit 105 of the safety rule generation device 10 receives the safety rule from the safety rule generation unit 104. Next, the safety rule output unit 105 generates safety rule information indicating the safety rule. Then, the safety rule output unit 105 transmits the safety rule information to the mobile object 20.

[0103] In step S10, the safety rule receiving unit 201 of the moving object 20 receives the safety rule information from the safety rule generation device 10. Next, the safety rule receiving unit 201 receives the received safety rule information as an input. Subsequently, the safety rule receiving unit 201 stores the received safety rule information in the safety rule storage unit 202.

[0104] In step S11, the driving control unit 203 of the moving body 20 accepts an operation to start automatic driving by the driver of the moving body 20. Next, the driving control unit 203 reads out the safety rule information stored in the safety rule storage unit 202. Next, the driving control unit 203 performs driving operations to complete a predetermined scenario based on the read out safety rule information. Thereafter, the driving control unit 203 autonomously performs driving operations in accordance with the safety rules until the scenario is completed or until the driver performs an operation to end automatic driving.

[0105] <Experimental Results> The results of verifying the safety of the safety rules in this embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the experimental results.

[0106] In the experiment, 23,328 simulations were performed using a simulator that implemented the intersection scenario shown in Figure 1. Specifically, eight initial positions of the oncoming vehicle POV were prepared, and 2,916 simulations were performed for each.

[0107] 9, RSS complying refers to the case where the initial state of the simulation satisfies the preconditions of the safety rules, and RSS non-complying refers to the case where the initial state of the simulation does not satisfy the preconditions of the safety rules.

[0108] Figure 9 shows the results of tallying the number of times the scenario was safely completed (experimentally safe) and the number of times the scenario was not safely completed (experimentally unsafe) for each simulation. As shown in Figure 9, with RSS compliance, the scenario was safely completed in all simulations. On the other hand, with no RSS compliance, the scenario could not be safely completed in approximately 90% of simulations.

[0109] The following conclusions can be drawn from the experimental results shown in Figure 9. First, if the preconditions of the safety rules are met, it is safe. Second, the safety rules are not overly conservative. If the safety rules are overly conservative, it is possible to safely complete the scenario even if the preconditions of the safety rules are not met. However, even if the scenario can be completed safely, it may take longer than necessary, and appropriate driving operations may not be performed. When the preconditions of the safety rules are not met, it can be evaluated that the fewer times the scenario is completed safely, the better.

[0110] As described above, the results of this experiment show that if the safety rules in this embodiment are used, safety can be guaranteed even in complex scenarios where multiple vehicles interact, such as intersection scenarios.

[0111] <Effects of the Embodiment> The safety rule generation device 10 in this embodiment integrates multiple control flows that indicate the state transitions of multiple moving objects operating in a predetermined scenario, and generates safety rules based on logical expression constraints assigned to each state. The safety rule generation device 10 can automatically calculate an integrated control flow that covers all combinations of the states of multiple moving objects. Therefore, according to this embodiment, safety rules that can accommodate complex scenarios can be easily formulated.

[0112] The safety rule generation device 10 in this embodiment may integrate multiple control flows by assigning transition conditions so that interactions between multiple moving objects are synchronized. The interactions may include causal interactions or temporal interactions. Therefore, according to this embodiment, safety rules can be formulated while taking interactions into consideration from various perspectives.

[0113] The safety rule generation device 10 in this embodiment may generate a safety rule when logical constraints are assigned to all states included in the integrated control flow. The logical constraints may include a precondition, a control strategy, and a postcondition. The logical constraints indicate a control strategy that ensures safety in that state, so that the mobile object 20 can execute the control strategy that ensures safety in all states that may occur in the scenario.

[0114] [Supplementary Note] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to execute each of the above-described functions.

[0115] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0116] This application claims priority from Japanese Patent Application No. 2023-90136, filed with the Japan Patent Office on May 31, 2023, the entire contents of which are incorporated herein by reference.

[0117] REFERENCE SIGNS LIST 1 Autonomous driving system 10 Safety rule generation device 20 Mobile body 30 User terminal 101 Control flow input unit 102 Control flow integration unit 103 Constraint assignment unit 104 Safety rule generation unit 105 Safety rule output unit 201 Safety rule reception unit 202 Safety rule storage unit 203 Driving control unit 301 Scenario input unit 302 Control flow creation unit 303 Control flow transmission unit

Claims

1. A safety rule generation device comprising: a control flow input unit configured to accept input of a plurality of control flows indicating state transitions of each of a plurality of moving objects operating in a specified scenario; a control flow integration unit configured to generate an integrated control flow that integrates the plurality of control flows based on transition conditions assigned to the control flows; a constraint assignment unit configured to assign logical expression constraints with guaranteed safety to each state included in the integrated control flow; and a safety rule generation unit configured to generate safety rules based on the logical expression constraints.

2. A safety rule generation device according to claim 1, wherein the control flow integration unit is configured to determine transition conditions to be assigned to states of the integrated control flow so that interactions between the multiple moving objects are synchronized.

3. The safety rule generating device according to claim 2, wherein the interaction includes a causal interaction or a temporal interaction.

4. A safety rule generation device according to any one of claims 1 to 3, wherein the control flow is a control flow graph including locations indicating the states of the moving body and directed edges indicating jumps between the locations, and the directed edges are assigned with transition conditions defined by event names.

5. A safety rule generation device according to any one of claims 1 to 3, wherein the constraint assignment unit is configured to assign the logical expression constraints in sequence, working backwards from a final state in the integrated control flow to an initial state.

6. A safety rule generation device according to claim 5, wherein the safety rule generation unit is configured to generate the safety rule when the logical expression constraints are assigned to all of the states included in the integrated control flow.

7. The safety rule generating device according to claim 6, wherein the logical expression constraints include a pre-condition, a control strategy, and a post-condition.

8. A safety rule generation device according to claim 7, wherein the logical expression constraints are predetermined such that the pre-condition is true, the post-condition is true while executing the control strategy, and a predetermined safety condition is true while executing the control strategy.

9. A safety rule generation device according to claim 7, wherein the safety rule generation unit is configured to generate the safety rule by combining the pre-conditions and the control strategies included in the logical expression constraints assigned to the integrated control flow in accordance with the control flow.

10. A mobile body comprising: a safety rule memory unit configured to store safety rules; and a driving control unit configured to execute driving operations in accordance with the safety rules, wherein the safety rules are generated based on the logical expression constraints by: generating an integrated control flow by integrating multiple control flows indicating state transitions of each of multiple mobile bodies operating in a specified scenario based on transition conditions assigned to the control flows; assigning logical expression constraints with guaranteed safety to each state included in the integrated control flow.

11. An autonomous driving system including a safety rule generation device and an autonomously driven mobile body, wherein the safety rule generation device comprises: a control flow input unit configured to accept input of a plurality of control flows indicating state transitions of each of a plurality of mobile bodies operating in a predetermined scenario; a control flow integration unit configured to generate an integrated control flow that integrates the plurality of control flows based on transition conditions assigned to the control flows; a constraint assignment unit configured to assign logical expression constraints with guaranteed safety to each state included in the integrated control flow; and a safety rule generation unit configured to generate safety rules based on the logical expression constraints; and the mobile body comprises: a safety rule memory unit configured to store the safety rules; and a driving control unit configured to perform driving operations in accordance with the safety rules.

12. A safety rule generation method in which a computer executes the following steps: accepting input of multiple control flows indicating state transitions of each of multiple moving objects operating in a specified scenario; generating an integrated control flow that integrates the multiple control flows based on transition conditions assigned to the control flows; assigning logical expression constraints with guaranteed safety to each state included in the integrated control flow; and generating safety rules based on the logical expression constraints.

13. A program for causing a computer to execute the following steps: accepting input of multiple control flows indicating state transitions of each of multiple moving objects operating in a specified scenario; generating an integrated control flow that integrates the multiple control flows based on transition conditions assigned to the control flows; assigning logical expression constraints with guaranteed safety to each state included in the integrated control flow; and generating safety rules based on the logical expression constraints.

Citation Information

Patent Citations

  • Methods for producing processed starch and fried food

    JP2023090136A

  • Vehicle guidance with systemic optimization

    WO2022231800A1