Method for evaluating an operating system, storage medium, and evaluation device.

Abstract

A driving system (2) is provided with a recognition system (10a), a determination system (20a) and a control system (30a) as subsystems. A method for evaluating the driving system (2) includes: modeling, as a loop structure, an interaction between the real world and each of the subsystems and identifying a closed loop (IL, EL); identifying an error that occurs in each of the subsystems; and evaluating an error that propagates along the closed loop (IL, EL).

Description

Cross - reference to related applications

[0001] This application is based on Japanese Patent Application No. 2022 - 9647 filed in Japan on January 25, 2022, the content of the base application is incorporated herein by reference in its entirety.

Technical Field

[0002] The disclosure according to this specification relates to a technology for realizing an operation system of a moving body.

Background Art

[0003] In the evaluation method of the operation system disclosed in Patent Document 1, in a game environment, based on the behavior of an object under automatic driving in response to the behavior of an object controlled by a person, the evaluation of the driving support function is carried out.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

[0005] However, the operation system has become complicated by including a plurality of subsystems. Therefore, in a simple test for evaluating the response to behavior, there is a limit in appropriately confirming the validity of the operation system including each subsystem. For this reason, it is difficult to optimize the operation system.

[0006] One of the purposes according to the disclosure of this specification is to provide an evaluation method of an operation system that can appropriately confirm the validity of the operation system 、 Storage medium and evaluation device is to provide.

[0007] One of the aspects disclosed herein is an operation system of a moving body including a recognition system, a judgment system, and a control system as subsystems Evaluate it using a computer.An evaluation method, Modeling the interaction between each subsystem and the real world as a loop structure and identifying closed loops, Identifying errors occurring in each subsystem, To enable the identification of complex factors between each subsystem, This includes evaluating errors that propagate according to a closed-loop system. Another aspect of the disclosed embodiment is an evaluation device for evaluating a mobile vehicle driving system, comprising a recognition system, a decision system, and a control system as subsystems, Equipped with at least one processor, The processor is, Modeling the interaction between each subsystem and the real world as a loop structure and identifying closed loops, Identifying errors occurring in each subsystem, To enable the identification of complex factors between each subsystem, we evaluate the errors that propagate according to the closed-loop principle and perform the following actions.

[0008] In this configuration, the interaction between each subsystem and the real world is modeled as a loop structure. The errors generated in each subsystem are then represented in a way that allows for the simulation of their propagation between subsystems, thanks to these identified closed loops. By evaluating the errors propagating according to the closed loops, the complex factors between subsystems can be identified. Therefore, the validity of an operating system with multiple subsystems can be properly verified.

[0009] Another aspect of the embodiments disclosed herein is a mobile vehicle driving system comprising a recognition system, a decision system, and a control system as subsystems. Evaluate it using a computer. An evaluation method, Modeling the interaction between each subsystem and the real world as a loop structure and identifying closed loops, To evaluate the complex factors between each subsystem, a confidence level is introduced for each subsystem as a common measure across them. To make it possible to identify multiple factors, This includes evaluating closed-loop systems based on their reliability. Another aspect of the disclosed aspect is, An evaluation device for evaluating a mobile vehicle driving system, comprising a recognition system, a decision system, and a control system as subsystems, Equipped with at least one processor, The processor is, Modeling the interaction between each subsystem and the real world as a loop structure and identifying closed loops, To evaluate the complex factors between each subsystem, a confidence level is introduced for each subsystem as a common measure across them. To enable the identification of multiple contributing factors, we will perform a closed-loop evaluation based on confidence levels.

[0010] According to such an aspect, the interaction between each subsystem and the real world is modeled as a loop structure. The evaluation of the closed loop thus specified is based on the reliability, which is a common measure among the subsystems. Since reliability is introduced as a common measure, even if the recognition system, the judgment system, and the control system have different functions respectively, the composite factors due to these interactions can be identified. Therefore, the validity of the driving system having a plurality of subsystems can be appropriately confirmed.

[0011] Another one of the aspects disclosed herein is a computer-readable storage medium configured to cause a computer to model the interaction between each subsystem and the real world as a loop structure to identify a closed loop for a driving system of a moving body having a recognition system, a judgment system, and a control system as subsystems, identify the errors occurring in each subsystem, evaluate the errors propagating according to the closed loop, and store a computer program for causing the computer to execute the above operations. To enable the identification of complex factors between each subsystem, According to such an aspect, the interaction between each subsystem and the real world is modeled as a loop structure. By the closed loop thus specified, the errors occurring in each subsystem are expressed in a form that can simulate the propagation among the subsystems. By evaluating the errors propagating according to the closed loop, the composite factors among the subsystems can be identified. Therefore, the validity of the driving system having a plurality of subsystems can be appropriately confirmed.

[0012]

[0013] ​Another aspect disclosed herein is a computer-readable storage medium, for a computer, with respect to a driving system of a moving body having a recognition system, a judgment system, and a control system as subsystems, to model the interaction between each subsystem and the real world as a loop structure to identify a closed loop, to introduce a reliability factor to each subsystem as a common measure between each subsystem for evaluating the combined factors between each subsystem, To make it possible to identify multiple factors, and to store a computer program that causes the computer to evaluate the closed loop based on the reliability factor.

[0014] According to such an aspect, the interaction between each subsystem and the real world is modeled as a loop structure. The evaluation of the closed loop thus identified is based on the reliability factor, which is a common measure between each subsystem. Since the reliability factor is introduced as a common measure, even if the recognition system, the judgment system, and the control system each have different functions, the combined factors due to these interactions can be confirmed. Therefore, the validity of the driving system including a plurality of subsystems can be appropriately confirmed.

[0015] The reference signs in the parentheses in the claims are for exemplarily showing the correspondence with the parts of the embodiments described later, and are not intended to limit the technical scope.

Brief Description of the Drawings

[0016] [Figure 1] It is a block diagram showing a schematic configuration of a driving system. [Figure 2] It is a block diagram showing a configuration of the technical level of a driving system. [Figure 3] It is a block diagram showing a configuration of the functional level of a driving system. [Figure 4] It is a diagram showing the control state space of a vehicle. [Figure 5]This is a block diagram showing the causal loops in the driving system. [Figure 6] This is a diagram illustrating the inner loop. [Figure 7] This is a diagram illustrating the outer loop. [Figure 8] This diagram shows the region where safety cannot be maintained, based on the concept of the first evaluation method. [Figure 9] This is a flowchart explaining the first evaluation method. [Figure 10] This diagram shows the region where safety cannot be maintained, based on the concept of the second evaluation method. [Figure 11] This is a flowchart explaining the second evaluation method. [Figure 12] This diagram shows the area where safety cannot be maintained, based on the concept of the third evaluation method. [Figure 13] This is a flowchart explaining the third evaluation method. [Figure 14] This is a flowchart explaining a reliability-based evaluation method. [Figure 15] This is a block diagram of an evaluation device and a design device. [Figure 16] This graph shows the relationship between error distribution and confidence level. [Figure 17] This is a flowchart explaining the first design method. [Figure 18] This is a block diagram showing the causal loops in the driving system. [Figure 19] This is a diagram illustrating the inner loop. [Figure 20] This is a diagram illustrating the outer loop. [Figure 21] This is a diagram illustrating the vehicle body stabilization loop. [Figure 22] This is a table showing various errors. [Figure 23] This is a flowchart explaining an evaluation method based on error. [Figure 24] This is a flowchart explaining the second design method. [Figure 25] This is a flowchart explaining the operation of the driving system. [Figure 26] This is a block diagram showing the configuration of the functional levels of the driving system. [Figure 27] This is a block diagram showing the configuration of the technical levels of the driving system. [Figure 28] This is a flowchart explaining the operation of the driving system. [Figure 29] This is a block diagram showing the configuration of the functional levels of the driving system. [Figure 30] This is a block diagram showing the configuration of the technical levels of the driving system. [Modes for carrying out the invention]

[0017] Several embodiments will be described below with reference to the drawings. In each embodiment, the same reference numerals are used for corresponding components, and redundant explanations may be omitted. If only a part of the configuration is described in each embodiment, the configuration of other embodiments described earlier can be applied to the other parts of that configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations from multiple embodiments can be partially combined even if not explicitly stated, as long as there are no particular problems with the combination.

[0018] (First Embodiment) The driving system 2 of the first embodiment shown in Figure 1 implements functions related to the driving of a mobile object. Part or all of the driving system 2 is mounted on the mobile object. The mobile object that the driving system 2 processes is a vehicle. This vehicle can be referred to as the self-vehicle 1 and corresponds to the host mobile object. The self-vehicle 1 may be configured to communicate with other vehicles directly or indirectly via a communication infrastructure. Other vehicles correspond to the target mobile objects.

[0019] Vehicle 1 is a road user capable of autonomous driving, such as a car or truck. Driving is categorized into levels depending on the extent to which the driver performs all dynamic driving tasks (DDTs). Automated driving levels are defined, for example, in SAE J3016. In levels 0-2, the driver performs some or all of the DDTs. Levels 0-2 may also be classified as so-called manual driving. Level 0 indicates that driving is not automated. Level 1 indicates that the driving system 2 assists the driver. Level 2 indicates that driving is partially automated.

[0020] At Level 3 and above, the driving system 2 performs all of the DDT while engaged. Levels 3-5 may be classified as so-called autonomous driving. A driving system 2 capable of performing driving at Level 3 or above may be referred to as an automated driving system. Level 3 indicates that driving is conditionally automated. Level 4 indicates that driving is highly automated. Level 5 indicates that driving is fully automated.

[0021] Furthermore, a driving system 2 that is unable to perform Level 3 or higher driving but is capable of performing at least one of Levels 1 and 2 driving may be referred to as a driver assistance system. In the following explanation, unless there is a specific reason to identify the highest possible level of automated driving, the automated driving system or driver assistance system will simply be referred to as driving system 2.

[0022] <Sense-Plan-Act Model> The architecture of the driving system 2 is selected to enable an efficient SOTIF (safety of the intended functionality) process. For example, the architecture of the driving system 2 may be based on a sense-plan-act model. The sense-plan-act model comprises a sense element, a plan element, and an act element as its main system elements. The sense element, plan element, and act element interact with each other. Here, sense can be interpreted as perception, plan as judgment, and act as control, and the following explanation will primarily use the terms perception, judgment, and control.

[0023] As shown in Figure 1, in this driving system 2, at the vehicle level, vehicle-level functions 3 are implemented based on the Vehicle Level Safety Strategy (VLSS). At the functional level (in other words, from a functional perspective), recognition functions, decision-making functions, and control functions are implemented. At the technical level (in other words, from a technical perspective), multiple sensors 40 corresponding to the recognition function, a processing system 50 corresponding to the decision-making function, and multiple motion actuators 60 corresponding to the control function are implemented.

[0024] More specifically, a recognition unit 10, which is a functional block that realizes a recognition function, may be constructed in the driving system 2, mainly consisting of multiple sensors 40, a processing system that processes detection information from the multiple sensors 40, and a processing system that generates an environmental model based on the information from the multiple sensors 40. A judgment unit 20, which is a functional block that realizes a judgment function, may be constructed in the driving system 2, mainly consisting of the processing system. A control unit 30, which is a functional block that realizes a control function, may be constructed in the driving system 2, mainly consisting of multiple motion actuators 60 and at least one processing system that outputs operation signals from the multiple motion actuators 60.

[0025] Here, the recognition unit 10 may be implemented as a recognition system 10a, a subsystem distinct from the judgment unit 20 and the control unit 30. The judgment unit 20 may be implemented as a judgment system 20a, a subsystem distinct from the recognition unit 10 and the control unit 30. The control unit 30 may be implemented as a control system 30a, a subsystem distinct from the recognition unit 10 and the judgment unit 20. The recognition system 10a, the judgment system 20a, and the control system 30a may constitute mutually independent components.

[0026] Furthermore, the vehicle 1 may be equipped with multiple HMI (Human Machine Interface) devices 70. Of the multiple HMI devices 70, the part that realizes the operation input function by the occupant may be part of the recognition unit 10. Of the multiple HMI devices 70, the part that realizes the information presentation function may be part of the control unit 30. On the other hand, the functions realized by the HMI devices 70 may be positioned as functions independent of the recognition function, judgment function and control function.

[0027] The recognition unit 10 is responsible for recognition functions, including localization of road users such as the vehicle itself 1 and other vehicles. The recognition unit 10 detects the external environment EE, internal environment, vehicle status, and the status of the driving system 2 of the vehicle itself 1. The recognition unit 10 fuses the detected information to generate an environmental model. The decision unit 20 applies its purpose and driving policy to the environmental model generated by the recognition unit 10 to derive control actions. The control unit 30 executes the control actions derived by the recognition elements.

[0028] <System configuration at a technical level> Figure 2 illustrates an example of the detailed configuration of the driving system 2 at the technical level. The configuration at the technical level may refer to the physical architecture. The driving system 2 includes multiple sensors 40, multiple motion actuators 60, multiple HMI devices 70, and at least one processing system 50, etc. These components can communicate with each other by either wireless or wired connections, or both. These components may also be able to communicate with each other through an in-vehicle network such as CAN (registered trademark).

[0029] Multiple sensors 40 include one or more external environment sensors 41. Multiple sensors 40 may include at least one of one or more internal environment sensors 42, one or more communication systems 43, and a map database 44. If sensor 40 is interpreted narrowly to refer to an external environment sensor 41, the internal environment sensors 42, communication systems 43, and map database 44 may be positioned as components separate from the sensor 40 corresponding to the recognition function at a technical level.

[0030] The external environment sensor 41 may detect targets present in the external environment EE of the vehicle 1. Examples of target-detection type external environment sensors 41 include cameras, LiDAR (Light Detection and Ranging / Laser imaging Detection and Ranging) laser radar, millimeter-wave radar, ultrasonic sonar, etc. Typically, multiple types of external environment sensors 41 can be combined and implemented to monitor the front, side, and rear directions of the vehicle 1.

[0031] As an example of mounting the external environment sensor 41, a plurality of cameras (for example, 11 cameras) configured to monitor the front, front side, side, rear side, and rear directions of the vehicle 1 may be mounted on the vehicle 1.

[0032] Other possible installations include multiple cameras (e.g., camera 4) configured to monitor the front, sides, and rear of the vehicle 1, multiple millimeter-wave radars (e.g., millimeter-wave radar 5) configured to monitor the front, front sides, sides, and rear of the vehicle 1, and a LiDAR configured to monitor the front of the vehicle 1.

[0033] Furthermore, the external environment sensor 41 may detect the atmospheric conditions and weather conditions in the external environment EE of the vehicle 1. Examples of the state-detection type external environment sensor 41 include an outside temperature sensor, a temperature sensor, a raindrop sensor, etc.

[0034] The internal environment sensor 42 may detect specific physical quantities related to vehicle motion (hereinafter referred to as motion physical quantities) in the internal environment of the vehicle 1. Examples of motion physical quantity detection type internal environment sensors 42 include speed sensors, acceleration sensors, gyro sensors, etc. The internal environment sensor 42 may also detect the state of the occupants in the internal environment of the vehicle 1. Examples of occupant detection type internal environment sensors 42 include actuator sensors, sensors and systems for monitoring the driver, biosensors, seating sensors, and in-vehicle equipment sensors, etc. In particular, actuator sensors include accelerator sensors, brake sensors, steering sensors, etc., which detect the occupant's operation state on motion actuators 60 related to the motion control of the vehicle 1.

[0035] The communication system 43 acquires communication data available to the driving system 2 via wireless communication. The communication system 43 may also receive positioning signals from GNSS (global navigation satellite system) satellites present in the external environment EE of the vehicle 1. Positioning type communication equipment in the communication system 43 is, for example, a GNSS receiver.

[0036] The communication system 43 may send and receive communication signals with V2X systems present in the external environment EE of the vehicle 1. Examples of V2X type communication equipment in the communication system 43 include DSRC (dedicated short range communications) communication devices and cellular V2X (C-V2X) communication devices. Examples of communication with V2X systems present in the external environment EE of the vehicle 1 include communication with communication systems of other vehicles (V2V), communication with infrastructure equipment such as communication devices set up on traffic lights (V2I), communication with pedestrian mobile terminals (V2P), and communication with networks such as cloud servers (V2N).

[0037] Furthermore, the communication system 43 may send and receive communication signals between itself and the internal environment of the vehicle 1, for example, with a mobile terminal such as a smartphone located inside the vehicle. Examples of terminal communication devices in the communication system 43 include Bluetooth devices, Wi-Fi devices, infrared communication devices, etc.

[0038] Map DB44 is a database that stores map data available to the driving system 2. Map DB44 is composed of at least one type of non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media. Map DB44 may include a database of a navigation unit that navigates the driving route of the vehicle 1 to its destination. Map DB44 may also include a database of PD maps generated using probe data (PD) collected from each vehicle. Map DB44 may also include a database of high-precision maps with a high level of accuracy, mainly used for applications in automated driving systems. Map DB44 may also include a database of parking maps that include detailed parking information, such as parking space information, used for applications in automated parking or parking assistance.

[0039] A map DB44 suitable for the driving system 2 acquires and stores the latest map data, for example, by communicating with a map server via a V2X type communication system 43. The map data is digitized in two or three dimensions as data representing the external environment EE of the vehicle 1. The map data may include road data representing at least one of the following: the position coordinates, shape, road surface condition, and standard roads of road structures. The map data may also include marking data representing at least one of the following: the position coordinates and shape of road signs, road markings, and lane markings attached to roads. The marking data included in the map data may represent landmarks such as traffic signs, arrow markings, lane markings, stop lines, direction signs, landmark beacons, business signs, and changes in road line patterns. The map data may also include structural data representing at least one of the following: the position coordinates and shape of buildings and traffic lights facing roads. The marking data included in the map data may represent landmarks such as streetlights, road edges, reflectors, and poles.

[0040] The motion actuator 60 can control vehicle motion based on input control signals. A drive-type motion actuator 60 is a powertrain including at least one of the following: an internal combustion engine, a drive motor, etc. A braking-type motion actuator 60 is, for example, a brake actuator. A steering-type motion actuator 60 is, for example, a steering wheel.

[0041] The HMI device 70 may be an operation input device that can receive driver input for transmitting the will or intentions of the occupants, including the driver of the vehicle 1, to the driving system 2. Examples of operation input type HMI devices 70 include an accelerator pedal, brake pedal, shift lever, steering wheel, turn signal lever, mechanical switch, touch panel of a navigation unit, etc. Of these, the accelerator pedal controls the powertrain as a motion actuator 60. The brake pedal controls the brake actuator as a motion actuator 60. The steering wheel controls the steering actuator as a motion actuator 60.

[0042] The HMI device 70 may be an information display device that presents information such as visual information, auditory information, and tactile information to the occupants of the vehicle 1, including the driver. Examples of visual information display type HMI devices 70 include combination meters, navigation units, CIDs (center information displays), HUDs (head-up displays), illumination units, etc. Examples of auditory information display type HMI devices 70 include speakers, buzzers, etc. Examples of tactile information display type HMI devices 70 include steering wheel vibration units, driver's seat vibration units, steering wheel reaction force units, accelerator pedal reaction force units, brake pedal reaction force units, air conditioning units, etc.

[0043] Furthermore, the HMI device 70 may communicate with a mobile terminal such as a smartphone via the communication system 43 to realize HMI functions in conjunction with the terminal. For example, the HMI device 70 may present information acquired from the smartphone to the occupants, including the driver. Alternatively, for example, inputting commands into the smartphone may be used as an alternative means of inputting commands into the HMI device 70.

[0044] At least one processing system 50 is provided. For example, the processing system 50 may be an integrated processing system that comprehensively executes processing related to recognition functions, processing related to judgment functions, and processing related to control functions. In this case, the integrated processing system 50 may further execute processing related to the HMI device 70, and a separate processing system dedicated to the HMI may be provided. For example, the processing system dedicated to the HMI may be an integrated cockpit system that comprehensively executes processing related to each HMI device.

[0045] For example, the processing system 50 may have a configuration that includes at least one processing unit corresponding to processing related to recognition functions, at least one processing unit corresponding to processing related to judgment functions, and at least one processing unit corresponding to processing related to control functions.

[0046] The processing system 50 has an external communication interface and is connected to at least one element related to processing by the processing system 50, such as the sensor 40, motion actuator 60, and HMI device 70, via at least one of the following: a LAN (Local Area Network), wire harness, internal bus, and wireless communication circuit.

[0047] The processing system 50 is comprised of at least one dedicated computer 51. The processing system 50 may combine multiple dedicated computers 51 to implement functions such as recognition, judgment, and control.

[0048] For example, the dedicated computer 51 constituting the processing system 50 may be an integrated ECU that integrates the driving functions of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be a judgment ECU that determines DDT. The dedicated computer 51 constituting the processing system 50 may be a monitoring ECU that monitors the driving of the vehicle. The dedicated computer 51 constituting the processing system 50 may be an evaluation ECU that evaluates the driving of the vehicle. The dedicated computer 51 constituting the processing system 50 may be a navigation ECU that navigates the driving route of the vehicle 1.

[0049] Furthermore, the dedicated computer 51 constituting the processing system 50 may be a locator ECU that estimates the position of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an image processing ECU that processes image data detected by the external environmental sensor 41. The dedicated computer 51 constituting the processing system 50 may be an actuator ECU that controls the motion actuator 60 of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an HCU (HMI Control Unit) that comprehensively controls the HMI equipment 70. The dedicated computer 51 constituting the processing system 50 may be at least one external computer that constructs an external center or mobile terminal that can communicate via the communication system 43, for example.

[0050] The dedicated computer 51 constituting the processing system 50 has at least one memory 51a and at least one processor 51b. The memory 51a may be at least one type of non-transitional physical storage medium, such as semiconductor memory, magnetic media, and optical media, which non-temporarily stores programs and data that can be read by the processor 51b. Furthermore, the memory 51a may also be a rewritable volatile storage medium, such as RAM (Random Access Memory). The processor 51b includes at least one type as a core, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and RISC (Reduced Instruction Set Computer)-CPU.

[0051] The dedicated computer 51 constituting the processing system 50 may be a System on a Chip (SoC) that integrates memory, a processor, and interfaces into a single chip, or it may have an SoC as a component of the dedicated computer.

[0052] Furthermore, the processing system 50 may include at least one database for executing dynamic operation tasks. The database is configured to include at least one type of non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media. The database may also be a scenario DB53, which databases the scenario structure described later.

[0053] Furthermore, the processing system 50 may include at least one recording device 55 for recording at least one of the recognition information, judgment information, and control information of the operating system 2. The recording device 55 may include at least one memory 55a and an interface 55b for writing data to the memory 55a. The memory 55a may be at least one type of non-transitional physical storage medium, such as a semiconductor memory, a magnetic medium, and an optical medium.

[0054] At least one of the memory 55a may be mounted on the circuit board in a form that is not easily removable or replaceable, in which case, for example, an eMMC (embedded Multi Media Card) using flash memory may be used. At least one of the memory 55a may be mounted on the recording device 55 in a form that is removable and replaceable, in which case, for example, an SD card may be used.

[0055] The recording device 55 may have a function to select the information to be recorded from among recognition information, judgment information, and control information. In this case, the recording device 55 may have a dedicated computer 55c. The processor provided in the recording device 55 may temporarily store information in RAM or the like. The processor may select the information to be recorded from the temporarily stored information and save the selected information to memory 51a.

[0056] The recording device 55 may access the memory 55a and perform recording in accordance with a data write command from the recognition system 10a, the decision system 20a, or the control system 30a. The recording device 55 may also determine the information flowing through the in-vehicle network and, based on the judgment of a processor provided in the recording device 55, access the memory 55a and perform recording.

[0057] <System configuration at the functional level> Next, an example of the detailed configuration of the operating system 2 at the functional level will be described using Figure 3. The configuration at the functional level may also refer to the logical architecture. The recognition unit 10 comprises an external recognition unit 11, a self-position recognition unit 12, a fusion unit 13, and an internal recognition unit 14 as subblocks that further classify the recognition functions.

[0058] The external recognition unit 11 processes the detection data detected by each external environmental sensor 41 individually to realize the function of recognizing objects such as targets and other road users. The detection data may be, for example, detection data provided by millimeter-wave radar, sonar, LiDAR, etc. The external recognition unit 11 may generate relative position data, including the direction, size, and distance of the object relative to the vehicle 1, from the raw data detected by the external environmental data.

[0059] Furthermore, the detection data may be image data provided from, for example, a camera, LiDAR, etc. The external recognition unit 11 processes the image data and extracts objects that are captured within the field of view of the image. Object extraction may include estimating the direction, size, and distance of the object relative to the vehicle 1. Object extraction may also include classifying the object using, for example, semantic segmentation.

[0060] The self-position recognition unit 12 performs localization of the vehicle 1. The self-position recognition unit 12 acquires global position data of the vehicle 1 from the communication system 43 (e.g., a GNSS receiver). In addition, the self-position recognition unit 12 may acquire at least one of the position information of targets extracted by the external recognition unit 11 and the position information of targets extracted by the fusion unit 13. The self-position recognition unit 12 also acquires map information from the map DB 44. The self-position recognition unit 12 integrates this information to estimate the position of the vehicle 1 on the map.

[0061] The fusion unit 13 fuses the external recognition information from each external environmental sensor 41 processed by the external recognition unit 11, the localization information processed by the self-position recognition unit 12, and the V2X information acquired by V2X.

[0062] The fusion unit 13 fuses object information of other road users and other objects individually recognized by each external environmental sensor 41 to identify the type and relative position of objects around the vehicle 1. The fusion unit 13 also fuses road landmark information individually recognized by each external environmental sensor 41 to identify the static structure of the road around the vehicle 1. The static structure of the road includes, for example, curve curvature, number of lanes, and open space.

[0063] Next, the fusion unit 13 fuses the types of objects, relative positions, and static road structures around the vehicle 1, as well as localization information and V2X information, to generate an environmental model. The environmental model can be provided to the decision unit 20. The environmental model may be an environmental model specifically designed for modeling the external environment EE.

[0064] The environmental model may be an integrated environmental model that fuses information such as the internal environment, vehicle status, and the status of the driving system 2, which is realized by expanding the information acquired. For example, the fusion unit 13 may acquire traffic rules such as the Road Traffic Act and reflect them in the environmental model.

[0065] The internal recognition unit 14 processes the detection data detected by each internal environment sensor 42 and realizes the function of recognizing the vehicle state. The vehicle state may include the state of the vehicle's physical motion quantities detected by the speed sensor, acceleration sensor, gyro sensor, etc. The vehicle state may also include at least one of the following: the state of the occupants, including the driver; the driver's operation state of the motion actuator 60; and the switch state of the HMI equipment 70.

[0066] The decision unit 20 includes an environmental decision unit 21, an operation planning unit 22, and a mode management unit 23 as subblocks that further classify the decision functions.

[0067] The environmental judgment unit 21 acquires the environmental model generated by the fusion unit 13 and the vehicle status recognized by the internal recognition unit 14, and makes environmental judgments based on these. Specifically, the environmental judgment unit 21 may interpret the environmental model and estimate the situation in which the vehicle 1 is currently located. The situation here may be the operating situation. The environmental judgment unit 21 may also interpret the environmental model and predict the trajectories of objects such as other road users. Furthermore, the environmental judgment unit 21 may interpret the environmental model and predict potential hazards.

[0068] Furthermore, the environmental judgment unit 21 may interpret the environmental model and make a judgment regarding the scenario in which the vehicle 1 is currently located. The judgment regarding the scenario may involve selecting at least one scenario in which the vehicle 1 is currently located from the catalog of scenarios built in scenario DB53. The judgment regarding the scenario may also involve a judgment regarding the scenario category, as described later.

[0069] Furthermore, the environmental judgment unit 21 may estimate the driver's intentions based on at least one of the predicted object trajectory, predicted potential hazards, and scenario-related judgments, as well as the vehicle status provided by the internal recognition unit 14.

[0070] The driving plan unit 22 plans the operation of the vehicle 1 based on at least one of the following: the estimated position of the vehicle 1 on the map by the self-position recognition unit 12, the judgment information and driver intention estimation information by the environment judgment unit 21, and the function constraint information by the mode management unit 23.

[0071] The driving planning unit 22 implements route planning, behavior planning, and trajectory planning functions. The route planning function plans at least one of the following: a route to the destination and a lane plan for the medium distance, based on estimated information of the vehicle's position on the map. The route planning function may further include a function to determine at least one of the following: a lane change request and a deceleration request, based on the lane plan for the medium distance. Here, the route planning function may be a mission / route planning function in the Strategic Function, and may output a mission plan and a route plan.

[0072] The behavior planning function plans the behavior of the vehicle 1 based on at least one of the following: the route to the destination planned by the route planning function, the lane plan for medium distances, lane change requests and deceleration requests, judgment information and driver intention estimation information from the environment judgment unit 21, and function constraint information from the mode management unit 23. The behavior planning function may also include a function to generate conditions related to the state transitions of the vehicle 1. The conditions related to the state transitions of the vehicle 1 may correspond to triggering conditions. Based on these conditions, the behavior planning function may also include a function to determine the state transitions of the application that realizes DDT, and further, the state transitions of driving behavior. Based on this state transition information, the behavior planning function may also include a function to determine longitudinal constraints and lateral constraints related to the path of the vehicle 1. The behavior planning function may be a tactical behavior plan in the DDT function and may output tactical behavior.

[0073] The track planning function is a function that plans the trajectory of the vehicle 1 based on judgment information from the environmental judgment unit 21, longitudinal constraints on the path of the vehicle 1, and lateral constraints on the path of the vehicle 1. The track planning function may also include a function to generate a path plan. The path plan may include a speed plan, and the speed plan may be generated as a plan independent of the path plan. The track planning function may also include a function to generate multiple path plans and select the optimal path plan from among the multiple path plans, or a function to switch between path plans. The track planning function may further include a function to generate backup data of the generated path plan. The track planning function may be the track planning function in the DDT function and may output a track plan.

[0074] The mode management unit 23 monitors the driving system 2 and sets constraints on driving-related functions. The mode management unit 23 may also monitor the status of subsystems related to the driving system 2 and determine if the system 2 is malfunctioning. The mode management unit 23 may determine a mode based on the driver's intent based on driver intent estimation information generated by the internal recognition unit 14. The mode management unit 23 may set constraints on driving-related functions based on the malfunction determination result of the system 2, the mode determination result, and at least one of the following: vehicle status from the internal recognition unit 14, sensor abnormality (or sensor failure) signal output from the sensor 40, application state transition information from the driving plan unit 22, and track plan.

[0075] Furthermore, the mode management unit 23 may comprehensively have functions to determine longitudinal constraints and lateral constraints on the path of the vehicle 1, in addition to constraints on driving functions. In this case, the driving planning unit 22 plans the behavior and trajectory according to the constraints determined by the mode management unit 23.

[0076] The control unit 30 comprises a motion control unit 31 and an HMI output unit 71 as subblocks that further classify the control functions. The motion control unit 31 controls the motion of the vehicle 1 based on the trajectory plan (e.g., path plan and speed plan) obtained from the driving plan unit 22. Specifically, the motion control unit 31 generates accelerator request information, shift request information, brake request information and steering request information according to the trajectory plan and outputs them to the motion actuator 60.

[0077] Here, the motion control unit 31 can directly obtain from the recognition unit 10 (particularly the internal recognition unit 14) at least one of the vehicle's current speed, acceleration, and yaw rate, and reflect it in the motion control of the vehicle 1.

[0078] The HMI output unit 71 outputs information related to the HMI based on at least one of the following: judgment information and driver intent estimation information from the environmental judgment unit 21, application state transition information and track plan from the driving plan unit 22, and function constraint information from the mode management unit 23. The HMI output unit 71 may also manage vehicle interactions. The HMI output unit 71 may generate notification requests based on the vehicle interaction management status and control the information notification function of the HMI device 70. Furthermore, the HMI output unit 71 may generate control requests for wipers, sensor cleaning devices, headlights, and air conditioning devices based on the vehicle interaction management status and control these devices.

[0079] <Scenario> A scenario-based approach may be employed to perform or evaluate dynamic driving tasks. As mentioned above, the processes required to perform dynamic driving tasks in autonomous driving are classified into disturbances in perception elements with different physical principles, disturbances in decision elements, and disturbances in control elements. The root causes that affect the processing results in each element are structured as a scenario structure.

[0080] The disturbance in the perception element is called a perception disturbance. A perception disturbance is a disturbance that indicates a state in which the perception unit 10 cannot correctly perceive a hazard due to internal or external factors of the sensor 40 and the vehicle 1. Internal factors include, for example, instability related to variations in the mounting or manufacturing of sensors such as the external environmental sensor 41, vehicle tilt due to uneven loads that change the direction of the sensor, and shielding of the sensor due to the mounting of parts on the outside of the vehicle. External factors include, for example, fogging or dirt on the sensor. The physical principles in perception disturbances are based on the sensor mechanism of each sensor.

[0081] The disturbance in the judgment element is traffic disturbance. Traffic disturbance is a disturbance that describes a traffic situation that may be dangerous as a result of a combination of the geometric shape of the road, the behavior of the vehicle itself, and the position and behavior of surrounding vehicles. The physical principles in traffic disturbance are based on a geometric perspective and the actions of road users.

[0082] The disturbance in a control element is a vehicle disturbance. Vehicle disturbances may also be called control disturbances. A vehicle disturbance is a disturbance that indicates a situation in which a vehicle may be unable to control its own dynamics due to internal or external factors. Internal factors include, for example, the total weight and weight balance of the vehicle. External factors include, for example, road surface irregularities, incline, and wind. The physical principles in vehicle disturbances are based on the mechanical effects applied to the tires and the vehicle body.

[0083] To address the risk of collisions with other road users or structures in the dynamic driving tasks of autonomous driving, a traffic disturbance scenario system is used as one of the scenario structures, in which traffic disturbance scenarios are systematized. For the traffic disturbance scenario system, a reasonably foreseeable range or reasonably foreseeable boundary may be defined, and an avoidable range or avoidable boundary may also be defined.

[0084] The avoidable range or avoidable boundary can be defined, for example, by defining and modeling the performance of a competent and careful human driver. The performance of a competent and careful human driver can be defined in three elements: perception, judgment, and control.

[0085] Traffic disturbance scenarios include, for example, cut-in scenarios, cut-out scenarios, and deceleration scenarios. A cut-in scenario is when another vehicle traveling in an adjacent lane merges in front of vehicle 1. A cut-out scenario is when another vehicle that vehicle 1 is following changes lanes to an adjacent lane. In this case, vehicle 1 is required to take an appropriate response to objects that suddenly appear in front of it, stopped vehicles at the end of a traffic jam, etc. A deceleration scenario is when another vehicle that vehicle 1 is following suddenly decelerates.

[0086] Traffic disturbance scenarios can be generated by systematically analyzing and classifying different combinations of elements such as the geometric shape of the road, the movement of the vehicle itself, the positions of other surrounding vehicles, and the movements of other surrounding vehicles.

[0087] Here, as an example of systematizing traffic disturbance scenarios, the structure of a traffic disturbance scenario on a highway is described. The road shape is classified into four categories: main lanes, merges, junctions, and ramps. The actions of vehicle 1 are classified into two categories: lane keeping and lane changes. The positions of other surrounding vehicles are defined, for example, by the adjacent positions in eight surrounding directions that may enter the trajectory of vehicle 1. Specifically, the eight directions are: Lead, Following, Parallel (Pr-f) to the front right, Parallel (Pr-s) to the side right, Parallel (Pr-r) to the rear right, Parallel (Pl-f) to the front left, Parallel (Pl-s) to the side left, and Parallel (Pl-r) to the rear left. The actions of other surrounding vehicles are classified into five categories: cut-in, cut-out, acceleration, deceleration, and synchronization. Deceleration may include stopping.

[0088] The combinations of positions and actions of other surrounding vehicles include combinations that can and cannot reasonably foresee potential disruptions. For example, cut-ins can occur in 6 categories of parallel driving. Cut-outs can occur in 2 categories of leading and following. Acceleration can occur in 3 categories of following, driving alongside to the right rear, and driving alongside to the left rear. Deceleration can occur in 3 categories of leading, driving alongside to the right front, and driving alongside to the left front. Synchronization can occur in 2 categories of driving alongside to the right and driving alongside to the left. Thus, the structure of traffic disturbance scenarios on highways consists of a matrix containing 40 possible combinations. The structure of traffic disturbance scenarios may be further extended to include more complex scenarios by considering motorcycles and at least one of multiple vehicles.

[0089] Next, we will explain the recognition disturbance scenario system. The recognition disturbance scenario may include not only sensor disturbance scenarios caused by external environmental sensors, but also blind spot scenarios (also called shielding scenarios) and communication disturbance scenarios.

[0090] Sensor disturbance scenarios can be generated by systematically analyzing and classifying different combinations of factors and elements of the sensor mechanism.

[0091] Among the factors causing sensor disturbances, those related to the vehicle and the sensor can be classified into three categories: the vehicle itself, the sensor, and the sensor front. Factors related to the vehicle itself include, for example, changes in vehicle attitude. Factors related to the sensor include, for example, variations in mounting and malfunctions of the sensor itself. Factors related to the sensor front include deposits and changes in characteristics, and in the case of a camera, reflections are also included. In relation to these factors, the influence of each external environmental sensor 41 according to its specific sensor mechanism can be assumed to be perceived disturbances.

[0092] Among the factors causing sensor disturbances, those related to the external environment can be classified into three categories: surrounding structures, space, and surrounding moving objects. Surrounding structures can be further classified into three categories based on their positional relationship with the vehicle: road surface, roadside structures, and overhead structures. Factors related to the road surface include, for example, shape, road surface condition, and material. Factors related to roadside structures include, for example, reflection, shielding, and background. Factors related to overhead structures include, for example, reflection, shielding, and background. Factors related to space include, for example, spatial obstacles, radio waves, and light in space. Factors related to surrounding moving objects include, for example, reflection, shielding, and background. The influence of these factors on the sensor mechanism specific to each external environment sensor can be assumed to be perceived disturbances.

[0093] Among the factors causing sensor disturbances, those related to the objects the sensor recognizes can be broadly categorized into four types: road paths, traffic information, road obstacles, and moving objects.

[0094] The road is classified into lane markings, elevated structures, and road edges based on the structure of the objects that mark the road. Road edges are classified into flat road edges and elevated road edges. Factors affecting lane markings include, for example, color, material, shape, dirt, fading, and relative position. Factors affecting elevated structures include, for example, color, material, dirt, and relative position. Factors affecting flat road edges include, for example, color, material, dirt, and relative position. Factors affecting elevated road edges include, for example, color, material, dirt, and relative position. The influence of these factors on the sensor mechanism specific to each external environmental sensor can be assumed as recognition disturbances.

[0095] Traffic information is classified into signals, signs, and road markings based on their display format. Factors related to signals include, for example, color, material, shape, light source, dirt, and relative position. Factors related to signs include, for example, color, material, shape, light source, dirt, and relative position. Factors related to road markings include, for example, color, material, shape, dirt, and relative position. The influence of these factors on each external environmental sensor 41 according to its specific sensor mechanism can be assumed as a recognition disturbance.

[0096] Road obstacles are classified into fallen objects, animals, and installed objects based on whether they are moving and the magnitude of their impact if they collide with the vehicle 1. Factors related to fallen objects include, for example, color, material, shape, size, relative position, and behavior. Factors related to animals include, for example, color, material, shape, size, relative position, and behavior. Factors related to installed objects include, for example, color, material, shape, size, dirt, and relative position. For these factors, the influence of the sensor mechanism specific to each external environment sensor 41 can be assumed as a recognition disturbance.

[0097] Moving objects are classified into other vehicles, motorcycles, bicycles, and pedestrians based on the type of traffic participant. Factors for other vehicles include, for example, color, material, paint, surface properties, attached objects, shape, size, relative position, and behavior. Factors for motorcycles include, for example, color, material, attached objects, shape, size, relative position, and behavior. Factors for bicycles include, for example, color, material, attached objects, shape, size, relative position, and behavior. Factors for pedestrians include, for example, the color and material of what they are wearing, posture, shape, size, relative position, and behavior. The influence of these factors on each external environmental sensor 41 according to its specific sensor mechanism can be assumed as a perception disturbance.

[0098] Sensor mechanisms that generate recognition disturbances are classified into recognition processing and others. Disturbances that occur in recognition processing are classified into disturbances related to signals from the object being recognized and disturbances that interfere with signals from the object being recognized. Disturbances that interfere with signals from the object being recognized include, for example, noise and unwanted signals.

[0099] In particular, in camera recognition processing, the physical quantities that characterize the signal of the object being recognized include, for example, intensity, direction, range, signal change, and acquisition time. Noise and unwanted signals can result in either low contrast or high noise.

[0100] In particular, in LiDAR recognition processing, the physical quantities that characterize the signal of the object to be recognized are, for example, scan timing, intensity, propagation direction, and velocity. Noise and unwanted signals are, for example, DC noise, pulsed noise, multiple reflections, and reflections or refractions from objects other than the object to be recognized.

[0101] In particular, with millimeter-wave radar, another type of disturbance is one caused by the orientation of the sensor. In millimeter-wave radar recognition processing, the physical quantities that characterize the signal of the object to be recognized are, for example, frequency, phase, and intensity. Noise and unwanted signals include, for example, small signal loss due to circuit signals, phase noise components of unwanted signals or signal silencing due to radio interference, and unwanted signals from sources other than the object to be recognized.

[0102] Blind spot scenarios are classified into three categories: surrounding vehicles, road structure, and road shape. In blind spot scenarios caused by surrounding vehicles, surrounding vehicles may induce blind spots that affect other vehicles. For this reason, the position of surrounding vehicles may be based on an extended definition that extends to adjacent positions in the eight surrounding directions. In blind spot scenarios caused by surrounding vehicles, the blind spot vehicle motions that may occur are classified into cut-in, cut-out, acceleration, deceleration, and synchronization.

[0103] Blind spot scenarios due to road structure are defined by considering the location of road structures and the relative movement patterns between the vehicle itself and other vehicles present in the blind spot or hypothetical other vehicles assumed to be in the blind spot. Blind spot scenarios due to road structure are classified into blind spot scenarios due to external barriers and blind spot scenarios due to internal barriers. For example, external barriers create blind spots on curves.

[0104] Blind spot scenarios due to road shape are classified into longitudinal gradient scenarios and adjacent lane gradient scenarios. Longitudinal gradient scenarios create blind spots in front of and behind one or both of the vehicle itself. Adjacent lane gradient scenarios create blind spots due to elevation differences with adjacent lanes at merging and junctioning roads, etc.

[0105] Communication disturbance scenarios are classified into three categories: sensors, environment, and transmitters. Sensor-related communication disturbances are classified into map-based and V2X-based factors. Environment-related communication disturbances are classified into static entities, spatial entities, and dynamic entities. Transmitter-related communication disturbances are classified into other vehicles, infrastructure equipment, pedestrians, servers, and satellites.

[0106] Next, we will explain the vehicle motion disturbance scenario system. Vehicle motion disturbance scenarios are classified into two categories: vehicle body input and tire input. Vehicle body input is an input in which an external force acts on the vehicle body and affects motion in at least one of the longitudinal, lateral, and yaw directions. The elements that affect the vehicle body are classified into road geometry and natural phenomena. Road geometry includes, for example, the superelevation of curves, longitudinal gradient, curvature, etc. Natural phenomena include, for example, crosswinds, tailwinds, headwinds, etc.

[0107] Tire input is an input that alters the force generated by the tire and affects motion in at least one of the following directions: longitudinal, lateral, vertical, and yaw. Factors that affect the tire are classified into road surface conditions and tire conditions.

[0108] Road surface conditions include, for example, the coefficient of friction between the road surface and the tire, and the external forces on the tire. Here, road surface factors that affect the coefficient of friction can be classified into, for example, wet roads, icy roads, snow-covered roads, partial gravel, road markings, etc. Road surface factors that affect the external forces on the tire include, for example, potholes, bumps, steps, ruts, joints, grooves, etc. Tire conditions include, for example, punctures, bursts, tire wear, etc.

[0109] Scenario DB53 may include at least one of the following: a functional scenario, a logical scenario, and a concrete scenario. A functional scenario defines the highest-level qualitative scenario structure. A logical scenario is a scenario that adds quantitative parameter ranges to a structured functional scenario. A concrete scenario defines the boundary for safety determination that distinguishes between safe and unsafe states.

[0110] An unsafe state is, for example, a hazardous situation. Furthermore, the range corresponding to a safe state may be called the safe range, and the range corresponding to an unsafe state may be called the unsafe range. Additionally, conditions in a scenario that contribute to dangerous behavior of vehicle 1, or to the inability to prevent, detect, or mitigate reasonably foreseeable misuse, may be trigger conditions.

[0111] Scenarios can be classified as either known or unknown, and also as either dangerous or not dangerous. In other words, scenarios can be classified into known dangerous scenarios, known non-dangerous scenarios, unknown dangerous scenarios, and unknown non-dangerous scenarios.

[0112] Scenario DB53 may be used to make environmental decisions in operating system 2, as described above, but it may also be used for verification and validation of operating system 2. The method for verifying and validating operating system 2 may be rephrased as the method for evaluating operating system 2.

[0113] <Peace of mind and safety> Driving system 2 estimates the situation and controls the behavior of vehicle 1. Driving system 2 is configured to avoid accidents and dangerous situations that could lead to accidents as much as possible, and to maintain a safe situation or safety. Dangerous situations may be caused as a result of the maintenance status of vehicle 1 or a malfunction of driving system 2. Dangerous situations may also be caused by external factors such as other road users. Driving system 2 is configured to maintain safety by reacting to events that make it impossible to maintain a safe situation due to external factors such as other road users and changing the behavior of vehicle 1.

[0114] The driving system 2 has control capabilities to stabilize the behavior of its own vehicle 1 to a safe state. The safe state depends not only on the behavior of its own vehicle 1 but also on the circumstances. If it is not possible to control the behavior of its own vehicle 1 to a safe state, the driving system 2 will behave in a way that minimizes the harm or risk of an accident. Here, the harm of an accident may mean the damage or magnitude of the damage inflicted on traffic participants (road users) when a collision occurs. Risk may be based on the magnitude and likelihood of harm, for example, the product of the magnitude and likelihood of harm.

[0115] Behavior that minimizes the harm or risk of an accident, or the best way to derive such behavior, may be referred to as best effort. Best effort may include best effort that the automated driving system can guarantee to minimize the severity or risk of an accident (hereinafter, best effort that guarantees minimum risk). Guaranteed best effort may mean minimal risk maneuver (MRM) or DDT fallback. Best effort may also include best effort that does not guarantee minimizing the harm or risk of an accident, but attempts to reduce and minimize the severity or risk of an accident to the extent that it is controllable (hereinafter, best effort that does not guarantee minimum risk).

[0116] Figure 4 illustrates the control state space SP, which spatially represents the control state of the vehicle. The driving system 2 may have control performance that stabilizes the behavior of its own vehicle 1 within a range that has a greater safety margin than the performance limit of the system that can ensure safety. The performance limit of the system that can ensure safety may be the boundary between a safe state and an unsafe state, that is, the boundary between a safe range and an unsafe range. The operational design domain (ODD) in the driving system 2 is typically set within the performance limit range R2, and more preferably outside the range of the stably controllable range R1.

[0117] A range that provides a safety margin beyond the performance limit may be referred to as the stable range. Within the stable range, the driving system 2 can maintain a safe state with nominal operation as designed. A state in which a safe state can be maintained with nominal operation as designed may be referred to as a stable state. A stable state can provide the occupants with "the usual sense of security." Here, the stable range may also be referred to as the stable controllable range R1, in which stable control is possible.

[0118] Furthermore, outside the stable control range R1 and within the performance limit range R2, the driving system 2 can return control to a stable state, provided that the environmental assumptions hold true. These environmental assumptions may be, for example, reasonably foreseeable assumptions. For example, the driving system 2 can react to the reasonably foreseeable behavior of road users, etc., to change the behavior of its own vehicle 1 to avoid a dangerous situation and return to stable control. The state in which control can be returned to a stable state can provide "safety in case of emergency" to the occupants, etc.

[0119] In the driving system 2, the decision unit 20 may decide whether to continue stable control or transition to the minimum risk condition (MRC) within the performance limit range R2 (in other words, before going outside the performance limit range R2). The minimum risk condition may also be a fallback condition. The decision unit 20 may also decide whether to continue stable control or transition to the minimum risk condition outside the stable controllable range R1 and within the performance limit range R2. Transitioning to the minimum risk condition may be the execution of MRM or DDT fallback.

[0120] Furthermore, for example, when performing autonomous driving of a Level 3 autonomous driving system, the decision unit 20 may perform a delegation of authority to the driver, such as a takeover. If driving is not handed over from the autonomous driving system to the driver, control may be employed to perform MRM or DDT fallback.

[0121] The judgment unit 20 may determine a state transition of driving behavior based on the situation estimated by the environmental judgment unit 21. A state transition of driving behavior may refer to a transition in the behavior of the vehicle 1 realized by the driving system 2, for example, a transition between behavior that maintains the consistency and predictability of the rules and the reactive behavior of the vehicle 1 in response to external factors such as other road users. In other words, a state transition of driving behavior may refer to a transition between action and reaction. Furthermore, the judgment of a state transition of driving behavior may refer to a judgment of whether to continue stable control or to move to the minimum risk condition. Stable control may refer to a state in which the behavior of the vehicle 1 does not fluctuate, does not suddenly accelerate, does not suddenly brake, etc., or occurs with extremely low frequency. Stable control may refer to a level of control in which a human driver perceives the behavior of the vehicle 1 as stable or free of abnormalities.

[0122] The conditions estimated by the environmental judgment unit 21, that is, the conditions estimated by the electronic system, may include differences from the real world. Therefore, the performance limits in the operating system 2 may be set based on the acceptable range of differences from the real world. In other words, the margin between the performance limit range R2 and the stable controllable range R1 may be defined based on the differences between the conditions estimated by the electronic system and the real world. Here, the differences between the conditions estimated by the electronic system and the real world may be examples of the effects or errors caused by disturbances.

[0123] Here, the conditions used to determine the transition to the minimum risk condition may be recorded in the recording device 55, for example, in a format estimated by the electronic system. In MRM or DDT fallback, if there is interaction between the electronic system and the driver, for example through the HMI device 70, the operation of the driver may be recorded in the recording device 55.

[0124] <Interaction in the driving system> The architecture of the driving system 2 can be represented by the relationship between the abstract layer and the physical interface layer (hereinafter referred to as the physical IF layer) and the real world. Here, the abstract layer and the physical IF layer may also refer to layers composed of electronic systems. As shown in Figure 5, the interaction between the recognition unit 10, the decision unit 20, and the control unit 30 can be represented by a block diagram showing a causal loop.

[0125] In detail, in the real world, the vehicle 1 affects the external environment EE. The recognition unit 10, which belongs to the physical IF layer, recognizes the vehicle 1 and the external environment EE. Errors or deviations may occur in the recognition unit 10 due to misrecognition, observation noise, recognition disturbances, etc. Errors or deviations that occur in the recognition unit 10 affect the decision unit 20, which belongs to the abstract layer. Also, assuming that the control unit 30 acquires the vehicle state for the control of the motion actuator 60, errors or deviations that occur in the recognition unit 10 directly affect the control unit 30, which belongs to the physical IF layer, without going through the decision unit 20. In the decision unit 20, judgment errors, traffic disturbances, etc. may occur. Errors or deviations that occur in the decision unit 20 affect the control unit 30, which belongs to the physical IF layer. When the control unit 30 controls the motion of the vehicle 1, vehicle motion disturbances occur. And again, in the real world, the vehicle 1 affects the external environment EE, and the recognition unit 10 recognizes the vehicle 1 and the external environment EE.

[0126] Thus, the driving system 2 constitutes a causal loop structure that spans across each layer. Furthermore, it constitutes a causal loop structure that moves between the real world, the physical IF layer, and the abstraction layer. Errors or deviations generated in the recognition unit 10, the judgment unit 20, and the control unit 30 can propagate along the causal loop.

[0127] Causal loops are classified into open loops and closed loops. An open loop can be described as a partial loop, a part of a closed loop. Examples of open loops include a loop consisting of a recognition unit 10 and a determination unit 20, and a loop consisting of a determination unit 20 and a control unit 30.

[0128] A closed-loop is a loop configured to circulate between the real world and at least one of the physical IF layer and the abstraction layer. Closed-loops are classified into an inner loop IL that is completed within vehicle 1, and an outer loop EL that includes the interaction between vehicle 1 and the external environment EE.

[0129] The inner loop IL, for example in Figure 6, is a loop that returns to the vehicle 1 via the recognition unit 10 and the control unit 30. As mentioned above, the parameters that directly affect the control unit 30 from the recognition unit 10 are, under one assumption, vehicle conditions such as vehicle speed, acceleration, and yaw rate, and do not include the recognition results of the external environment sensor 41. Therefore, the inner loop IL can be said to be a loop that is completed within the vehicle 1. The outer loop EL, for example in Figure 7, is a loop that returns to the vehicle 1 via the external environment EE, the recognition unit 10, the judgment unit 20, and the control unit 30.

[0130] <Verification and Validation> The verification and validation of the operating system 2 may include an evaluation of at least one, preferably all, of the following functions and capabilities. The functions and capabilities evaluated here may be referred to as verification targets or validation targets.

[0131] For example, the evaluation targets related to the recognition unit 10 include the functionality of the sensor or external data source (e.g., map data source), the functionality of the sensor processing algorithm that models the environment, and the reliability of the infrastructure and communication system.

[0132] For example, an evaluation target related to the decision unit 20 is the capability of the decision algorithm. The capability of the decision algorithm includes the ability to safely handle potential functional deficiencies and the ability to make appropriate decisions according to the environmental model, driving policy, current destination, etc. Another evaluation target related to the decision unit 20 is the absence of unreasonable risks due to dangerous behavior of the intended function, the system's ability to safely handle ODD use cases, the robustness of the driving policy execution across the entire ODD, the suitability of DDT fallback, and the suitability of the minimum risk conditions.

[0133] Another example of an evaluation target is the robustness of the system or function. System or function robustness includes the system's robustness under harsh environmental conditions, the appropriateness of system operation in response to known trigger conditions, the sensitivity of intended functions, and the monitoring capability under various scenarios.

[0134] Next, we will specifically explain some examples of evaluation methods for the driving system 2 using Figures 8 to 13. The evaluation method referred to here may be a configuration method or a design method for the driving system 2. In Figures 8, 10, and 12 below, circles A1, A2, and A3 respectively virtually and schematically show regions where safety cannot be maintained due to factors related to the recognition unit 10, the determination unit 20, and the control unit 30.

[0135] The first evaluation method, as shown in Figure 8, is a method of independently evaluating the recognition unit 10, the determination unit 20, and the control unit 30. That is, the first evaluation method includes individually evaluating the nominal performance of the recognition unit 10, the nominal performance of the determination unit 20, and the nominal performance of the control unit 30. Individual evaluation may mean evaluating the recognition unit 10, the determination unit 20, and the control unit 30 based on mutually different viewpoints and means.

[0136] For example, the control unit 30 may be evaluated based on control theory. The decision unit 20 may be evaluated based on a logical model that demonstrates safety. The logical model may be an RSS (Responsibility Sensitive Safety) model, an SFF (Safety Force Field) model, or the like.

[0137] The recognition unit 10 may be evaluated based on its recognition failure rate. For example, the evaluation criterion may be whether the recognition result of the entire recognition unit 10 is less than or equal to a target recognition failure rate. The target recognition failure rate for the entire recognition unit 10 may be a value smaller than the statistically calculated collision accident occurrence rate for human drivers. The target recognition failure rate may be, for example, 10⁻⁹, which is two orders of magnitude lower than the accident occurrence rate. The recognition failure rate referred to here is a value normalized to 1 if there is a 100% failure rate.

[0138] Furthermore, when multiple subsystems (for example, a camera subsystem, a subsystem of external environmental sensors 41 excluding the camera, and a map subsystem) are configured by multiple sensors 40, reliability may be ensured by a majority vote of the multiple subsystems. Assuming a majority vote of subsystems, the target recognition failure rate for each subsystem may be greater than the target recognition failure rate for the entire recognition unit 10. The target recognition failure rate for each subsystem may be, for example, 10⁻⁵. In the first evaluation method, target values ​​or target conditions may be set based on a positive risk balance.

[0139] An example of the first evaluation method will be explained using the flowchart in Figure 9. The implementing body for each step S11 to S13 is at least one of the following: the vehicle manufacturer, the vehicle designer, the manufacturer of the driving system 2, the designer of the driving system 2, the manufacturers of the subsystems constituting the driving system 2, the designers of those subsystems, a person commissioned by these manufacturers or designers, a testing or certification body for the driving system 2, etc. When the evaluation is performed by simulation, the actual implementing body may be at least one processor. In each step S11 to S13, the implementing bodies may be common bodies or different bodies.

[0140] In S11, the nominal performance of the recognition unit 10 is evaluated. In S12, the nominal performance of the determination unit 20 is evaluated. In S13, the nominal performance of the control unit 30 is evaluated. The order of S11 to S13 can be changed as appropriate, and they can also be performed simultaneously.

[0141] The second evaluation method, as shown in Figure 10, includes evaluating the nominal performance of the decision unit 20 and evaluating the robust performance of the decision unit 20 by considering at least one of the errors of the recognition unit 10 and the control unit 30. This evaluation method may further include evaluating the nominal performance of the recognition unit 10 and the nominal performance of the control unit 30 as prerequisites. The nominal performance of the decision unit 20 may be evaluated based on the traffic disturbance scenario described above.

[0142] The robust performance of the decision unit 20 may be evaluated by verifying traffic disturbance scenarios in which the error range is specified using a physically based error model that represents the error of the recognition unit 10, such as sensor errors. For example, traffic disturbance scenarios under environmental conditions in which recognition disturbances occur can be evaluated. This allows the second evaluation method to include the region A12 in which circle A1 of the recognition unit 10 and circle A2 of the decision unit 20 overlap, as shown in Figure 10, or in other words, the combined factors of the recognition unit 10 and the decision unit 20, as the target of evaluation. The evaluation of the combined factors of the recognition unit 10 and the decision unit 20 may be achieved by evaluating an open loop that goes directly from the recognition unit 10 to the decision unit 20 in the causal loop described above.

[0143] The robust performance of the decision unit 20 may be evaluated by verifying traffic disturbance scenarios in which the error range is specified using a physically based error model that represents the error of the control unit 30, such as vehicle motion errors. For example, traffic disturbance scenarios under environmental conditions in which vehicle motion disturbances occur can be evaluated. This allows the second evaluation method to include the region A23 in which the circle A2 of the decision unit 20 and the circle A3 of the control unit 30 shown in Figure 12 overlap, in other words, the combined factors of the decision unit 20 and the control unit 30, as the target of evaluation. The evaluation of the combined factors of the decision unit 20 and the control unit 30 may be achieved by an open-loop evaluation that goes directly from the decision unit 20 to the control unit 30 in the causal loop described above.

[0144] An example of the second evaluation method will be explained using the flowchart in Figure 11. The implementing body for S21-24 is at least one of the following: for example, the vehicle manufacturer, the vehicle designer, the manufacturer of the driving system 2, the designer of the driving system 2, the manufacturers of the subsystems constituting the driving system 2, the designers of the subsystems, those commissioned by these manufacturers or designers, the testing or certification body for the driving system 2, etc. When the evaluation is carried out by simulation, the actual implementing body may be at least one processor. In each step of S21-24, the implementing bodies may be common to all of them or different to each other.

[0145] In S21, the nominal performance of the recognition unit 10 is evaluated. In S22, the nominal performance of the control unit 30 is evaluated. In S23, the nominal performance of the decision unit 20 is evaluated. In S24, the robust performance of the decision unit 20 is evaluated, taking into account the errors of the recognition unit 10 and the control unit 30. The order of S21 to S24 can be changed as appropriate, and they can also be performed simultaneously.

[0146] The third evaluation method includes, as shown in Figure 12, regions A12, A23, A13, and AA, which are areas where at least two of the circles A1 of the recognition unit 10, A2 of the judgment unit 20, and A3 of the control unit 30 overlap, as the evaluation target. The third evaluation method first includes evaluating the nominal performance of the recognition unit 10, the nominal performance of the judgment unit 20, and the nominal performance of the control unit 30. The evaluation of nominal performance may be based on the first evaluation method itself, or a part of the first evaluation method may be used. On the other hand, a method completely different from the first evaluation method may be used to evaluate nominal performance.

[0147] Furthermore, the third evaluation method includes focusing on evaluating the robust performance of the recognition unit 10, the determination unit 20, and the control unit 30, specifically by evaluating composite factors that combine at least two of the recognition unit 10, the determination unit 20, and the control unit 30. Here, the composite factors of at least two of the recognition unit 10, the determination unit 20, and the control unit 30 are the composite factors of the recognition unit 10 and the determination unit 20, the composite factors of the determination unit 20 and the control unit 30, the composite factors of the recognition unit 10 and the control unit 30, and the three composite factors of the recognition unit 10, the determination unit 20, and the control unit 30.

[0148] Focusing on evaluating multiple factors may involve, for example, extracting specific conditions with relatively large interactions between the recognition unit 10, the decision unit 20, and the control unit 30 on a scenario basis, and evaluating those specific conditions in more detail than other conditions with relatively small interactions. Detailed evaluation may include at least one of the following: evaluating the specific condition in more detail than other conditions, or increasing the number of tests. The conditions to be evaluated (e.g., the specific condition and other conditions mentioned above) may include trigger conditions. Here, the magnitude of the interaction may be determined using the causal loop described above.

[0149] Some of the evaluation methods described above may include defining the object to be evaluated, designing a test plan based on the definition of the object to be evaluated, and executing the test plan to demonstrate the absence of unreasonable risks from known or unknown hazardous scenarios. The tests may be physical tests, simulation tests, or a combination of physical and simulation tests. Physical tests may be, for example, field operational tests (FOTs). Target values ​​in FOTs may be set using FOT data, etc., in the form of the number of failures that can be tolerated for a given mileage of the test vehicle (e.g., tens of thousands of kilometers).

[0150] An example of the third evaluation method will be explained using the flowchart in Figure 13. The implementing body for steps S31 to S34 is at least one of the following: for example, the vehicle manufacturer, the vehicle designer, the manufacturer of the driving system 2, the designer of the driving system 2, the manufacturers of the subsystems constituting the driving system 2, the designers of the subsystems, a person commissioned by these manufacturers or designers, or a testing or certification body for the driving system 2. When the evaluation is performed by simulation, the actual implementing body may be at least one processor. In each step of S31 to S34, the implementing bodies may be common to all of them or different to each other.

[0151] In S31, the nominal performance of the recognition unit 10 is evaluated. In S32, the nominal performance of the determination unit 20 is evaluated. In S33, the nominal performance of the control unit 30 is evaluated. In S34, the robust performance is evaluated with a focus on the composite regions A12, A23, A13, and AA. The order of S31 to S34 can be changed as appropriate, and they can also be performed simultaneously.

[0152] <Evaluation Strategy for Driving Systems> The evaluation strategy for the operating system 2 includes a pre-evaluation strategy and a post-evaluation strategy. The pre-evaluation strategy may include selecting the optimal method from among multiple evaluation methods, such as the first evaluation method, second evaluation method, third evaluation method, and other evaluation methods described above, that enhances or guarantees at least one of the performance and validity of the operating system 2.

[0153] The pre-evaluation strategy may be one in which the recognition unit 10, the decision unit 20, and the control unit 30 are each evaluated independently, as shown in the first evaluation method. This strategy can be implemented by an open-loop approach to evaluate nominal performance.

[0154] Furthermore, the pre-evaluation strategy may be one that focuses on evaluating the combined factors resulting from the combination of the recognition unit 10 and the determination unit 20, and the combined factors resulting from the combination of the determination unit 20 and the control unit 30, as shown in the second evaluation method. This strategy can be implemented by including an open-loop approach to evaluate robust performance.

[0155] Furthermore, the pre-evaluation strategy may be one that focuses on evaluating the combined factors resulting from the combination of the control unit 30 and the recognition unit 10, and the combined factors resulting from the combination of the recognition unit 10, the judgment unit 20, and the control unit 30. This strategy can be realized in the concretization of the third evaluation method by including an approach that evaluates robust performance in a closed loop. More specifically, the evaluation of the combined factors resulting from the combination of the control unit 30 and the recognition unit 10 can be realized by evaluation in an inner loop IL that is completed within the vehicle 1. The evaluation of the combined factors resulting from the combination of the recognition unit 10, the judgment unit 20, and the control unit 30 can be realized by evaluation in an outer loop EL that includes the interaction between the vehicle 1 and the external environment EE.

[0156] The following sections will provide a detailed explanation of an evaluation method for assessing robust performance using a closed-loop approach, a design method for the operating system 2 using this evaluation method, and several specific examples of the operating system 2 realized through these methods.

[0157] <Assignment of confidence level / Assignment of tolerance> The first design method is a design method that takes into account the division of responsibilities among each subsystem (i.e., the recognition system 10a, the decision system 20a, and the control system 30a), and is a design method based on the assignment of reliability to each subsystem. When evaluating complex factors, it is preferable to use a unified index among each subsystem. A unified index is, for example, reliability.

[0158] Therefore, in this design method and the evaluation method used in the design, reliability can be newly introduced as an indicator for evaluating the control unit 30. Furthermore, the concept of probabilistic robust control is introduced, such that the operating system 2 keeps the allowable error below ε with a probability of (1-δ) or higher. This concept of probabilistic robust control may be just one example of an operating policy. In this way, when using evaluation based on a combination of reliability and allowable error, the need to calculate the probability distribution of errors propagating through the recognition unit 10, the judgment unit 20, and the control unit 30 can be avoided. Consequently, the evaluation load can be reduced.

[0159] The reliability of the driving system 2 may be set based on technical or social grounds. For example, the reliability of the driving system 2 may be set to a value less than or equal to the statistically calculated collision accident rate for human drivers.

[0160] In probabilistic robust control, reliability has a greater impact on comfort than error, while error has a greater impact on safety than reliability. By separating and evaluating reliability and error separately, comfort and safety in the driving system 2 can be optimized. Reliability is assigned to each subsystem based on the safety specifications required for the driving system 2. Therefore, the first design method based on reliability assignment can be described as a top-down design method that breaks down the specifications of the entire driving system 2 into the specifications of each subsystem.

[0161] If the reliability required for operating system 2 is directly applied to the reliability of each subsystem, the performance required for each subsystem will be high. Therefore, by allocating, or distributing, the reliability of operating system 2 to each subsystem, it is possible to avoid requiring excessive performance from each subsystem.

[0162] Here, an example of an evaluation method used in the first design method will be explained using the flowchart in Figure 14. The implementing body for S101 to S104 is at least one of the following: the vehicle manufacturer, the vehicle designer, the manufacturer of the driving system 2, the designer of the driving system 2, the manufacturers of the subsystems constituting the driving system 2, the designers of the subsystems, a person commissioned by these manufacturers or designers, a testing or certification body for the driving system 2, etc. When the evaluation is performed by simulation, the actual implementing body may be, for example, the evaluation device 81 or the design device 82 shown in Figure 15. In each step of S101 to S104, the implementing bodies may be common to all of them or different to each other.

[0163] The evaluation device 81 comprises at least one memory 81a and at least one processor 81b, and the evaluation function is realized by the execution of a program stored in the memory 81a by at least one processor 81b. The memory 81a may be at least one type of non-transitional physical storage medium, such as semiconductor memory, magnetic media, and optical media, which non-temporarily stores programs and data that can be read by a computer (in this case, for example, the processor 81b). The processor 81b includes at least one type as a core, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a RISC (Reduced Instruction Set Computer)-CPU. Furthermore, the evaluation device 81 may have an interface that allows it to connect and communicate with another computer located outside the device that reproduces the operating system 2 or its architecture during evaluation. In addition, the evaluation device 81 may further include a scenario DB 53 used to define the assumptions for the simulation during evaluation.

[0164] The design apparatus 82 comprises at least one memory 82a and at least one processor 82b, and the design function is realized by the execution of a program stored in the memory 82a by the at least one processor 82b. The memory 82a may be at least one type of non-transitional physical storage medium, such as semiconductor memory, magnetic media, and optical media, which non-temporarily stores programs and data that can be read by a computer (in this case, for example, the processor 82b). The processor 82b includes at least one type as a core, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a RISC (Reduced Instruction Set Computer)-CPU. The design function may also include an evaluation function. Furthermore, the design apparatus 82 may have an interface that can communicate with another computer located outside the apparatus that reproduces the architecture of the operating system 2. In addition, the design apparatus 82 may further include a scenario DB 53 used to define the assumptions for simulation during evaluation. The memories 81a and 82a may be implemented as storage media independently provided outside the devices 81 and 82 and configured to be readable by other computers.

[0165] In S101, the interaction between each subsystem and the real world is modeled as a loop structure. Based on the architecture of the operating system 2 being evaluated, causal loops spanning the abstraction layer, physical IF layer, and the real world are modeled, for example, as shown in Figure 5. The causal loops may be modeled in more detail to more faithfully reproduce the complexity of the architecture (see example in Figure 18).

[0166] This identifies at least one closed loop. For example, as shown in Figures 6 and 7, two closed loops are identified: the outer loop EL and the inner loop IL. After S101, proceed to S102.

[0167] In S102, a reliability metric is introduced as a unified indicator for each subsystem. After S102, proceed to S103.

[0168] In S103, errors occurring in each subsystem are identified. For example, as shown in Figure 5, errors caused by misrecognition in the recognition unit 10, errors caused by judgment errors in the judgment unit 20, and errors caused by vehicle motion disturbances in the control unit 30 are identified. These errors may include errors based on quantitative errors and errors based on qualitative errors, as will be described later. These errors may be identified individually for each scenario based on the scenario-based approach described above. These errors may also be identified based on the relationship with the ODD.

[0169] Furthermore, as shown in Figure 16, these errors may have a boundary value ε set in the probability density function representing the error distribution, where the probability of the error is 1-δ, which corresponds to the confidence level. After S103, proceed to S104.

[0170] In S104, the closed-loops identified in S101 are evaluated based on the confidence level introduced in S102. If multiple closed-loops are identified, evaluation may be performed on all of them. On the other hand, evaluation of some closed-loops with a small impact as a combined factor may be omitted.

[0171] The closed-loop evaluation based on confidence involves, for example, evaluating the error propagating through the closed loop based on probabilistic robust control. That is, it can be evaluated whether the error propagating according to the closed loop falls within the acceptable error with a probability of a predetermined confidence level or higher. This evaluation may also be performed using equations 1 to 4 described later. The series of evaluations ends with S104. Note that the order of S101 to S103 can be changed as appropriate, and they can also be performed simultaneously.

[0172] Next, an example of the first design method will be explained using the flowchart in Figure 17. The implementing body for steps S111 to S114 may be, for example, the vehicle manufacturer, the vehicle designer, the manufacturer of the driving system 2, the designer of the driving system 2, the manufacturer of the subsystems constituting the driving system 2, the designer of the subsystems, or a person commissioned by these manufacturers or designers. The implementing body may also be the design device 82. In each step of S111 to S114, the implementing bodies may be common bodies or different bodies.

[0173] In S111, the overall specifications of the driving system 2 are determined. The overall specifications here may include the overall architecture of the driving system 2, which is comprised of its constituent components. The overall specifications do not need to include detailed specifications of subsystem components, such as the detailed specifications of the camera. After S111, the program proceeds to S112.

[0174] In S112, a reliability level is assigned to each subsystem of the recognition system 10a, the decision system 20a, and the control system 30a, based on the overall specifications of the driving system 2 determined in S111. The reliability level may be assigned as a uniform, fixed value, independent of the ODD, scenario, etc. This assignment may be called a static assignment.

[0175] On the other hand, individual values ​​may be assigned to each assignment category, such as ODD and scenario. This assignment may be called a dynamic assignment. For example, if excessive reliability is required for the recognition system 10a in a recognition disturbance scenario, an extremely high performance specification will be required for the external environment sensor 41, leading to increased costs for the operating system 2. For this reason, in a recognition disturbance scenario, an assignment may be made that reduces the reliability of the recognition system 10a and increases the reliability of the decision system 20a and the control system 30a accordingly.

[0176] The assignment categories may be further subdivided. For example, in the communication disturbance scenario among the recognition disturbance scenarios, the information in the map DB 44 may not be able to be updated to the latest information. In this case, it is difficult to require excessive confidence in the map DB 44. Therefore, the assignment may be changed so that the confidence assigned to the map DB 44 is reduced and the confidence assigned to other external environmental sensors 41 such as cameras, or the decision system 20a and control system 30a, etc. After S112, proceed to S113.

[0177] In S113, the allowable error distribution or tolerance for each subsystem is calculated based on the confidence level assigned in S112. The closed-loop evaluation method shown in S101-104 can be used to calculate this error distribution or tolerance. After S113, proceed to S114.

[0178] In S114, the specifications of each subsystem are determined based on the error distribution or tolerance calculated in S113. In other words, each subsystem is designed to achieve the error distribution or tolerance allowed for that subsystem. The series of processes ends with S114.

[0179] The second design method is a design method that uses the sensitivity of the operating system 2 and is based on the assignment of tolerance errors to each subsystem. This design method includes, for example, evaluating the propagating errors in the causal loop structure shown in Figure 5.14.

[0180] For example, the causal loop structure in Figure 18 is a more concrete representation of the causal loop structure in Figure 5. In Figure 18, the self-position estimation block 10y corresponds to the self-position recognition unit 12 and the internal recognition unit 14 of the recognition unit 10. The object recognition / path recognition block 10x corresponds to the external recognition unit 11 and the fusion unit 13 of the recognition unit 10. The action planning / trajectory generation block 20x corresponds to the decision unit 20. The position control / attitude control block 30x corresponds to the motion control unit 31 of the control unit 30.

[0181] In this causal loop structure, there are two closed loops: an inner loop IL that is completed within the vehicle 1, and an outer loop EL that includes the interaction between the vehicle 1 and the external environment EE. The inner loop IL shown in Figure 19 is a loop that returns to the vehicle 1 from the vehicle 1 via the self-position estimation block 10y and the position control / attitude control block 30x. The outer loop EL shown in Figure 20 is a loop that returns to the vehicle 1 from the vehicle 1 via the external environment EE, object recognition / path recognition block 10x, action planning / trajectory generation block 20x, and position control / attitude control block 30x.

[0182] Furthermore, as shown in Figure 21, an actual vehicle has a closed loop (hereinafter referred to as the vehicle body stabilization loop SL) that occurs within the vehicle body 1, or between the vehicle body 1 and the control unit 30. The vehicle body stabilization loop SL can be achieved, for example, by stabilizing the vehicle body through motor control or suspension control in the powertrain.

[0183] As shown in Figure 18, various errors can be input into the causal loop. In the object recognition / path recognition block 10x, errors classified as misrecognition may occur. In the self-position estimation block 10y, errors classified as observation noise may occur. In the action planning / trajectory generation block 20x, errors classified as judgment errors may occur. In the position control / attitude control block 30x, errors classified as vehicle motion disturbances may occur. Misrecognition and observation noise may be replaced with the recognition disturbances described above. Judgment errors may be replaced with the traffic disturbances described above.

[0184] As shown in Figure 22, the targets of misrecognition are, for example, object recognition and path recognition. Quantitative errors in misrecognition include, for example, errors in object position and velocity. Qualitative errors in misrecognition include, for example, failure to detect, false detection, and misinterpretation. The targets of observation noise are, for example, self-localization estimation. Quantitative errors in observation noise include, for example, errors in self-localization and attitude.

[0185] The areas affected by judgment errors are action plans and trajectory generation. Quantitative errors in judgment errors include, for example, errors in the target trajectory. Qualitative errors in judgment errors include, for example, incorrect scenario selection and incorrect mode selection.

[0186] Vehicle motion disturbances affect position control and attitude control. Quantitative errors in vehicle motion disturbances include, for example, errors in control inputs.

[0187] Quantitative errors can be directly expressed as errors using numerical values ​​corresponding to physical quantities. Furthermore, quantitative errors can be evaluated by the probability that the error falls within the acceptable margin of error. Here, probability corresponds to confidence level.

[0188] On the other hand, qualitative errors can be expressed as errors using discrete values ​​such as True or False (T / F) or 1 or 0. Errors expressed in this way, when statistically collected and processed for each event, ultimately represent the reliability. Qualitative errors in observational noise and vehicle motion disturbances do not need to be considered. If an unknown qualitative error is discovered, it can be evaluated using reliability, similar to other qualitative errors.

[0189] Here, assuming that each subsystem can be linearized, we consider the sensitivity to various errors using a sensitivity function and a complementary sensitivity function. For example, as shown in Figure 18, the transfer function from the target value to the output in each block of the causal loop is assumed to be P for the vehicle 1, E for the external environment EE, L for the self-position estimation block 10y, S for the object recognition / path recognition block 10x, D for the action planning / trajectory generation block 20x, and K for the position control / attitude control block 30x, respectively.

[0190] In the following, "error" will be used to mean the numerical value of the error, and "deviation" will be used to mean the difference between the target value and the output value that appears in the operating system 2 due to the error.

[0191] However, if the context does not distinguish between error and deviation, the term "error" may refer to a concept that includes both a numerical value representing the error and the difference between the target value and the output value that appears in the operating system 2 as a result of that numerical value representing the error.

[0192] If we denote the error due to vehicle motion disturbances as d, the resulting deviation from the target value can be expressed as shown in Equation 1 below.

[0193]

number

[0194] If we denote the error due to misrecognition as m, the resulting deviation from the target value can be expressed as shown in equation 2 below.

[0195]

number

[0196]

number

[0197]

number

[0198] The transfer function E of the external environment EE may be set based on a combination with the transfer function D of the action plan. For example, in the traffic disturbance scenario described above, the interaction between a certain action or reaction of vehicle 1 and external factors such as other road users may be functioned, which may substantially correspond to setting the transfer function E of the external environment EE.

[0199] Furthermore, the transfer function E of the external environment EE may be set according to the assumption that external factors such as other road users will act or react based on reasonably foreseeable assumptions, for example, following safety-related models.

[0200] On the other hand, it is preferable to set the transfer function E of the external environment EE and the transfer function D of the action plan as separate and independent functions.

[0201] Each subsystem has errors that may occur due to specifications or technical reasons. When the allowable deviation e_max for the entire operating system 2 is determined, these errors d, m, n, and j will be readjusted so that they do not exceed the maximum allowable errors d_max, m_max, n_max, and j_max calculated from the deviations assigned to each subsystem using equations 1 to 4. Therefore, the second design method based on error assignment can be said to be a bottom-up design method in which the specifications of the entire operating system 2 are adjusted after the specifications of each subsystem have been determined.

[0202] Here, an example of an evaluation method used in the first design method will be explained using the flowchart in Figure 23. The implementing body for S121 to S124 is at least one of the following: the vehicle manufacturer, the vehicle designer, the manufacturer of the driving system 2, the designer of the driving system 2, the manufacturers of the subsystems constituting the driving system 2, the designers of the subsystems, a person commissioned by these manufacturers or designers, a testing or certification body for the driving system 2, etc. When the evaluation is performed by simulation, the actual implementing body may be, for example, the evaluation device 81 or the design device 82 shown in Figure 15. In each step of S121 to S124, the implementing bodies may be common to all of them or different to each other.

[0203] In S121, the interaction between each subsystem and the real world is modeled as a loop structure, similar to the method used in S101. This identifies at least one closed loop. After S121, the program proceeds to S122.

[0204] In S122, the allowable deviation e_max for the entire operating system is determined. After S122, proceed to S123.

[0205] In S123, errors that occur in relation to each subsystem are identified. The method of identifying these errors varies depending on the intent and purpose of the evaluation. For example, if the goal is to evaluate the deviation that occurs in operating system 2 under the current subsystem specifications or performance, the errors are set based on the current subsystem specifications or performance.

[0206] In S124, the closed loop identified in S121 is evaluated based on the allowable deviation e_max identified in S122. If multiple closed loops are identified, evaluation may be performed for all of them. On the other hand, evaluation of some closed loops with a small influence as a combined factor may be omitted. The series of evaluations ends with S124. Note that the order of S121 to S123 can be changed as appropriate, and they can also be performed simultaneously.

[0207] Next, an example of the second design method will be explained using the flowchart in Figure 24. The implementing bodies for S131 to S136 may be, for example, the vehicle manufacturer, the vehicle designer, the manufacturer of the driving system 2, the designer of the driving system 2, the manufacturer of the subsystems constituting the driving system 2, the designer of the subsystems, or persons commissioned by these manufacturers or designers. The actual implementing body may be the design device 82. In each step of S131 to S136, the implementing bodies may be common bodies or different bodies.

[0208] In S131, each subsystem is provisionally designed. For each provisionally designed subsystem, errors based on their respective performance are identified. After S131, proceed to S132.

[0209] In S132, the permissible deviation for the entire operating system 2 is identified. This permissible deviation may be determined based on the specifications of the entire operating system 2. For example, the permissible deviation may be determined by working backward from the positive risk balance to find a safe margin. After S132, the process proceeds to S133.

[0210] In S133, the allowable deviation for each subsystem is provisionally allocated based on the allowable deviation of the entire driving system 2. Here, the provisional allocation may be an equal allocation to each subsystem. An equal allocation means that, of the allowable deviation of the entire driving system 2, the recognition system 10a is effectively responsible for 1 / 3 (33%), the decision system 20a is effectively responsible for 1 / 3 (33%), and the control system 30a is effectively responsible for 1 / 3 (33%). In the case where the recognition system 10a is divided into an object recognition / path recognition block 10x and a self-position estimation block 10y, as shown in Figure 17, the deviation that the recognition system 10a is responsible for may be further distributed between the object recognition / path recognition block 10x and the self-position estimation block 10y.

[0211] If an appropriate allocation has been determined empirically, that empirically obtained allocation may be used provisionally. After S133, proceed to S134.

[0212] In S134, by working backward from equations 1 to 4, which mathematically represent the errors propagating in a closed loop, the maximum allowable errors d_max, m_max, n_max, and j_max required for each subsystem can be calculated from the allowable deviations of each subsystem. After S134, proceed to S135.

[0213] In S135, it is determined for each subsystem whether the errors d, m, n, and j identified in S131 fall within the maximum allowable errors d_max, m_max, n_max, and j_max provisionally assigned to that subsystem. If all subsystems are found to be in a positive state, the assignment of allowable errors for each system is finalized, and the series of processes ends. If at least one subsystem is found to be in a negative state, the process proceeds to S136.

[0214] In S136, the allocation to each subsystem is adjusted. Specifically, the allocation to subsystems whose errors exceeded the tolerance in S135 is increased, and the allocation to subsystems whose errors fell within the tolerance is decreased.

[0215] For example, consider the case where each subsystem is provisionally allocated equally in S132. Suppose in S135 it is determined that the error of the recognition system 10a falls within the provisionally allocated tolerance for the recognition system 10a, and that the error of the control system 30a falls within the provisionally allocated tolerance for the control system 30a. On the other hand, suppose in S135 it is determined that the error of the decision system 20a exceeds the provisionally allocated tolerance for the decision system 20a. In this case, adjustments may be made, for example, by reducing the allocation to the recognition system 10a to 20%, increasing the allocation to the decision system 20a to 60%, and decreasing the allocation to the control system 30a to 20%. After S136, the process returns to S134.

[0216] By repeatedly performing adjustments in S134-S136, if a solution is found in which the errors d, m, n, and j generated in all subsystems fall within the allowable errors d_max, m_max, n_max, and j_max, then the allowable error assignments for each subsystem can be finalized at that point. On the other hand, if a solution is not found in which the errors d, m, n, and j generated in all subsystems fall within the allowable errors d_max, m_max, n_max, and j_max, then the specifications of at least one subsystem need to be reviewed. In other words, the performance of the subsystem needs to be revised to a higher level in order to reduce the errors generated.

[0217] The first and second design methods may be implemented selectively. On the other hand, if the first and second design methods are implemented in combination, a more reasonable operating system 2 can be designed. For example, by assigning tolerances using the second design method and then assigning reliability using the first design method, an operating system 2 in which both tolerances and reliability are optimized may be designed. For example, by assigning reliability using the first design method and then assigning tolerances using the second design method, an operating system 2 in which both tolerances and reliability are optimized may be designed.

[0218] <An operating system realized through the evaluation of multiple factors> An operating system 2 designed according to the design method described above, and in particular an operating system 2 that performs a processing method using an assigned reliability level, will be described.

[0219] The operating system 2 stores the dynamic reliability assignments for each assignment category, which were determined during the design phase. The storage medium (e.g., non-transitional physical storage medium) that stores the reliability assignments may be one or more. The storage medium may be the memory 51a of the dedicated computer 51 of the processing system 50, the scenario DB 53, or the memory 55a of the recording device 55.

[0220] The driving system 2 modifies the conditions for executing dynamic driving tasks by referring to the confidence level assignment for each assignment category. The assignment categories are set based on types such as ODD use cases and scenarios. In other words, while vehicle 1 is in motion, the confidence level assignment in the driving system 2 changes dynamically in effect according to the situation in which vehicle 1 is currently located.

[0221] For example, the driving system 2 may decide which components of the driving system 2 will be the main components used to realize the dynamic driving task, depending on the ODD, scenario, etc. In other words, the driving system 2 may flexibly switch the combination of main components to realize the dynamic driving task, depending on the ODD, scenario, etc. Some of the multiple sensors 40 that realize the recognition system 10a may be selected as the main components. For example, in the interpretation of the environmental model, the contribution of the recognition result of the main components is increased compared to other components. The combinations referred to here include, for example, a combination of camera, map and control, a combination of millimeter-wave radar, map and control, a combination of camera, millimeter-wave radar and control, etc.

[0222] For example, the driving system 2 may decide whether or not to plan a cautious control action based on the product of the reliability of the recognition system 10a and the reliability of the control system 30a, with respect to the reliability assigned according to the ODD, scenario, etc. The driving system 2 may decide to plan a cautious control action if the value of the product becomes smaller than a preset value. This preset value may be set according to at least one of the stable controllable range R1 and the performance limit range R2.

[0223] The conditions for performing a dynamic driving task may include conditions for the environment determination unit 21 to determine the environment. The environment determination unit 21 selects a scenario and refers to the confidence level assignment corresponding to that scenario. The environment determination unit 21 may then interpret the environment model taking that confidence level into consideration. For example, if a communication disturbance scenario is selected, in order to interpret the environment model to achieve the confidence level assigned to the recognition system 10a corresponding to this scenario, the environment determination unit 21 may ensure the overall confidence level of the recognition system 10a by performing an interpretation of the environment model on the premise that the contribution of the map and the information acquired by V2X has been reduced.

[0224] The conditions for executing a dynamic operation task may include conditions for the operation planning unit 22 to determine the behavior plan and track plan. The operation planning unit 22 may determine the behavior plan and track plan taking into account the assignment of confidence levels according to the scenario selected by the environment judgment unit 21. For example, if a high confidence level is required for the judgment system 20a due to low confidence levels of the recognition system 10a and the control system 30a, the operation planning unit 22 may plan more cautious control actions than a normal plan. Cautious control actions may include transitioning to degenerate behavior, executing MRM, or transitioning to DDT fallback.

[0225] The conditions for performing a dynamic operation task may include conditions for determining at least one of the modes and constraints managed by the mode management unit 23. The mode management unit 23 may set functional constraints, taking into account the assignment of reliability according to the scenario selected by the environment determination unit 21. For example, if a high reliability is required for the determination system 20a due to low reliability of the recognition system 10a and the control system 30a, the mode management unit 23 may set constraints such as an upper limit on speed and an upper limit on acceleration in the behavior plan and track plan planned by the operation planning unit 22.

[0226] The conditions for executing a dynamic operation task may include trigger conditions, minimum risk conditions, fallback conditions, and so on. Furthermore, changes to the conditions for executing a dynamic operation task may involve changing the conditional expression itself, or changing the numerical values ​​entered into the conditional expression.

[0227] The following describes an example of the process related to changing the conditions for realizing a dynamic operation task, within the operation flow of the operating system 2, using the flowchart in Figure 25. The series of processes shown in steps S141 to S144 are repeatedly executed by the operating system 2 at predetermined time intervals or based on predetermined triggers.

[0228] In S141, the environmental judgment unit 21 selects a scenario based on the current situation of the vehicle 1. After S141, the process proceeds to S142.

[0229] In S142, at least one of the execution entities among the environmental judgment unit 21, the operation planning unit 22, and the mode management unit 23 acquires the scenario selected in S141 and retrieves the confidence level assignment corresponding to the scenario from the storage medium that stores the confidence level assignment. After processing in S142, the process moves to S143.

[0230] In S143, the entity executing S142 modifies the conditions for realizing the dynamic operation task based on the acquired confidence level assignment. After processing in S143, the process moves to S144.

[0231] In S144, the operation planning unit 22 derives a control action based on the conditions or the result of calculation processing performed according to the conditions. The series of processes ends with S144.

[0232] The scenarios used in processing S141-S144 may be replaced with ODDs, or with a combination of scenarios and ODDs.

[0233] <Effects and Effects> The effects and advantages of the first embodiment described above are explained below.

[0234] According to the first embodiment, the interaction between each subsystem and the real world is modeled as a loop structure. The errors generated in each subsystem are then represented in a way that allows for the simulation of their propagation between subsystems through these identified closed loops. By evaluating the errors propagating according to the closed loops, the complex factors between subsystems can be identified. Therefore, the validity of the operating system 2, which comprises multiple subsystems, can be appropriately verified.

[0235] Furthermore, according to the first embodiment, the interaction between each subsystem and the real world is modeled as a loop structure. The evaluation of the closed loop thus identified is based on confidence, which is a common measure among the subsystems. Since confidence is introduced as a common measure, even if the recognition system 10a, the decision system 20a, and the control system 30a each have different functions, the complex factors resulting from their interactions can be confirmed. Therefore, the validity of the operating system 2, which has multiple subsystems, can be appropriately confirmed.

[0236] Furthermore, according to the first embodiment, it is evaluated that errors propagating according to a closed loop fall within the allowable error with a probability of a predetermined confidence level or higher. By applying evaluation based on probability theory to each subsystem, the validity of the operating system 2 can be appropriately confirmed. Moreover, it becomes easy to realize a system configuration that enhances the continuity of operation of the operating system 2 through the mutual complementarity of each subsystem.

[0237] Furthermore, according to the first embodiment, the closed loop includes an inner loop IL that circulates within the vehicle 1, the recognition system 10a, and the control system 30a in the real world. By evaluating such an inner loop IL, it becomes possible to confirm the propagation of errors that could not be detected by evaluations related to the judgment system 20a alone.

[0238] Furthermore, according to the first embodiment, the closed loop includes an outer loop EL that circulates between the vehicle 1 in the real world, the external environment EE in the real world, the recognition system 10a, the decision system 20a, and the control system 30a, and evaluates the interaction between the vehicle 1 and the external environment EE. By evaluating such an outer loop EL, the combined factors of the three systems, the recognition system 10a, the decision system 20a, and the control system 30a, can be more appropriately confirmed.

[0239] Furthermore, according to the first embodiment, the allocation of tolerances to each subsystem is adjusted. This adjustment involves comparing the error of each provisionally designed subsystem with its tolerance. Here, the tolerance is determined by evaluating the deviation of the tolerance of the entire operating system 2 that is provisionally allocated to each subsystem, and the error propagating through the operating system 2. As a result of using the evaluation of the error propagating through the operating system 2, complex factors based on the interaction between each subsystem can be reflected in the design. Therefore, the validity of the operating system 2, which has multiple subsystems, can be improved.

[0240] Furthermore, according to the first embodiment, the specifications of each subsystem are determined such that errors propagating through the operating system 2 fall within the allowable error with a probability of a predetermined confidence level or higher. That is, a confidence level is introduced as a common measure by applying a probability-based evaluation to each subsystem. Therefore, even if the recognition system 10a, the decision system 20a, and the control system 30a each have different functions, the complex factors resulting from their interactions can be appropriately reflected in the design. Thus, the validity of the operating system 2, which has multiple subsystems, can be increased. Moreover, it becomes easy to realize a system configuration that enhances the continuity of operation of the operating system 2 through the mutual complementarity of each subsystem.

[0241] Furthermore, according to the first embodiment, errors propagating through the operating system 2 are evaluated according to a closed-loop model that models the interaction between each subsystem and the real world as a loop structure. The closed-loop model allows the errors generated in each subsystem to be represented in a form that allows for the simulation of propagation between subsystems, thus making it easy to identify the complex factors between subsystems. Therefore, the validity of the operating system 2, which comprises multiple subsystems, can be appropriately verified.

[0242] Furthermore, according to the first embodiment, the closed loop includes an inner loop IL that circulates within the vehicle 1, the recognition system 10a, and the control system 30a in the real world. By evaluating such an inner loop IL, it becomes possible to confirm the propagation of errors that could not be detected by evaluations related to the judgment system 20a alone.

[0243] Furthermore, according to the first embodiment, the closed loop includes an outer loop EL that circulates between the vehicle 1 in the real world, the external environment EE in the real world, the recognition system 10a, the decision system 20a, and the control system 30a, and evaluates the interaction between the vehicle 1 and the external environment EE. By evaluating such an outer loop EL, the combined factors of the three systems, the recognition system 10a, the decision system 20a, and the control system 30a, can be more appropriately confirmed.

[0244] Furthermore, according to the first embodiment, the conditions for realizing the dynamic operation task are changed based on the assignment of confidence levels to each subsystem stored in storage media such as memory 51a, scenario DB 53, and memory 55a. That is, since a confidence level, which is a common measure among each subsystem, is used, even if the recognition system 10a, the decision system 20a, and the control system 30a each have different functions, it is possible to change the conditions while considering the load on each subsystem, which may differ depending on the assigned category. Therefore, a high degree of validity can be achieved in the operation system 2 which has multiple subsystems.

[0245] Furthermore, according to the first embodiment, the scenario in which the vehicle 1 is currently located is selected. In addition, when changing the conditions for realizing the dynamic driving task, the system refers to the reliability assignment determined in accordance with the scenario and determines whether or not to transition to degraded behavior based on the product of the reliability of the recognition system 10a and the reliability of the control system 30a. Therefore, even if the reliability of one of the recognition system 10a and the control system 30a is low, if the reliability of the other is high, the system can avoid transitioning to degraded behavior and continue with appropriate driving behavior. Thus, the driving system 2 can achieve a highly flexible response.

[0246] (Second Embodiment) As shown in Figures 26 and 27, the second embodiment is a modified version of the first embodiment. The second embodiment will be described focusing on the differences from the first embodiment.

[0247] As shown in FIG. 26, the operation system 202 of the second embodiment may further include a monitoring unit 221 that monitors the determination unit 220 at the functional level. In other words, a monitoring system 221a may be provided as a subsystem that monitors the determination system 220a. On the other hand, the monitoring unit 221 to the monitoring system 221a may be located in a part of the determination unit 220 to the determination system 220a included in the determination unit 220 to the determination system 220a.

[0248] As shown in FIG. 27, the operation system 202 further includes a dedicated computer 252 for realizing a monitoring function at the technical level. The dedicated computer 252 may be configured on the same substrate as the dedicated computer 51 in the processing system 250 that realizes the determination function and may be capable of communicating with each other on board. On the other hand, the dedicated computer 252 may be implemented in the form of a monitoring ECU provided separately from the processing system 250 that realizes the determination function. The dedicated computer 252 may be, for example, an RSS system that realizes a safety-related model such as an RSS model.

[0249] The dedicated computer 252 has at least one memory 252a and at least one processor 252b. The memory 252a is at least one type of non-transitory physical storage medium such as a semiconductor memory, a magnetic medium, and an optical medium that non-temporarily stores a program and data readable by the computer 252. Further, as the memory 252a, a rewritable volatile storage medium such as a RAM (Random Access Memory) may be provided. The processor 252b includes at least one type such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a RISC (Reduced Instruction Set Computer)-CPU as a core.

[0250] The dedicated computer 252 may be a SoC (System on a Chip) that integrally implements a memory, a processor, and an interface on one chip, or may have a SoC as a component of the dedicated computer.

[0251] The monitoring unit 221 acquires information such as an environmental model and vehicle state from the recognition unit 10, and monitors at least one of the risks occurring between the host vehicle 1 and other road users and the risks due to judgment errors of the judgment unit 220. The monitoring unit 221, for example, sets a safety envelope. The monitoring unit 221 detects a violation of the safety envelope in at least one of the control actions derived by the host vehicle 1 and the judgment unit 220.

[0252] The safety envelope may be set according to assumptions based on a safety-related model. The assumptions based on the safety-related model may be reasonably foreseeable assumptions regarding other road users. Such assumptions may be, for example, reasonably worst-case assumptions regarding other road users in the RSS model, in which the minimum safe longitudinal distance and the minimum safe lateral distance are calculated. Such assumptions may be set based on a scenario selected by the recognition unit 10, the judgment unit 220, or the monitoring unit 221. The safety envelope may define a boundary around the host vehicle 1. The safety envelope may be set based on the kinematic characteristics of other road users, traffic rules, regions, etc.

[0253] When a violation of the safety envelope is detected, the monitoring unit 221 may change the control action derived by the judgment unit 220. The change of the control action here may correspond to an appropriate response, may correspond to a transition to a minimum risk condition, or may correspond to a DDT fallback.

[0254] The monitoring unit 221 may reject the control action derived by the decision unit 220 if a violation of the safety envelope is detected. In this case, the monitoring unit 221 may set constraints on the decision unit 220. If the control action is rejected, the decision unit 220 may derive a control action again based on the set constraints.

[0255] The safety-related model or mathematical model used by the monitoring unit 221 for monitoring may be capable of invalidating quantitative and qualitative errors in judgment errors in the decision unit 220. The safety-related model or mathematical model may be capable of forcibly correcting errors caused by quantitative and qualitative errors in judgment errors in the decision unit 220 to within an acceptable range.

[0256] In other words, by installing the monitoring unit 221, it becomes possible to effectively consider the error j due to judgment errors as zero. On the other hand, errors d due to vehicle motion disturbances, errors m due to misrecognition, and errors n due to observation noise remain, and these errors propagate according to a closed loop.

[0257] Therefore, verification and validation of complex factors between subsystems are also effective in the operating system 202, which includes the monitoring function of the monitoring unit 221. The evaluation method and design method of the first embodiment may also be applied to the operating system 202. Furthermore, similar to the first embodiment, the decision unit 220 or the monitoring unit 221 can change the conditions for realizing the dynamic operation task based on the assignment of reliability.

[0258] The following describes an example of the monitoring function processing by the monitoring system 221a within the operation flow of the operating system 202, using the flowchart in Figure 28. The series of processes shown in steps S201 to S206 are repeatedly executed by the operating system 202 at predetermined intervals or based on predetermined triggers.

[0259] In S201, the scenario in which your vehicle 1 is currently located is selected. After S201, proceed to S202.

[0260] In S202, based on the scenario selected in S201, the movements of other road users are assumed to a reasonable and foreseeable extent. After S202, proceed to S203.

[0261] In S203, a safety envelope is set based on the assumptions and mathematical model in S202. The mathematical model here is either a mathematical model that neutralizes quantitative and qualitative errors in judgment errors in the decision function, or a mathematical model that forcibly corrects errors due to quantitative and qualitative errors in judgment errors in the decision function to within an acceptable range. After processing in S203, the process moves to S204.

[0262] In S204, safety envelope violations are detected using information such as the environmental model. In other words, it is determined whether or not a violation has occurred. If the result in S204 is negative, the process moves to S205. If the result in S205 is positive, the process moves to S206.

[0263] In S205, the control action derived by the decision function is adopted. The series of processes ends with S205.

[0264] In S206, the control action derived by the decision function is modified or rejected. The series of processes ends with S206.

[0265] (Third embodiment) As shown in Figure 29, the third embodiment is a modified version of the first embodiment. The second embodiment will be described focusing on the differences from the first embodiment.

[0266] In the third embodiment of the driving system 302, there is no direct input or output of information between the recognition unit 10 and the control unit 30. That is, the information output by the recognition unit 10 is input to the control unit 30 via the judgment unit 20. For example, at least one of the vehicle state recognized by the internal recognition unit 14, such as the current speed, acceleration, and yaw rate of the vehicle 1, is directly passed to the motion control unit 31 via the environment judgment unit 321 and the driving plan unit 322, or via the mode management unit 323 and the driving plan unit 322.

[0267] In other words, the environmental judgment unit 321 and the operation planning unit 322, or the mode management unit 323 and the operation planning unit 322, have the function of processing some of the information acquired from the internal recognition unit 14 and outputting it to the motion control unit 31 in the form of a track plan, etc., and outputting other parts of the information acquired from the internal recognition unit 14 to the motion control unit 31 as unprocessed information.

[0268] Therefore, the interaction between the recognition unit 10 and the control unit 30 in the physical IF layer of the causal loop shown in Figure 5 is substantially realized.

[0269] (Fourth Embodiment) As shown in Figure 30, the fourth embodiment is a modified version of the first embodiment. The second embodiment will be described focusing on the differences from the first embodiment.

[0270] The fourth embodiment of the driving system 402 employs a domain-type architecture that provides driving assistance up to Level 2. An example of the detailed configuration of the driving system 402 at the technical level will be explained based on Figure 30.

[0271] The driving system 402 includes, as in the first embodiment, a plurality of sensors 41, 42, a plurality of motion actuators 60, a plurality of HMI devices 70, and a plurality of processing systems, etc. Each processing system is a domain controller that aggregates processing functions for each respective functional domain. The domain controller may have the same configuration as the processing system or ECU of the first embodiment. For example, the driving system 402 includes, as processing systems, an ADAS domain controller 451, a power train domain controller 452, a cockpit domain controller 453, a connectivity domain controller 454, etc.

[0272] The ADAS domain controller 451 aggregates functions related to ADAS (Advanced Driver-Assistance Systems). The ADAS domain controller 451 may comprehensively implement a part of the recognition function, a part of the judgment function, and a part of the control function. A part of the recognition function implemented by the ADAS domain controller 451 may be, for example, a function corresponding to the fusion unit 13 of the first embodiment or a simplified function thereof. A part of the judgment function implemented by the ADAS domain controller 451 may be, for example, a function corresponding to the environmental judgment unit 21 and the driving plan unit 22 of the first embodiment or a simplified function thereof. A part of the control function implemented by the ADAS domain controller 451 may be, for example, a function of generating request information to the motion actuator 60 among the functions corresponding to the motion control unit 31 of the first embodiment.

[0273] Specifically, the functions implemented by the ADAS domain controller 451 are functions for providing driving support in non-dangerous scenarios, such as a lane keeping support function for driving the host vehicle 1 along a white line and a following distance maintenance function for following a preceding other vehicle located in front of the host vehicle 1 with a predetermined following distance. Also, the functions implemented by the ADAS domain controller 451 are functions for realizing appropriate responses in dangerous scenarios, such as a collision damage mitigation brake function for applying brakes when likely to collide with other road users or obstacles and an automatic steering avoidance function for avoiding collisions by steering when likely to collide with other road users or obstacles.

[0274] The powertrain domain controller 452 integrates functions related to powertrain control. The powertrain domain controller 452 may implement at least some of the recognition functions and at least some of the control functions in combination. Some of the recognition functions implemented by the powertrain domain controller 452 may be, for example, a function that recognizes the driver's operating state to the motion actuator 60, which corresponds to the internal recognition unit 14 of the first embodiment. Some of the control functions implemented by the powertrain domain controller 452 may be, for example, a function that controls the motion actuator 60, which corresponds to the motion control unit 31 of the first embodiment.

[0275] The cockpit domain controller 453 aggregates functions related to the cockpit. The cockpit domain controller 453 may implement at least some of the recognition functions and at least some of the control functions in combination. Some of the recognition functions implemented by the cockpit domain controller 453 may be, for example, the function of recognizing the switch state of the HMI device 70 among the internal recognition unit 14 of the first embodiment. Some of the control functions implemented by the cockpit domain controller 453 may be, for example, a function corresponding to the HMI output unit 71 of the first embodiment.

[0276] The connectivity domain controller 454 aggregates connectivity-related functions. The connectivity domain controller 454 may implement at least a portion of the recognition functions in a complex manner. Some of the recognition functions implemented by the connectivity domain controller 454 may be functions that organize and convert global location data, V2X information, etc., of the vehicle 1 acquired from the communication system 43 into a format that can be used by the ADAS domain controller 451.

[0277] In this fourth embodiment as well, at the functional level, it is possible to associate the functions of the operating system 402, including each domain controller 451, 452, 453, and 454, with the recognition unit 10, the decision unit 20, and the control unit 30. Therefore, evaluation using a causal loop structure similar to that of the first embodiment is possible.

[0278] (Other embodiments) Although several embodiments have been described above, this disclosure is not limited to those embodiments and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0279] Driving system 2 is applicable to various types of mobile bodies other than vehicles. Examples of mobile bodies include ships, aircraft, drones, construction machinery, agricultural machinery, etc.

[0280] The control unit and method described herein may be implemented by a dedicated computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the apparatus and method described herein may be implemented by a dedicated hardware logic circuit. Alternatively, the apparatus and method described herein may be implemented by one or more dedicated computers comprising a combination of a processor that executes a computer program and one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium.

[0281] (Explanation of terms) Terms related to this disclosure are defined below. This definition is included in embodiments of this disclosure.

[0282] A road user may be a person who uses a road, including sidewalks and other adjacent spaces. A road user may be a road user who is on or adjacent to an active road for the purpose of moving from one place to another.

[0283] A dynamic driving task (DDT) may be a real-time operational and tactical function for controlling a vehicle in traffic.

[0284] An automated driving system may be a set of hardware and software capable of continuously performing the entire DDT, regardless of whether it is limited to a specific operational design domain.

[0285] SOTIF (safety of the intended functionality) may be the absence of undue risk resulting from insufficient functionality of the intended function or its implementation.

[0286] A driving policy may be a set of strategies and rules that define control actions at the vehicle level.

[0287] Vehicle motion can be defined as the vehicle state and its dynamics as measured in terms of physical quantities (e.g., velocity, acceleration).

[0288] The "situation" may be any factor that could affect the system's behavior. This may include circumstances, traffic conditions, weather, and the behavior of the vehicle itself.

[0289] The estimation of the situation may involve reconstructing a set of parameters representing the situation using an electronic system based on the information obtained from the sensor.

[0290] A scenario may be a description of the temporal relationships between several scenes within a series of scenes, including goals and values ​​in a specific situation influenced by actions and events. A scenario may be a description of a continuous time-series of activities integrating the subject vehicle, all its external environment, and their interactions in the process of performing a specific driving task.

[0291] The behavior of one's own vehicle may be interpreted as vehicle motion in the context of traffic conditions.

[0292] A triggering condition may be a specific condition in a scenario that acts as a trigger for a subsequent system response that contributes to the failure to prevent, detect, or mitigate dangerous behavior or reasonably foreseeable indirect misuse.

[0293] A proper response may be an action that resolves a dangerous situation when other road users are acting in accordance with reasonably foreseeable assumptions about their behavior.

[0294] A hazardous situation may be a scenario that represents the increased level of risk present in DDT unless preventive actions are taken.

[0295] A safe situation may be one in which the system is within its performance limits that ensure safety. It should be noted that, depending on the definition of the performance limits, a safe situation becomes a design concept.

[0296] A minimal risk maneuver (MRM) may be a function of an (automated) driving system that transitions the vehicle between nominal and minimal risk conditions.

[0297] DDT fallback may be a driver or automated system response to perform DDT or transition to minimum risk conditions after detecting a failure or malfunction, or when detecting potentially dangerous behavior.

[0298] Performance limits can be design limits that enable a system to achieve its objectives. Performance limits can be set for multiple parameters.

[0299] The operational design domain (ODD) may be the specific conditions under which a given (autonomous) driving system is designed to function. The operational design domain is the operating conditions specifically designed for a given (autonomous) driving system or feature to function, and may include, but are not limited to, the necessary presence or absence of environmental, geographical, and time constraints, and / or specific traffic or road features.

[0300] The (stable) controllable range may be the range of design values ​​within which the system can continue to perform its intended purpose. The (stable) controllable range can be set for multiple parameters.

[0301] The minimum risk condition may be a vehicle condition designed to mitigate the risk of being unable to complete a given trip. The minimum risk condition may also be a condition in which the user or the automated driving system brings the vehicle after performing a minimum risk operation to mitigate the risk of collision if the given trip cannot be completed.

[0302] The takeover may be the transfer of the driving task between the autonomous driving system and the driver.

[0303] An unreasonable risk may be one that is deemed unacceptable in a particular situation, according to reasonable social and moral concepts.

[0304] Safety-related models may represent safety-related aspects of driving behavior based on assumptions about the reasonably foreseeable behavior of other road users. Safety-related models may be onboard or offboard safety confirmation or safety analysis devices, mathematical models, sets of more conceptual rules, sets of scenario-based behaviors, or a combination thereof.

[0305] A safety envelope may be a set of limitations and conditions designed to ensure that an (automated) driving system operates as a constraint or control in order to maintain operations within an acceptable level of risk. A safety envelope may be a general concept that can be used to address all principles to which a driving policy may adhere, under which a vehicle operated by an (automated) driving system may have one or more boundaries around it.

[0306] (Additional note) This disclosure also includes the following technical features based on the embodiments described above.

[0307] <Technical Feature 1> A method for evaluating a mobile vehicle driving system, comprising a recognition system, a decision-making system, and a control system as subsystems, Evaluating the nominal performance of the recognition system, Evaluating the nominal performance of the decision system, An evaluation method, including evaluating the nominal performance of a control system.

[0308] <Technical Feature 2> A method for evaluating a mobile vehicle driving system, comprising a recognition system, a decision-making system, and a control system as subsystems, Evaluating the nominal performance of the decision system, An evaluation method comprising: evaluating the robust performance of a decision system by taking into account at least one of the errors of the recognition system and the errors of the control system.

[0309] <Technical Feature 3> A method for evaluating a mobile vehicle driving system, comprising a recognition system, a decision-making system, and a control system as subsystems, The nominal performance of the recognition system, the nominal performance of the decision-making system, and the nominal performance of the control system will be evaluated independently. An evaluation method comprising evaluating the robust performance of the entire operating system, such that the evaluation targets include combined factors of the recognition system and the judgment system, combined factors of the judgment system and the control system, and combined factors of the recognition system and the control system.

[0310] <Technical Feature 4> A mobile vehicle driving system comprising a recognition system, a decision-making system, and a control system as subsystems, A loop that shows the interaction between each subsystem and the real world, comprising a first closed loop that circulates within the mobile body, the recognition system and the control system in the real world, and is self-contained within the mobile body. A loop representing the interaction between each subsystem and the real world, comprising a second closed loop including the interaction between the mobile object and the external environment, which circulates between the mobile object and the external environment, the mobile object in the real world, the external environment in the real world, the recognition system, the decision system, and the control system. An operating system configured such that the errors propagating through the first closed loop and the second closed loop fall within a predetermined tolerance.

[0311] <Technical Feature 5> A mobile vehicle driving system comprising a recognition system, a decision-making system, and a control system as subsystems, A loop that shows the interaction between each subsystem and the real world, comprising a first closed loop that circulates within the mobile body, the recognition system and the control system in the real world, and is self-contained within the mobile body. A loop representing the interaction between each subsystem and the real world, comprising a second closed loop including the interaction between the mobile object and the external environment, which circulates between the mobile object and the external environment, the mobile object in the real world, the external environment in the real world, the recognition system, the decision system, and the control system. An operating system configured such that errors propagating through the first closed loop and the second closed loop fall within a predetermined tolerance with a probability greater than a predetermined confidence level.

[0312] <Technical Feature 6> A monitoring system comprising at least one processor, which monitors the decision-making function in the operation of a mobile vehicle, The processor is, Based on mathematical models that neutralize quantitative and qualitative errors in judgment errors in the decision-making function, the system detects violations of the safety envelope, A monitoring system configured to modify or reject control actions derived by a decision function when a violation of the safety envelope is detected.

[0313] <Technical Feature 7> A monitoring system comprising at least one processor, which monitors the decision-making function in the operation of a mobile vehicle, The processor is, Based on a mathematical model that forcibly corrects errors caused by quantitative and qualitative errors in judgment functions to within an acceptable range, the system detects violations of the safety envelope. A monitoring system configured to modify or reject control actions derived by a decision function when a violation of the safety envelope is detected.

Claims

1. An evaluation method for evaluating the driving system (2, 202, 302, 402) of a mobile body (1), which includes a recognition system (10a), a decision system (20a, 220a), and a control system (30a) as subsystems, using a computer, The interaction between each subsystem and the real world is modeled as a loop structure to identify closed loops (IL, EL, SL), Identifying the errors that occur in each of the subsystems, An evaluation method comprising: evaluating the error propagating according to the closed loop so as to enable the confirmation of the combined factors between each subsystem.

2. An evaluation method for evaluating the driving system (2, 202, 302, 402) of a mobile body (1), which includes a recognition system (10a), a decision system (20a, 220a), and a control system (30a) as subsystems, using a computer, The interaction between each subsystem and the real world is modeled as a loop structure to identify closed loops (IL, EL, SL), To evaluate the complex factors between each subsystem, a reliability metric is introduced for each subsystem as a common measure among them. An evaluation method comprising: evaluating the closed loop based on the confidence level so as to enable the confirmation of the aforementioned combined factors.

3. Identifying the errors that occur in each of the subsystems, The evaluation method according to claim 2, wherein, in the evaluation, it is evaluated whether the error propagating according to the closed loop falls within the allowable error with a probability of a predetermined confidence level or higher.

4. The evaluation method according to any one of claims 1 to 3, wherein the closed loop includes a loop (IL) that circulates within the mobile body, the recognition system, and the control system in the real world.

5. The evaluation method according to claim 4, wherein the closed loop includes a loop (EL) that circulates between the moving body in the real world, the external environment (EE) in the real world, the recognition system, the judgment system, and the control system, and evaluates the interaction between the moving body and the external environment.

6. The evaluation method according to any one of claims 1 to 3, wherein the closed loop includes a loop (EL) that circulates between the moving body in the real world, the external environment (EE) in the real world, the recognition system, the judgment system, and the control system, and evaluates the interaction between the moving body and the external environment.

7. A storage medium configured to be readable by a computer, To the aforementioned computer, Regarding the operating system (2, 202, 302, 402) of a mobile body (1) which includes a recognition system (10a), a decision system (20a, 220a), and a control system (30a) as subsystems, the interaction between each subsystem and the real world is modeled as a loop structure to identify closed loops (IL, EL, SL), Identifying the errors that occur in each of the subsystems, A storage medium that stores a computer program that performs the following: evaluate the error propagating according to the closed loop, so as to enable the confirmation of the combined factors between each of the subsystems.

8. A storage medium configured to be readable by a computer, To the aforementioned computer, Regarding the operating system (2, 202, 302, 402) of a mobile body (1) which includes a recognition system (10a), a decision system (20a, 220a), and a control system (30a) as subsystems, the interaction between each subsystem and the real world is modeled as a loop structure to identify closed loops (IL, EL, SL), To evaluate the complex factors between each subsystem, a reliability metric is introduced for each subsystem as a common measure among them. A storage medium that stores a computer program that performs the following: evaluating the closed loop based on the reliability so that the aforementioned combined factors can be confirmed.

9. An evaluation device for evaluating the driving system (2,202,302,402) of a mobile body (1), comprising a recognition system (10a), a decision system (20a,220a), and a control system (30a) as subsystems, It comprises at least one processor (81a), The aforementioned processor, The interaction between each subsystem and the real world is modeled as a loop structure to identify closed loops (IL, EL, SL), Identifying the errors that occur in each of the subsystems, An evaluation device that performs the following: evaluating the error propagating according to the closed loop, so as to enable the confirmation of the combined factors between each subsystem.

10. An evaluation device for evaluating the driving system (2,202,302,402) of a mobile body (1), comprising a recognition system (10a), a decision system (20a,220a), and a control system (30a) as subsystems, It comprises at least one processor (81a), The aforementioned processor, The interaction between each subsystem and the real world is modeled as a loop structure to identify closed loops (IL, EL, SL), To evaluate the complex factors between each subsystem, a reliability metric is introduced for each subsystem as a common measure among them. An evaluation device that performs the following: evaluating the closed loop based on the reliability so that the aforementioned combined factors can be confirmed.

Images