Processing system

Abstract

This processing system (50) executes processing related to driving a vehicle (1). The processing system (50) comprises: a plurality of guidelines (211, 212, 21n) by which individual evaluation results pertaining to strategic guidelines are outputted on the basis of sensor data, at least some of the sensor data output sources differing from each other; guideline integration (221) by which the individual evaluation results are integrated and the integrated evaluation result is outputted; and an operation planning unit (22) that plans operation actions on the basis of the evaluation results after the integration.

Description

Cross-reference of related applications

[0001] This application is based on Japanese Patent Application No. 2022-187493, filed in Japan on November 24, 2022, and incorporates the contents of the basic application by reference in whole. [Technical Field]

[0002] The disclosures in this specification relate to the operation of a vehicle. [Background technology]

[0003] Patent Document 1 describes a guideline processor that acquires sensor data from multiple sensors and performs an evaluation of strategic guidelines based on the sensor data in order to plan driving behavior. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] U.S. Patent Application Publication No. 2021 / 0356962 [Overview of the Initiative]

[0005] However, in configurations like that described in Patent Document 1, where sensor data from all sensors is evaluated together, it is difficult to identify the causal relationship between the evaluation results regarding the derived driving behavior or strategic guidelines and the multiple sensors used. In particular, when the evaluation of strategic guidelines is performed using artificial intelligence or the like, the difficulty of verification increases significantly. Thus, there is room for improvement in traceability to planned driving behavior.

[0006] One of the purposes of this disclosure is to provide a processing system that improves traceability to planned driving behavior.

[0007] The embodiments disclosed herein are processing systems that perform processes relating to the operation of a vehicle, Sensor data Using this method, we evaluate each rule in a rule set containing multiple rules regarding traffic laws, and then apply this to the rule set. A plurality of individual evaluation units that output individual evaluation results, wherein at least a portion of the sensor data output sources differ from each other, An integrated evaluation unit that combines each individual evaluation result and outputs the integrated evaluation result, Based on the evaluation results after integration, By deriving driving behaviors that minimize rule violations, It includes a driving planning unit that plans driving actions, 。

[0008] Furthermore, other embodiments disclosed include a processing system that performs processes related to the operation of a vehicle, Sensor data Using this method, we evaluate each rule in a rule set containing multiple rules regarding traffic laws, and then apply this to the rule set. A plurality of individual evaluation units that output individual evaluation results, wherein at least a portion of the sensor data output sources differ from each other, Each individual evaluation unit is provided to correspond to a specific individual evaluation unit, and based on the individual evaluation results output by the corresponding individual evaluation unit, By deriving driving behaviors that minimize rule violations, Multiple individual driving planning units that plan individual driving actions, It includes an integrated driving planning unit that integrates individual driving actions and plans the integrated driving actions.

[0010] The symbols in parentheses included in the claims, etc., are illustrative examples illustrating the correspondence with the embodiments described later, and are not intended to limit the technical scope. [Brief explanation of the drawing]

[0011] [Figure 1] A schematic diagram of the operating system. [Figure 2] A schematic diagram showing the hardware configuration of the driving system. [Figure 3] Hardware configuration diagram of the operating system. [Figure 4] Software configuration diagram of the operating system. [Figure 5] A diagram illustrating an example of the relationship between rules. [Figure 6] A diagram illustrating an example of the relationship between rules. [Figure 7] A diagram showing an example of the relationship of rules. [Figure 8] A diagram showing an implementation example of a rule set. [Figure 9] A flowchart showing an example of a processing method. [Figure 10] A diagram showing an implementation example of a rule set. [Figure 11] A diagram showing an implementation example of a rule set. [Figure 12] A diagram showing an implementation example of a rule set. [Figure 13] A flowchart showing an example of a processing method. [Figure 14] A diagram showing an implementation example of a rule set.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, a plurality of embodiments will be described based on the drawings. In each embodiment, the same reference numerals may be assigned to corresponding components, and redundant descriptions may be omitted. When only a part of the configuration is described in each embodiment, the configuration of other embodiments described previously can be applied to other parts of the said configuration. Also, not only the combinations of configurations explicitly shown in the description of each embodiment, but also the configurations of a plurality of embodiments can be partially combined with each other as long as there is no problem with the combination, even if not explicitly shown.

[0013] In the following plurality of embodiments, the content of “Safety First for Automated Driving,” Tech.Rep., 2019 by Aptiv, Audi, Baidu, BMW, Continental, Daimler, FCA, here, Infineon, Intel, and Volkswagen is incorporated by reference in its entirety.

[0014] (First Embodiment) The driving system 2 of the first embodiment realizes functions related to the driving of a mobile object. Part or all of the driving system 2 is mounted on the mobile object. The mobile object that the driving system 2 processes is vehicle 1. This vehicle 1 may be referred to as the self-vehicle, host vehicle, etc. Vehicle 1 may be configured to communicate with other vehicles directly or indirectly via a communication infrastructure. Other vehicles may be referred to as target vehicles.

[0015] Vehicle 1 may be a road user capable of manual driving, such as a car or truck. Vehicle 1 may also be capable of automated driving. Driving is categorized into levels depending on the extent to which the driver performs all dynamic driving tasks (DDTs). Automated driving levels are defined, for example, in SAE J3016. In levels 0-2, the driver performs some or all of the DDTs. Levels 0-2 may be classified as so-called manual driving. Level 0 indicates that driving is not automated. Level 1 indicates that the driving system 2 assists the driver. Level 2 indicates that driving is partially automated.

[0016] At Level 3 and above, the driving system 2 performs all of the DDT while engaged. Levels 3-5 may be classified as so-called autonomous driving. Systems capable of performing driving at Level 3 or above may be called automated driving systems. Vehicles equipped with automated driving systems, or vehicles capable of performing driving at Level 3 or above, may be called automated vehicles (AVs). Level 3 indicates that driving is conditionally automated. Level 4 indicates that driving is highly automated. Level 5 indicates that driving is fully automated.

[0017] Furthermore, a driving system 2 that is unable to perform Level 3 or higher driving but is capable of performing at least one of Levels 1 and 2 driving may be referred to as a driver assistance system. In the following, when there is little need to specifically identify the feasible level of autonomous driving, the autonomous driving system or driving system may simply be referred to as driving system 2.

[0018] <Overview of the Operating System> The architecture of the driving system 2 is selected to enable an efficient SOTIF (safety of the intended functionality) process. For example, the architecture of the driving system 2 may be based on a sense-plan-act model. The sense-plan-act model comprises a sense element, a plan element, and an act element as its main system elements. The sense element, plan element, and act element interact with each other. Here, sense can be interpreted as perception, plan as judgment, and act as control.

[0019] As shown in Figure 1, at the functional level (in other words, from a functional perspective), recognition, judgment, and control functions are implemented in this driving system 2. As shown in Figure 2, at the technical level (in other words, from a technical perspective), at least multiple sensors 40 corresponding to the recognition function, at least one processing system 50 corresponding to the judgment function, and multiple motion actuators 60 corresponding to the control function are implemented.

[0020] More specifically, a recognition unit 10 may be constructed in the driving system 2 as a functional block that realizes recognition functionality, mainly consisting of multiple sensors 40, a processing system that processes detection information from the multiple sensors 40, and a processing system that generates an environmental model based on the information from the multiple sensors 40. A judgment unit 20 may be constructed in the driving system 2 as a functional block that realizes judgment functionality, mainly consisting of a processing system 50. A control unit 30 may be constructed in the driving system 2 as a functional block that realizes control functionality, mainly consisting of multiple motion actuators 60 and at least one processing system that outputs operation signals from the multiple motion actuators 60.

[0021] Here, the recognition unit 10 may be implemented as a recognition system 10a, a subsystem distinct from the judgment unit 20 and the control unit 30. The judgment unit 20 may be implemented as a judgment system 20a, a subsystem distinct from the recognition unit 10 and the control unit 30. The control unit 30 may be implemented as a control system 30a, a subsystem distinct from the recognition unit 10 and the judgment unit 20. The recognition system 10a, the judgment system 20a, and the control system 30a may constitute mutually independent components.

[0022] Furthermore, multiple HMI (Human Machine Interface) devices 70 may be installed in the vehicle 1. The HMI devices 70 realize human-machine interaction, which is the interaction between the occupants of the vehicle 1 (including the driver) and the driving system 2. Of the multiple HMI devices 70, the part that realizes the operation input function by the occupants may be part of the recognition unit 10. Of the multiple HMI devices 70, the part that realizes the information presentation function may be part of the control unit 30. On the other hand, the functions realized by the HMI devices 70 may be positioned as functions independent of the recognition function, judgment function and control function.

[0023] The recognition unit 10 is responsible for recognition functions, including the localization of road users such as vehicle 1 and other vehicles (e.g., location estimation). The recognition unit 10 detects the external environment, internal environment, vehicle state, and the state of the driving system 2 of vehicle 1. The recognition unit 10 fuses (fusions) the detected information to generate an environmental model. The environmental model may also be called a world model. The decision unit 20 applies its purpose and driving policy to the environmental model generated by the recognition unit 10 to derive a control action. The control unit 30 executes the control action derived by the decision unit 20.

[0024] <Outline of Physical Architecture> An example of the physical architecture of the driving system 2 is illustrated using Figure 2. The driving system 2 includes multiple sensors 40, multiple motion actuators 60, multiple HMI devices 70, and at least one processing system 50, etc. These components can communicate with each other by either wireless or wired connections, or both. These components may also be able to communicate with each other through an in-vehicle network such as CAN (registered trademark). These components will be explained in more detail using Figure 3.

[0025] Multiple sensors 40 include one or more external environment sensors 41. Multiple sensors 40 may include at least one of one or more internal environment sensors 42, one or more communication systems 43, and a map database 44. If sensor 40 is interpreted narrowly to refer to an external environment sensor 41, the internal environment sensors 42, communication systems 43, and map database 44 may be positioned as components separate from the sensor 40 corresponding to the recognition function at a technical level.

[0026] The external environment sensor 41 may detect targets present in the external environment of the vehicle 1. Examples of target-detection type external environment sensors 41 include cameras, LiDAR (Light Detection and Ranging / Laser imaging Detection and Ranging) laser radar, millimeter-wave radar, ultrasonic sonar, etc. Typically, multiple types of external environment sensors 41 can be combined and implemented to monitor the front, side, and rear directions of the vehicle 1.

[0027] As an example of mounting the external environment sensor 41, a plurality of cameras (for example, 11 cameras) configured to monitor the front, front side, side, rear side, and rear directions of the vehicle 1 may be mounted on the vehicle 1.

[0028] Other possible installations include a plurality of cameras (e.g., four cameras) configured to monitor the front, sides, and rear of the vehicle 1, a plurality of millimeter-wave radars (e.g., five millimeter-wave radars) configured to monitor the front, front sides, sides, and rear of the vehicle 1, and a LiDAR configured to monitor the front of the vehicle 1.

[0029] Furthermore, the external environment sensor 41 may detect atmospheric conditions and weather conditions in the external environment of the vehicle 1. Examples of state-detection type external environment sensors 41 include outside temperature sensors, temperature sensors, and raindrop sensors.

[0030] The internal environment sensor 42 may detect specific physical quantities (hereinafter referred to as motion physical quantities) related to vehicle motion in the internal environment of the vehicle 1. Examples of motion physical quantity detection type internal environment sensors 42 include speed sensors, acceleration sensors, gyro sensors, etc. The internal environment sensor 42 may also detect the state of the occupants in the internal environment of the vehicle 1. Examples of occupant detection type internal environment sensors 42 include actuator sensors, sensors and systems for monitoring the driver, biosensors, seating sensors, and in-vehicle equipment sensors, etc. In particular, actuator sensors include accelerator sensors, brake sensors, steering sensors, etc., which detect the occupant's operation state on motion actuators 60 related to the motion control of the vehicle 1.

[0031] The communication system 43 acquires communication data available to the driving system 2 via wireless communication. The communication system 43 may also receive positioning signals from GNSS (global navigation satellite system) satellites present in the external environment of the vehicle 1. Positioning-type communication equipment in the communication system 43 is, for example, a GNSS receiver.

[0032] The communication system 43 may send and receive communication signals with the external system 96 present in the external environment of vehicle 1. Examples of V2X type communication equipment in the communication system 43 include DSRC (dedicated short range communications) communication devices and cellular V2X (C-V2X) communication devices. Examples of communication with V2X systems present in the external environment of vehicle 1 include communication with communication systems of other vehicles (V2V), communication with infrastructure equipment such as communication devices set up on traffic lights or roadside devices (V2I), communication with pedestrian mobile terminals (V2P), and communication with networks such as cloud servers (V2N). The architecture of V2X communication, including V2I communication, may adopt the architecture specified in ISO21217, ETSI TS 102 940~943, IEEE 1609, etc.

[0033] Furthermore, the communication system 43 may send and receive communication signals with the internal environment of the vehicle 1, for example, with a mobile terminal 91 such as a smartphone located inside the vehicle. Examples of terminal communication type communication devices in the communication system 43 include Bluetooth devices, Wi-Fi devices, infrared communication devices, etc.

[0034] Map DB44 is a database that stores map data available to the driving system 2. Map DB44 is composed of at least one type of non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media. Map DB44 may include a database of navigation units that navigate the driving route of vehicle 1 to its destination. Map DB44 may include a database of PD maps generated using probe data (PD) collected from each vehicle. Map DB44 may include a database of high-precision maps with a high level of accuracy, mainly used for applications in automated driving systems. Map DB44 may include a database of parking maps that include detailed parking information, such as parking space information, used for applications in automated parking or parking assistance.

[0035] The map DB44 suitable for the driving system 2 acquires and stores the latest map data, for example, by communicating with a map server via a V2X type communication system 43. The map data is digitized in two or three dimensions as data representing the external environment of the vehicle 1. The map data may include road data representing at least one of the following: the position coordinates, shape, road surface condition, and standard roads of road structures. The map data may also include marking data representing at least one of the following: the position coordinates and shape of road signs, road markings, and lane markings attached to roads. The marking data included in the map data may represent landmarks such as traffic signs, arrow markings, lane markings, stop lines, direction signs, landmark beacons, business signs, and changes in road line patterns. The map data may also include structural data representing at least one of the following: the position coordinates and shape of buildings and traffic lights facing roads. The marking data included in the map data may represent landmarks such as streetlights, road edges, reflectors, and poles.

[0036] The motion actuator 60 can control vehicle motion based on input control signals. A drive-type motion actuator 60 is a powertrain including at least one of the following: an internal combustion engine, a drive motor, etc. A braking-type motion actuator 60 is, for example, a brake actuator. A steering-type motion actuator 60 is, for example, a steering wheel.

[0037] The HMI device 70 may be an operation input device that can receive driver input for transmitting the will or intention of the occupants, including the driver of the vehicle 1, to the driving system 2. Examples of operation input type HMI devices 70 include an accelerator pedal, brake pedal, shift lever, steering wheel, turn signal lever, mechanical switch, touch panel of a navigation unit, etc. Of these, the accelerator pedal controls the powertrain as a motion actuator 60. The brake pedal controls the brake actuator as a motion actuator 60. The steering wheel controls the steering actuator as a motion actuator 60.

[0038] The HMI device 70 may be an information display device that presents information such as visual information, auditory information, and tactile information to the occupants of the vehicle 1, including the driver. Examples of visual information display type HMI devices 70 include combination meters, graphic meters, navigation units, CIDs (center information displays), HUDs (head-up displays), illumination units, etc. Examples of auditory information display type HMI devices 70 include speakers, buzzers, etc. Examples of tactile information display type HMI devices 70 include steering wheel vibration units, driver's seat vibration units, steering wheel reaction force units, accelerator pedal reaction force units, brake pedal reaction force units, air conditioning units, etc.

[0039] Furthermore, the HMI device 70 may communicate with a mobile terminal such as a smartphone via the communication system 43 to realize HMI functions in conjunction with the terminal. For example, the HMI device 70 may present information acquired from the smartphone to the occupants, including the driver. Alternatively, for example, input to the smartphone may be used as an alternative means of input to the HMI device 70.

[0040] At least one processing system 50 is provided. For example, the processing system 50 may be an integrated processing system that comprehensively executes processing related to recognition functions, processing related to judgment functions, and processing related to control functions. In this case, the integrated processing system 50 may further execute processing related to the HMI device 70, and a separate processing system dedicated to the HMI may be provided. For example, the processing system dedicated to the HMI may be an integrated cockpit system that comprehensively executes processing related to each HMI device.

[0041] For example, the processing system 50 may have a configuration that includes at least one processing unit corresponding to processing related to recognition functions, at least one processing unit corresponding to processing related to judgment functions, and at least one processing unit corresponding to processing related to control functions.

[0042] The processing system 50 has an external communication interface and is connected to at least one element related to processing by the processing system 50, such as the sensor 40, motion actuator 60, and HMI device 70, via at least one of the following: a LAN (Local Area Network), wire harness, internal bus, and wireless communication circuit.

[0043] The processing system 50 is comprised of at least one dedicated computer 51. The processing system 50 may combine multiple dedicated computers 51 to implement functions such as recognition, judgment, and control.

[0044] For example, the dedicated computer 51 constituting the processing system 50 may be an integrated ECU that integrates the driving functions of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be a judgment ECU that determines DDT. The dedicated computer 51 constituting the processing system 50 may be a monitoring ECU that monitors the driving of the vehicle. The dedicated computer 51 constituting the processing system 50 may be an evaluation ECU that evaluates the driving of the vehicle. The dedicated computer 51 constituting the processing system 50 may be a navigation ECU that navigates the driving route of the vehicle 1.

[0045] Furthermore, the dedicated computer 51 constituting the processing system 50 may be a locator ECU that estimates the position of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an image processing ECU that processes image data detected by the external environmental sensor 41. The dedicated computer 51 constituting the processing system 50 may be an actuator ECU that controls the motion actuator 60 of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an HCU (HMI Control Unit) that comprehensively controls the HMI device 70. The dedicated computer 51 constituting the processing system 50 may be at least one external computer that constructs an external center or mobile terminal that can communicate via, for example, the communication system 43.

[0046] The dedicated computer 51 constituting the processing system 50 has at least one memory 51a and at least one processor 51b. The memory 51a may be at least one type of non-transitional physical storage medium, such as semiconductor memory, magnetic media, and optical media, which non-temporarily stores programs and data that can be read by the processor 51b. Furthermore, the memory 51a may also be a rewritable volatile storage medium, such as RAM (Random Access Memory). The processor 51b includes at least one type as a core, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and RISC (Reduced Instruction Set Computer)-CPU.

[0047] The dedicated computer 51 constituting the processing system 50 may be a System on a Chip (SoC) that integrates memory, a processor, and interfaces into a single chip, or it may have an SoC as a component of the dedicated computer 51.

[0048] Furthermore, the processing system 50 may include at least one database for executing dynamic operation tasks. The database may consist of at least one non-transitional physical storage medium, such as a semiconductor memory, a magnetic medium, and an optical medium, and an interface for accessing the storage medium. The database may be a scenario database (hereinafter referred to as Scenario DB) 59, as detailed below. The database may be a rule database (hereinafter referred to as Rule DB) 58, as detailed below. At least one of Scenario DB 59 and Rule DB 58 may not be provided in the processing system 50, but may be provided in the operation system 2 independently of the other systems 10a, 20a, and 30a. At least one of Scenario DB 59 and Rule DB 58 may be provided in an external system 96 and configured to be accessible from the processing system 50 through the communication system 43.

[0049] Furthermore, the processing system 50 may include at least one recording device 55 for recording at least one of the recognition information, judgment information, and control information of the operating system 2. The recording device 55 may include at least one memory 55a and an interface 55b for writing data to the memory 55a. The memory 55a may be at least one type of non-transitional physical storage medium, such as a semiconductor memory, a magnetic medium, and an optical medium.

[0050] At least one of the memory 55a may be mounted on the circuit board in a form that is not easily removable or replaceable, in which case, for example, an eMMC (embedded Multi Media Card) using flash memory may be used. At least one of the memory 55a may be mounted on the recording device 55 in a form that is removable and replaceable, in which case, for example, an SD card may be used.

[0051] The recording device 55 may have a function to select the information to be recorded from among recognition information, judgment information, and control information. In this case, the recording device 55 may have a dedicated computer 55c. The processor provided in the recording device 55 may temporarily store information in RAM or the like. The processor may select the information to be recorded from the temporarily stored information and save the selected information to memory 51a.

[0052] The recording device 55 may access the memory 55a and perform recording in accordance with a data write command from the recognition system 10a, the decision system 20a, or the control system 30a. The recording device 55 may also determine the information flowing through the in-vehicle network and, based on the judgment of a processor provided in the recording device 55, access the memory 55a and perform recording.

[0053] The recording device 55 may not be provided in the processing system 50, but may be provided in the operating system 2 independently of the other systems 10a, 20a, and 30a. The recording device 55 may be provided in the external system 96 and configured to be accessible from the processing system 50 via the communication system 43.

[0054] <Logical Architecture Outline> Next, an example of the logic architecture in the driving system 2 will be explained using Figure 4. The recognition unit 10 may include an environment recognition unit 11, a self-position recognition unit 12, and an internal recognition unit 13 as subblocks that further classify the recognition functions.

[0055] The environmental recognition unit 11 individually processes information acquired from each sensor 40 regarding the external environment (sometimes referred to as sensor data) to realize a function of recognizing the external environment, including targets, other road users, etc. The environmental recognition unit 11 individually processes the detection data detected by each external environment sensor 41. The detection data may be, for example, detection data provided by millimeter-wave radar, sonar, LiDAR, etc. The environmental recognition unit 11 may generate relative position data, including the direction, size, and distance of objects relative to the vehicle 1, from the raw data detected by the external environment sensors 41.

[0056] Furthermore, the detection data may be image data provided from, for example, a camera, LiDAR, etc. The environment recognition unit 11 processes the image data and extracts objects that are captured within the field of view of the image. Object extraction may include estimating the direction, size, and distance of the objects relative to the vehicle 1. Object extraction may also include classifying objects using, for example, semantic segmentation.

[0057] Furthermore, the environment recognition unit 11 processes information acquired through the V2X function of the communication system 43. The environment recognition unit 11 also processes information acquired from the map DB 44.

[0058] The environment recognition unit 11 may be further classified into multiple sensor recognition units optimized for each sensor group. When a sensor recognition unit is associated with recognizing information from a sensor group, it may fuse the information from that sensor group.

[0059] The self-position recognition unit 12 performs localization of the vehicle 1. The self-position recognition unit 12 acquires global position data of the vehicle 1 from the communication system 43 (e.g., a GNSS receiver). In addition, the self-position recognition unit 12 may acquire position information of targets extracted by the environment recognition unit 11. Furthermore, the self-position recognition unit 12 acquires map information from the map DB 44. The self-position recognition unit 12 integrates this information to estimate the position of the vehicle 1 on the map.

[0060] The internal recognition unit 13 processes the detection data detected by each internal environment sensor 42 and realizes the function of recognizing the vehicle state. The vehicle state may include the state of the vehicle's physical motion quantities detected by the speed sensor, acceleration sensor, gyro sensor, etc. The vehicle state may also include at least one of the following: the state of the occupants, including the driver; the driver's operation state of the motion actuator 60; and the switch state of the HMI device 70.

[0061] The decision unit 20 may include a prediction unit 21, an operation planning unit 22, and a mode management unit 23 as subblocks that further classify the decision functions.

[0062] The prediction unit 21 acquires information about the external environment recognized by the environment recognition unit 11 and the self-position recognition unit 12, and the vehicle state recognized by the internal recognition unit 13. Based on the acquired information, the prediction unit 21 may interpret the environment and estimate the situation in which vehicle 1 is currently located. The situation here may be the operating situation, or may include the operating situation.

[0063] The prediction unit 21 may interpret the environment and predict the behavior of objects such as other road users. These objects may be safety-relevant objects. The behavior prediction may include at least one of the following: prediction of the object's velocity, prediction of the object's acceleration, and prediction of the object's trajectory. The behavior prediction should be based on reasonably foreseeable assumptions.

[0064] The prediction unit 21 may interpret the environment and make a decision regarding the scenario in which vehicle 1 is currently located. The decision regarding the scenario may involve selecting at least one scenario in which vehicle 1 is currently located from the catalog of scenarios built in scenario DB59. The prediction unit 21 may interpret the environment or predict potential hazards based on the selected scenario.

[0065] Furthermore, the prediction unit 21 may estimate the driver's intentions based on the predicted behavior, predicted potential hazards, and acquired vehicle status.

[0066] The driving plan unit 22 plans the autonomous driving of the vehicle 1 based on at least one of the following: the estimated position information of the vehicle 1 on the map from the self-position recognition unit 12, the prediction information and driver intention estimation information from the prediction unit 21, and the function constraint information from the mode management unit 23.

[0067] The driving planning unit 22 implements route planning, behavior planning, and trajectory planning functions. The route planning function plans at least one of the following: a route to the destination and a lane plan for the medium distance, based on estimated information of the vehicle 1's position on the map. The route planning function may further include a function to determine at least one of the following: a lane change request and a deceleration request, based on the lane plan for the medium distance. Here, the route planning function may be a mission / route planning function in the strategic function, and may be a function that outputs a mission plan and a route plan.

[0068] The behavior planning function plans the behavior of vehicle 1 based on at least one of the following: the route to the destination planned by the route planning function, lane planning for medium distances, lane change requests and deceleration requests, prediction information and driver intention estimation information from the prediction unit 21, and function constraint information from the mode management unit 23. The behavior planning function may also include a function to generate conditions related to the state transitions of vehicle 1. The conditions related to the state transitions of vehicle 1 may correspond to triggering conditions. Based on these conditions, the behavior planning function may also include a function to determine the state transitions of the application that realizes DDT, and further, the state transitions of driving actions. Based on this state transition information, the behavior planning function may also include a function to determine longitudinal constraints on the path of vehicle 1 and lateral constraints on the path of vehicle 1. The behavior planning function may be a tactical behavior plan in the DDT function and may output tactical behavior.

[0069] The trajectory planning function is a function that plans the trajectory of vehicle 1 based on judgment information from the prediction unit 21, longitudinal constraints on the path of vehicle 1, and lateral constraints on the path of vehicle 1. The trajectory planning function may also include a function to generate a path plan. The path plan may include a speed plan, and the speed plan may be generated as a plan independent of the path plan. The trajectory planning function may also include a function to generate multiple path plans and select the optimal path plan from among the multiple path plans, or a function to switch between path plans. The trajectory planning function may further include a function to generate backup data of the generated path plans. The trajectory planning function may be the trajectory planning function in the DDT function and may output a trajectory plan.

[0070] The mode management unit 23 monitors the driving system 2 and sets constraints on driving-related functions. The mode management unit 23 may manage the mode of automatic driving, for example, the state of the automatic driving level. Management of the automatic driving level may include switching between manual driving and automatic driving, i.e., the transfer of authority between the driver and the driving system 2, in other words, the management of takeover. The mode management unit 23 may monitor the state of subsystems related to the driving system 2 and determine system malfunctions (e.g., errors, unstable operation, system failures, malfunctions). The mode management unit 23 may determine a mode based on the driver's intent based on driver intent estimation information generated by the internal recognition unit 13. The mode management unit 23 may set constraints on driving-related functions based on the system malfunction determination result, the mode determination result, and at least one of the following: vehicle state from the internal recognition unit 13, sensor abnormality (or sensor failure) signal output from the sensor 40, application state transition information from the driving plan unit 22, and trajectory plan.

[0071] Furthermore, the mode management unit 23 may comprehensively have functions to determine longitudinal constraints and lateral constraints on the path of vehicle 1, in addition to constraints on driving functions. In this case, the driving planning unit 22 plans the behavior and trajectory according to the constraints determined by the mode management unit 23.

[0072] The control unit 30 may further include a motion control unit 31 and an HMI output unit 71 as subblocks that classify the control functions. The motion control unit 31 controls the motion of the vehicle 1 based on the trajectory plan (e.g., path plan and speed plan) obtained from the driving plan unit 22. Specifically, the motion control unit 31 generates accelerator request information, shift request information, brake request information and steering request information according to the trajectory plan and outputs them to the motion actuator 60.

[0073] Here, the motion control unit 31 can directly obtain from the recognition unit 10 (particularly the internal recognition unit 13) at least one of the vehicle's current speed, acceleration, and yaw rate, and reflect it in the motion control of the vehicle 1.

[0074] The HMI output unit 71 outputs information related to the HMI based on at least one of the following: prediction information and driver intent estimation information from the prediction unit 21, application state transition information and trajectory plan from the driving plan unit 22, and function constraint information from the mode management unit 23. The HMI output unit 71 may also manage vehicle interactions. The HMI output unit 71 may generate notification requests based on the vehicle interaction management status and control the information presentation function of the HMI device 70. Furthermore, the HMI output unit 71 may generate control requests for wipers, sensor cleaning devices, headlights, and air conditioning devices based on the vehicle interaction management status and control these devices.

[0075] <Strategic Guidelines> The decision unit 20 or the driving planning unit 22 can perform its functions in accordance with strategic guidelines based on the driving policy. The set of strategic guidelines is obtained by analyzing basic principles. The set of strategic guidelines can be implemented in the driving system 2 as one or more databases. The set of strategic guidelines may be a set of rules. The set of rules may be, for example, rulebooks. The rules included in the set of rules may be defined to include traffic laws, safety rules, ethical rules, and local culture rules.

[0076] The strategic guidelines are represented by the following four items: Item 1 is one or more states. Item 2 is one or more actions associated with one or more states. Item 3 is the strategic factor associated with the state and appropriate action. Strategic factors include, for example, distance, speed, acceleration, deceleration, direction, time, temperature, season, chemical concentration, area, height, and weight. Item 4 is a deviation metric that quantitatively evaluates the deviation from appropriate action during machine operation. The deviation metric may be a violation metric, or may include a violation metric.

[0077] The basic principles may include laws, regulations, etc., and may also include combinations thereof. The basic principles may include preferences that are not affected by laws, regulations, etc. The basic principles may include exercise behaviors based on past experience. The basic principles may include characterization of the exercise environment. The basic principles may include ethical concerns.

[0078] Furthermore, the basic principles may include human feedback regarding the autonomous driving system. The basic principles may include feedback from vehicle occupants and other road users regarding at least one of the following: comfort and predictability. Predictability may refer to a reasonably foreseeable range. Feedback on predictability may be feedback based on a reasonably foreseeable range. And the basic principles may include rules.

[0079] Strategic guidelines may include clear, systematic, and comprehensive logical relationships between fundamental principles or rules for corresponding motor behaviors. These fundamental principles or rules may have quantitative measures.

[0080] <Scenario> In driving system 2, a scenario-based approach may be employed to perform or evaluate dynamic driving tasks. The processes required to perform dynamic driving tasks in autonomous driving are classified into disturbances in perception elements with different physical principles, disturbances in decision elements, and disturbances in control elements. The root causes that affect the processing results in each element are structured as a scenario structure.

[0081] The disturbance in the perception element is called a perception disturbance. A perception disturbance is a disturbance that indicates a state in which the perception unit 10 cannot correctly perceive a hazard due to internal or external factors of the sensor 40 and the vehicle 1. Internal factors include, for example, instability related to variations in the mounting or manufacturing of sensors such as the external environmental sensor 41, vehicle tilt due to uneven loads that change the direction of the sensor, and shielding of the sensor due to the mounting of parts on the outside of the vehicle. External factors include, for example, fogging or dirt on the sensor. The physical principles in perception disturbances are based on the sensor mechanism of each sensor.

[0082] The disturbance in the judgment element is traffic disturbance. Traffic disturbance is a disturbance that describes a traffic situation that may be dangerous as a result of a combination of the geometric shape of the road, the behavior of vehicle 1, and the position and behavior of surrounding vehicles. The physical principles in traffic disturbance are based on a geometric perspective and the actions of road users.

[0083] The disturbance in the control element is a vehicle disturbance. Vehicle disturbances may also be called control disturbances. A vehicle disturbance is a disturbance that indicates a situation in which vehicle 1 may not be able to control its own dynamics due to internal or external factors. Internal factors include, for example, the total weight and weight balance of vehicle 1. External factors include, for example, road surface irregularity, slope, and wind. The physical principles in vehicle disturbances are based on the mechanical effects applied to the tires and the vehicle body.

[0084] To address the risk of collisions between vehicle 1 and other road users or structures in the dynamic driving tasks of autonomous driving, a traffic disturbance scenario system is used as one of the scenario structures, in which traffic disturbance scenarios are systematized. For the traffic disturbance scenario system, a reasonably foreseeable range or reasonably foreseeable boundary may be defined, and an avoidable range or avoidable boundary may also be defined.

[0085] The avoidable range or avoidable boundary can be defined, for example, by defining and modeling the performance of a competent and careful human driver. The performance of a competent and careful human driver can be defined in three elements: perception, judgment, and control.

[0086] For example, the scenario structure may be stored in Scenario DB59. Scenario DB59 may store multiple scenarios, including at least one of the following: functional scenarios, logical scenarios, and concrete scenarios. Functional scenarios define the highest-level qualitative scenario structure. Logical scenarios are scenarios that assign quantitative parameter ranges to the structured functional scenario. Concrete scenarios define the safety determination boundaries that distinguish between safe and unsafe states.

[0087] An unsafe state is, for example, a hazardous situation. Furthermore, the range corresponding to a safe state may be called the safe range, and the range corresponding to an unsafe state may be called the unsafe range. Additionally, conditions in a scenario that contribute to dangerous behavior of vehicle 1, or to the inability to prevent, detect, or mitigate reasonably foreseeable misuse, may be trigger conditions.

[0088] Scenarios can be classified as either known or unknown, and also as either dangerous or not dangerous. In other words, scenarios can be classified into known dangerous scenarios, known non-dangerous scenarios, unknown dangerous scenarios, and unknown non-dangerous scenarios.

[0089] <Rule Set> A rule set is a data structure that implements a priority structure for a set of rules arranged based on their relative importance. In any given rule in the priority structure, rules with higher priority are more important than rules with lower priority. The priority structure may be one of the following: hierarchical, non-hierarchical, or hybrid priority structures. A hierarchical structure may, for example, show a pre-order for rule violations of varying degrees. A non-hierarchical structure may, for example, be a weighting system for rules. A rule set may contain a subset of rules. A subset of rules may be hierarchical.

[0090] As shown by directed graphs in Figures 5-7, the relationship between two rules in a rule set can be defined. Figure 5 shows that rule A is more important than rule B. Figure 6 shows that rule A and rule B are incomparable. Figure 7 shows that rule A and rule B are of equal importance.

[0091] Furthermore, the rule set can be customized and implemented as follows: For example, multiple rules can be merged into a single rule. Specifically, if there is a rule α with a higher priority than rule β and rule γ, and it is shown that rule β and rule γ cannot be compared, then rule β and rule γ may be merged into a single rule.

[0092] For example, by adding other perspectives, it is possible to clarify the relationship between two rules whose priority relationship is not clearly defined. Specifically, if there is a rule α that has a higher priority than rule β and rule γ, and the relationship between rule β and rule γ is not clear, adding other perspectives may clarify that rule γ has a higher priority than rule β.

[0093] Furthermore, for example, it is possible to consider new elements by adding rules. Specifically, if there is a rule α with a higher priority than rule β and rule γ, and the relationship between rule β and rule γ is not clear, the priority structure can be improved by adding a rule δ with a higher priority than rule β and rule γ.

[0094] The rule set should be implemented in the form described below to facilitate the verification and validation of SOTIF.

[0095] The rule set may be configured as a hardware independent of the driving plan module, such as the driving plan unit 22. For example, the rule set may be stored in a rule DB 58, which is provided independently of the dedicated computer 51 that implements the driving plan unit 22 in the processing system 50. By separating the rule set from modules that are black boxes such as artificial intelligence, traceability can be achieved between the rules defined in the rule set and traffic laws, etc.

[0096] The rule set should be implemented in a way that facilitates verification against known scenarios. For example, the deviation metric or violation metric for each rule against known scenarios should be made memorizable by a recording device 55 or the like. This would facilitate scoring the decision system 20a and the overall performance of the driving system 2. As mentioned above, separating the rule set from the driving plan module makes it easier to verify the quality of the rule set's specifications.

[0097] The rule set should be implemented in a form that facilitates validation against unknown scenarios. For example, when vehicle 1 encounters an unknown dangerous scenario, the poor performance of the driving system 2 can be associated with rule violations by breaking down the vehicle 1's motion behavior or actual planning process into rules. For example, this can identify rules that vehicle 1 finds difficult to follow and provide feedback to the driving system 2. At least some of the processing or verification in this feedback loop may be performed within the driving system 2 or the processing system 50, or the information may be aggregated and executed by the external system 96.

[0098] Each rule included in the rule set is implemented so that it can be evaluated using metrics such as deviation metrics and violation metrics. These metrics may be functions of strategic factors related to one or both of the state of the strategic guidelines and / or appropriate actions. These metrics may be weighted sums of strategic factors. These metrics may be the strategic factors themselves, or they may be proportional to the strategic factors. These metrics may be inversely proportional to the strategic factors. These metrics may be probability functions of the strategic factors. In addition, these metrics may include at least one of the following: energy consumption, time loss, and economic loss.

[0099] Furthermore, the violation metric is an expression of the futility associated with driving behavior that violates the rule statements defined in the rule set. The violation metric may also be a value determined using empirical evidence to assess the degree of the violation. This empirical evidence may include crowdsourced data on what humans consider reasonable, driver preferences, experiments measuring driver parameters, and research on judicial or other authorities.

[0100] The term "driving behavior" as used herein may or may not be interpreted as being limited to the track. In this case, the deviation metric or violation metric includes longitudinal and lateral distance metrics. That is, if vehicle 1 does not maintain appropriate longitudinal and lateral distance metrics, the rule is considered a violation. The distance metric as used herein may correspond to or be equivalent to a safety envelope, safety distance, etc. The distance metric may also be expressed as an inverse function of distance.

[0101] <Implementation Example> The following describes in more detail an example of implementing a rule set in the driving system 2 using Figure 8. In this example, the rule set is used for pre-processing of the driving plan. The driving system 2 that implements the rule set consists of multiple sensor groups 101, 102, 10n, a rule DB 58, a scenario DB 59, a guideline processor 200, and a planning processor 230. The guideline processor 200 provides guidelines that correspond individually to the sensor groups 101, 102, 10n, and realizes the same number of guidelines 211, 212, 21n as the sensor groups 101, 102, 10n, as well as a guideline integration 221 that integrates the same number of guidelines 211, 212, 21n, by executing a computer program.

[0102] Here, the functions implemented by the guideline processor 200 may correspond to at least some of the functions of the prediction unit 21. The functions implemented by the planning processor 230 may correspond to at least some of the functions of the operation planning unit 22. Each processor 200, 230 is a specific implementation example of at least one processor 51b described above. Each processor 200, 230 may be mainly composed of an independent semiconductor chip. The guideline processor 200 and the planning processor 230 may be mounted on a common substrate. The guideline processor 200 and the planning processor 230 may be mounted on separate substrates.

[0103] Multiple sensor groups 101, 102, and 10n are formed by classifying multiple sensors 40 mounted on the vehicle 1 into multiple groups. The multiple sensors 40 here may include external environment sensors 41, and may also include a communication system 43 and a map database 44. The number of sensor groups can be any number of two or more, but using any odd number of three or more makes it easier to perform majority voting, median extraction, etc. in the integration process described later. A single sensor group may contain one or more sensors. Also, some sensors may be shared between sensor groups, for example, sensor group 101 may contain sensors A and B, and sensor group 102 may contain sensors B and C.

[0104] The multiple sensor groups 101, 102, and 10n may be classified according to the type of sensor 40, according to the direction monitored by the sensor 40, or according to the coordinate system adopted (or detected) by the sensor 40. Furthermore, other suitable classification methods may be employed.

[0105] Classification by type is, for example, the classification of sensors 40 by type. Classifying by type may simplify the sensor fusion processing for each sensor group 101, 102, and 10n. In addition, the characteristics of each sensor group 101, 102, and 10n, such as their strengths and weaknesses in different scenes, become clearer. Therefore, it becomes easier to define the rules for applying rule sets to the sensor data of each sensor group 101, 102, and 10n.

[0106] In the following examples of classification by type, the first sensor group includes multiple cameras positioned to monitor the front, sides, and rear of the vehicle 1, respectively. The second sensor group includes multiple millimeter-wave radars positioned to monitor the front, front-side, sides, and rear of the vehicle, respectively. The third sensor group includes LiDAR positioned to monitor the front, sides, and rear of the vehicle 1. The fourth sensor group includes a map DB 44 and a communication system 43.

[0107] Classification based on direction includes, for example, classification by monitoring direction, and classification where a single sensor group covers all directions (360 degrees) around a vehicle. Classifying by monitoring direction makes it easier to select the rules to apply to each guideline according to the scenario. On the other hand, classifying so that a single sensor group covers all directions around a vehicle increases redundancy because even if one sensor group fails to function due to malfunction, the rule set related to each direction can be applied to the sensor data of other sensor groups.

[0108] In the example of classification by direction, the first sensor group includes a camera, millimeter-wave radar, and LiDAR positioned to monitor the front of vehicle 1. The second sensor group includes a camera and millimeter-wave radar positioned to monitor the sides of vehicle 1. The third sensor group includes a camera and millimeter-wave radar positioned to monitor the rear of vehicle 1.

[0109] As an example of a classification that covers all directions, the first sensor group includes a camera positioned to monitor the front of vehicle 1, and millimeter-wave radar positioned to monitor the sides and rear of vehicle 1. The second sensor group includes a LiDAR positioned to monitor the front of vehicle 1, and cameras configured to monitor the sides and rear of vehicle 1. The third sensor group includes a millimeter-wave radar configured to monitor the front and front-side of vehicle 1, combined with a map DB 44 and a communication system 43 that can also obtain information from the rear, etc.

[0110] Each sensor group 101, 102, and 10n belongs to the recognition system 10a and implements the functions of the recognition unit 10. Each sensor group 101, 102, and 10n outputs sensor data to its respective paired guideline 211, 212, and 21n. In other words, each guideline 211, 212, and 21n receives sensor data from different output sources and from each other.

[0111] Each guideline 211, 212, and 21n is an evaluator that evaluates rules based on sensor data from paired sensor groups 101, 102, and 10n, a rule set stored in rule DB 58, and scenario data stored in scenario DB 59. Guidelines 211, 212, and 21n are configured to perform processing in accordance with strategic guidelines. Here, guidelines 211, 212, and 21n may refer to guidelines for driving behavior.

[0112] Rule DB58 stores the rule set in a format that can be read by the processor 200 through computer programs that implement each guideline 211, 212, and 21n.

[0113] Scenario DB59 stores the scenario structure in a format that can be read by the processor 200 through the computer programs in each guideline 211, 212, and 21n. Each guideline 211, 212, and 21n may recognize the environment in which vehicle 1 is located based on sensor data input from their respective paired sensor groups 101, 102, and 10n. In recognizing the environment, each guideline 211, 212, and 21n may refer to the scenario structure and select the scenario that vehicle 1 is encountering. The selected scenario may be a single scenario or a combination of multiple scenarios. Each guideline 211, 212, and 21n may proceed with processing by selecting different scenarios in parallel.

[0114] Each guideline 211, 212, and 21n may identify known hazardous scenarios, known non-hazardous scenarios, unknown hazardous scenarios, and unknown non-hazardous scenarios in the selection of scenarios. Each guideline 211, 212, and 21n may evaluate rules by referencing scenario structures, selecting scenarios, and identifying them. For example, in known hazardous scenarios, rules associated with the hazard factors should be evaluated negatively (as being violated).

[0115] The first guideline 211 calculates the first violation degree sequence using the first sensor group 101 for the rule sequence corresponding to the rule set. The second guideline 212 calculates the second violation degree sequence using the second sensor group 102 for the rule sequence corresponding to the rule set. The k-th guideline calculates the k-th violation degree sequence using the k-th sensor group for the rule sequence corresponding to the rule set. s Guideline 21n specifies that for a rule sequence corresponding to a rule set, the nth s Using the sensor group 10n, the nth s Calculate the violation degree column. Here n s This represents the total number of sensors. Note that k = 1, 2, n s That is the case.

[0116] The rule column can be represented, for example, by the following formula 1.

[0117]

Number

[0118] The violation degree sequence output by the k-th guideline to the guideline integration 221 is represented, for example, by the following Equation 2.

[0119]

Number

[0120] The evaluation of the rules, that is, the calculation of equation 2 in response to the input of equation 1, may be implemented solely by a computer program, or it may be implemented by a trained model using artificial intelligence.

[0121] Guideline Integration 221 is an integrator that combines the evaluation results of rules from each guideline 211, 212, and 21n. Guideline Integration 221 calculates the combined violation score using an integration function that combines the violation scores.

[0122] The integration function can be expressed, for example, by the following equation 3.

[0123]

number

[0124]

number

[0125] Here, we will explain specific examples 1 and 2 of the calculation. In example 1, the degree of violation of the rule is calculated from three sensor groups. Assume that there is a rule that "there are no objects in the path of a given vehicle." In this rule, each guideline outputs a violation degree of 0 if it is determined that there are no objects in the path, and a violation degree of 1 if it is determined that there are objects.

[0126] Here, we consider a case where there is actually no object in the path, but the first sensor group mistakenly identifies a ghost in the path as an object, and the second and third sensor groups do not recognize the ghost in the path. In this case, the first guideline outputs a violation degree of 1, and the second and third guidelines output a violation degree of 0. The integrated guideline adopts the median value of 0 as the violation degree after integration.

[0127] Furthermore, consider a case where there is an object in the path, but the first sensor group fails to recognize the object due to recognition disturbances, etc., while the second and third sensor groups are able to recognize the object in the path. In this case, the first guideline outputs a violation degree of 0, and the second and third guidelines output a violation degree of 1. The integrated guideline adopts the median value of 1 as the violation degree after integration.

[0128] In specific example 2, the degree of rule violation is calculated from five sensor groups. "Lateral distance d to the stopped vehicle" lat Assume there is a rule that states, "Ensure that the threshold d0 is greater than or equal to the threshold d0."

[0129] In this rule, the degree of violation shall be expressed by the following formula 5.

[0130]

number

[0131] Actual lateral distance d lat When the threshold d0 is smaller than the threshold, and only the first sensor group has a lateral distance d lat Consider the case where a false detection occurs when the detection distance is greater than d0. In this case, the first guideline outputs a violation degree of 0, while the other second to fifth guidelines output violation degrees corresponding to their respective detection distances. The integrated guideline uses the median as the integrated violation degree, but four of the five guidelines use the horizontal distance d, although there is some error. lat It is determined that this is less than the threshold d0. Therefore, the degree of violation after integration is the lateral distance d lat This indicates a violation degree that is less than the threshold d0.

[0132] The planning processor 230 plans driving behavior based on the integrated violation degree output from the guideline integration 221, the rule set stored in the rule DB 58, and the scenario data stored in the scenario DB 59.

[0133] The planning processor 230 refers to the integrated violation score and derives a driving action that allows vehicle 1 to avoid the violation. There may be cases where vehicle 1 cannot avoid the violation. In this case, the planning processor 230 derives a driving action that minimizes the violation score. In minimizing the violation score, the priority structure in the rule set may be referred to.

[0134] The trajectory of vehicle 1 consists of a series of positions over the course of the scenario. Therefore, the planning processor 230 may derive a driving action by aggregating instantaneous violations over time. The aggregation may be, for example, by accumulating the violations over time. The derived driving action may depend on whether the rule was violated minorly over a long period or seriously over a short period. The derived driving action may depend on at least one of the average violations over time and the maximum violations over time.

[0135] Here, we will explain the points of agreement and differences in rule evaluation between guidelines 211, 212, and 21n. In one method, each guideline 211, 212, and 21n may be configured to evaluate multiple rules in a rule set that are common to each other. This configuration, which evaluates common rules, is particularly suitable when combined with a classification that covers all directions (360 degrees) around the vehicle using a single sensor group. Furthermore, because common rules are being evaluated, the effect of improving evaluation accuracy through integration between each guideline is significant.

[0136] Each guideline 211, 212, and 21n may be configured to evaluate only some of the rules in the rule set, rather than all of the rules stored in the rule DB 58. Furthermore, the rules evaluated may differ in some or all ways among the guidelines 211, 212, and 21n. In this case, the rule DB 58 may additionally store information on which guideline 211, 212, and 21n each rule in the rule set applies to.

[0137] If a guideline contains rules in a rule column that are not subject to evaluation, the violation degree of those rules not calculated in that guideline should not return a valid numerical value. Not returning a valid numerical value may include returning an invalid numerical value, or not setting a numerical value at all and maintaining the initial state. An invalid numerical value may be a negative number or a number greater than 1, for example, if the valid numerical values ​​for the violation degree are shown in the range of 0 to 1. The initial state may be a Null Value or a Null Character. If the integration target by Guideline Integration 221 includes violation degrees for which no valid numerical value is returned, Guideline Integration 221 will treat those violation degrees as non-existent (invalid) and calculate the integrated violation degree only from the violation degrees of the other guidelines.

[0138] The configuration for evaluating differing rules among guidelines 211, 212, and 21n is particularly suitable for combination with classification by type or by direction. For example, a group of sensors classified by a specific sensor type can be made to evaluate rules related to scenes in which that sensor type excels, while excluding the evaluation of rules related to scenes in which it does not excel. This improves the accuracy of the final violation assessment before integration.

[0139] Furthermore, even when performing calculations based on the same rule, the algorithms, parameters, etc., used in each guideline 211, 212, and 21n may differ from one another. Since the data format, coordinate system, dimensions, resolution, reliability, error, and timing delay effects of the sensor data input to guidelines 211, 212, and 21n may differ for each sensor group, algorithms, parameters, etc., that are adjusted according to these factors may be adopted.

[0140] As a concrete example, consider the case where the rule set is common among multiple guidelines 211, 212, and 21n. Ultrasonic sonar is suitable for detecting objects at short distances. Therefore, the guideline corresponding to a sensor group mainly composed of ultrasonic sonar may be configured to be used only for calculating the degree of violation of rules targeting objects at short distances, and not for calculating the degree of violation of rules targeting objects at long distances. On the other hand, the communication system 43 is suitable for detecting information at long distances or in blind spots that cannot be detected by cameras, LiDAR, ultrasonic sonar, etc. Therefore, the guideline corresponding to a sensor group mainly composed of the communication system 43 may be configured to be used for calculating the degree of violation of rules targeting objects at long distances and in blind spots, and not for calculating the degree of violation of other rules. In other words, each guideline 211, 212, and 21n is based on a common rule set, and depending on the difference in the source of the sensor data output, some of the multiple rules included in the rule set that differ from each other are excluded from evaluation. In this way, each guideline 211, 212, and 21n may evaluate rules that differ from each other in some respects among the multiple rules.

[0141] <Handling of related data> The driving system 2 is configured to record sufficient relevant data for accident analysis. The relevant data may include calculation-related data from the guideline processor 200 and the planning processor 230. The guideline processor 200 and the planning processor 230 sequentially output the calculation-related data to the recording device 55. The recording device 55 sequentially stores the relevant data in the memory 55a.

[0142] The calculation-related data may include the violation degree or violation degree column data itself calculated in each guideline 211, 212, and 21n. The calculation-related data may further include the integrated violation degree or violation degree column data associated with the violation degree or violation degree column calculated in each guideline 211, 212, and 21n.

[0143] Calculation-related data is data associated with the violation degree or violation degree column data calculated in each guideline 211, 212, and 21n, and may further include the sensor data on which the violation degree or violation degree column calculation was based. If the sensor data includes camera data, images captured by the camera may be recorded. If the sensor data includes LiDAR data, point cloud data representing the reflection positions of reflected light may be recorded.

[0144] The calculation-related data is data associated with the violation degree or violation degree column data calculated in each guideline 211, 212, and 21n, and may further include selection information of the scenario on which the violation degree or violation degree column calculation was based.

[0145] The guideline processor 200 or other processor (e.g., an anomaly detection processor) 51b may further have a function to detect failures or false detections of sensor groups 101, 102, 10n based on calculation-related data. Failures or false detections of any sensor group 101, 102, 10n may be determined by whether the absolute value of the difference between the violation degree calculated using the sensor data of the sensor group 101, 102, 10n and the integrated violation degree adopted by the guideline integration 221 is greater than or equal to a detection threshold.

[0146] For example, consider a case where the violation scores output by three sensor groups are 0, 0.1, and 1 respectively, and the combined violation score is the median value of 0.1. If the detection threshold is set to 0.5, the sensor group that outputted a violation score of 1 will be judged to have malfunctioned or falsely detected because the absolute value of the difference is 0.8 (>0.5).

[0147] In configurations that determine such failures or false positives, the determination result may be associated with and further recorded the degree of violation or the degree of violation column calculated in each guideline.

[0148] The guideline processor 200 or other processor 51b may, in detecting failures or false positives, further classify and detect permanent failures of the sensor group and temporary false positives due to the characteristics of the sensor group (e.g., unfavorable scenes). These classification results may be further recorded, associated with the degree of violation or the degree of violation column calculated by each guideline.

[0149] The operating system 2 or recording device 55 may generate recording data such that it stores at least one of the following in a dedicated data format for recording: the degree of violation or the degree of violation calculated in each guideline 211, 212, 21n; the rule column; the integrated degree of violation or the degree of violation column; the sensor data for each sensor group 101, 102, 10n; the scenario selection information in each guideline 211, 212, 21n; and the judgment result of failure or false detection for each sensor group 101, 102, 10n. In this case, only the data relating to the sensor group in which a failure or false detection was detected among the data 101, 102, 10n relating to multiple sensor groups may be generated and recorded.

[0150] The results of the fault or false detection determination may be used for various actions other than recording. For example, constraints may be set so that the sensor group in which a fault or false detection was detected is not used for the function of the driving system 2 (e.g., driving plan). The setting of constraints may be performed by the mode management unit 23.

[0151] For example, notification may be provided regarding the presence of sensor group 101, 102, 10n that has been detected as malfunctioning or falsely detecting something. Specifically, the driving system 2 may provide information to the driver about the abnormality of sensor group 101, 102, 10n using an information-presenting HMI device 70. The driving system 2 may also notify third parties such as external systems 96, remote centers, the vehicle operation management company, the vehicle seller, the vehicle manufacturer, sensor manufacturers, other vehicles, administrative agencies managing traffic infrastructure, and certification bodies that certify the safety of autonomous driving systems, etc., through the communication system 43.

[0152] <Processing Flow> Next, an example of a processing method for realizing the driving function will be explained using the flowchart in Figure 9. The series of processes shown in steps S11 to S15 are executed by the driving system 2 at predetermined intervals or based on predetermined triggers. Specifically, the series of processes may be executed at predetermined intervals when the automatic driving mode is managed to automatic driving level 3 or higher. As another specific example, the series of processes may be executed at predetermined intervals when the automatic driving mode is managed to automatic driving level 2 or higher.

[0153] In S11, the latest sensor data is acquired from the paired sensor groups 101, 102, and 10n, respectively, for each guideline 211, 212, and 21n. After processing in S11, the process proceeds to S12.

[0154] In S12, each guideline 211, 212, and 21n retrieves a rule from rule DB58 and evaluates the rule using the sensor data acquired in S1. After processing in S12, the process proceeds to S13.

[0155] In S13, the guideline integrator 221 integrates the evaluation results of the rules in each guideline 211, 212, and 21n, and outputs the integrated evaluation results to the planning processor 230. After processing in S13, the process proceeds to S14.

[0156] In S14, the planning processor 230 plans the driving behavior of vehicle 1 based on the integrated evaluation results. After processing in S14, the process proceeds to S15.

[0157] In S15, at least one of the guideline processor 200 and the planning processor 230 generates recording data and outputs the recording data to the recording device 55. The recording device 55 stores the recording data in the memory 55a. The series of processes ends with S15.

[0158] According to the first embodiment described above, evaluations of strategic guidelines are performed individually, with at least some of the sensor data output sources differing from one another. This process makes it possible to analyze the causal relationship between the final driving behavior and the sensors by breaking it down into individual evaluation results. Therefore, traceability to planned driving behavior can be improved.

[0159] Furthermore, according to the first embodiment, each guideline 211, 212, 21n outputs a matrix of violation metrics, which are individual evaluation results for a rule set containing multiple rules. By comparing the matrices of multiple violation metrics output individually based on sensor data from at least some different sources, it becomes easier to identify the causal relationship between the violation metrics used to derive driving behavior and the sensors. Therefore, traceability to driving behavior can be improved.

[0160] Furthermore, according to the first embodiment, a matrix of integrated violation metrics is generated based on the matrices of multiple violation metrics output from each guideline 211, 212, and 21n, and output as the integrated evaluation result. Therefore, it becomes possible to derive driving behavior by appropriately reflecting each sensor data.

[0161] Furthermore, according to the first embodiment, the group of sensors that experienced a failure or false detection is identified based on the matrix of each violation metric. Therefore, traceability to driving behavior can be improved.

[0162] Furthermore, according to the first embodiment, each of the multiple guidelines 211, 212, and 21n evaluates multiple rules common to each other based on a rule set commonly provided among them. By standardizing the evaluation targets, the integration processing of evaluation results can be made highly accurate and easy.

[0163] Furthermore, according to the first embodiment, each guideline 211, 212, and 21n performs evaluation of the same rule using different algorithms depending on the differences in the source of the sensor data output. In this way, rule evaluation can be appropriately performed for various types of sensors.

[0164] Furthermore, according to the first embodiment, each guideline 211, 212, and 21n performs evaluation of the same rule using the same algorithm and different parameters depending on the differences in the source of the sensor data output. In this way, the rule evaluation can be performed appropriately according to the differences in the characteristics of the source sensor.

[0165] Furthermore, according to the first embodiment, based on a common rule set established among multiple guidelines 211, 212, and 21n, some of the multiple rules included in the rule set are excluded from evaluation depending on the differences in the source of the sensor data output. As a result, rules that differ in some respects from each other are evaluated. By performing evaluations focused on the appropriate rules according to the differences in the characteristics of the source sensor, it is possible to exclude individual evaluation results that are expected to have low accuracy. Since the accuracy of individual evaluation results will be good, the validity of the integrated evaluation results can be improved.

[0166] Furthermore, according to the first embodiment, data is generated that associates the individual evaluation results with the integrated evaluation results, and is stored in memory 55a as a storage medium. By storing the evaluation results together, it becomes possible to appropriately perform post-evaluation verification and validation.

[0167] Furthermore, according to the first embodiment, multiple guidelines 211, 212, and 21n are implemented by a single common processor 200. By using a common processor 200, it becomes unnecessary to aggregate information such as individual evaluation results between devices during integration, thus enabling the integration process to be executed with reduced latency.

[0168] In the first embodiment, guidelines 211, 212, and 21n correspond to individual evaluation units. Guideline integration 221 corresponds to integrated evaluation unit.

[0169] (Second Embodiment) As shown in Figure 10, the second embodiment is a modification of the first embodiment. The second embodiment will be described focusing on the differences from the first embodiment.

[0170] The driving system 2 of the second embodiment includes a plurality of sensor groups 101, 102, 10n, a plurality of guideline processors 201, 202, 20n, 220, and a planning processor 230. The guideline processors 201, 202, 20n, and 220 are provided in a number equal to the total number of sensor groups 101, 102, 10n plus one. Each guideline processor 201, 202, 20n includes the same number of individual processors 201, 202, 20n as the sensor groups 101, 102, 10n, and one integrated processor 220.

[0171] Each individual processor 201, 202, and 20n corresponds individually to one of the sensor groups 101, 102, and 10n. Each individual processor 201, 202, and 20n implements one guideline 211, 212, and 21n respectively, using sensor data input from the paired sensor groups 101, 102, and 10n, by executing a computer program. The processing of the guidelines 211, 212, and 21n is the same as in the first embodiment.

[0172] The integrated processor 220 obtains the violation degree or violation degree sequence calculated by each individual processor 201, 202, and 20n, and implements guideline integration 221, which integrates these, by executing a computer program. The processing of guideline integration 221 is the same as in the first embodiment.

[0173] Multiple guideline processors 201, 202, 20n, 220 and the planning processor 230 may be mounted on a common board. Multiple guideline processors 201, 202, 20n, 220 and the planning processor 230 may be mounted on separate boards. Furthermore, multiple individual processors 201, 202, 20n and the integrated processor 220 may be mounted on separate boards. Multiple individual processors 201, 202, 20n may be mounted on a common board, or they may each be mounted on separate boards.

[0174] The rule DB58 may be provided in common for multiple individual processors 201, 202, and 20n. In this case, each individual processor 201, 202, and 20n accesses the common rule DB58 and refers to the rule set.

[0175] On the other hand, multiple rule DB58s may be provided to correspond individually to each individual processor 201, 202, and 20n. In this configuration, the rule set can be optimized for each guideline 211, 212, and 21n. In other words, it becomes possible to evaluate rules suitable for each sensor group 101, 102, and 10n in a suitable manner.

[0176] According to the second embodiment described above, the multiple guidelines 211, 212, and 21n are each implemented by separate processors 201, 202, and 20n, each corresponding to an individual. This allows the evaluation to continue even if some of the processors 201, 202, and 20n malfunction. Therefore, the redundancy of the processing system 50 can be improved.

[0177] (Third embodiment) As shown in Figure 11, the third embodiment is a modification of the first embodiment. The third embodiment will be described focusing on the differences from the first embodiment.

[0178] The operating system 2 of the second embodiment includes a plurality of sensor groups 101, 102, 10n and a plurality of processors 241, 242, 24n, 260. The number of processors 241, 242, 24n, 260 is the total number of sensor groups 101, 102, 10n plus one. Each processor 241, 242, 24n includes the same number of individual processors 241, 242, 24n as the number of sensor groups 101, 102, 10n, and one integrated processor 260.

[0179] Each individual processor 201, 202, and 20n corresponds individually to one of the sensor groups 101, 102, and 10n. Each individual processor 201, 202, and 20n implements one guideline 211, 212, and 21n, and one plan 251, 252, and 25n, respectively, by executing a computer program, using the sensor data input from the paired sensor groups 101, 102, and 10n.

[0180] Each of the plans 251, 252, and 25n is a planner that refers to the degree of violation or the degree of violation sequence obtained from the corresponding guidelines 211, 212, and 21n, respectively, to derive driving actions that allow vehicle 1 to avoid violations or driving actions that minimize violations.

[0181] The processing of guidelines 211, 212, and 21n is the same as in the first embodiment. On the other hand, planning 251, 252, and 25n are provided individually for each sensor group 101, 102, and 10n and each guideline 211, 212, and 21n. That is, each planning 251, 252, and 25n derives driving behavior and outputs it to the integrated processor 260.

[0182] The integrated processor 260 integrates multiple driving actions derived by each individual processor 241, 242, and 24n into one. The integrated processor 260 may, for example, decide by majority vote whether or not to change the lane of vehicle 1 from the driving actions derived by each individual processor 241, 242, and 24n.

[0183] According to the third embodiment described above, evaluations of strategic guidelines are performed individually, with at least some of the sensor data output sources differing from one another. This process makes it possible to analyze the causal relationship between the final driving behavior and the sensors by breaking it down into individual evaluation results. Therefore, traceability to planned driving behavior can be improved.

[0184] In the third embodiment, the guidelines 211, 212, and 21n correspond to the individual evaluation units. The planning units 241, 242, and 24n correspond to the individual operation planning units. The integrated planning unit 261 corresponds to the integrated operation planning unit.

[0185] (Fourth Embodiment) As shown in Figures 12 and 13, the fourth embodiment is a modification of the first embodiment. The fourth embodiment will be described focusing on the differences from the first embodiment.

[0186] In the fourth embodiment, the rule set is used to check or monitor the operation plan. The operation system 2 is configured to include a plurality of sensor groups 101, 102, 10n, a guideline processor 300, a sensor fusion processor 160, and a planning processor 330 (see Figure 12). The functions implemented by the guideline processor 300 may correspond to some of the functions of the prediction unit 21 and the operation planning unit 22. The functions implemented by the sensor fusion processor 160 may correspond to some of the functions of the recognition unit 10. The functions implemented by the planning processor 330 may correspond to at least some of the functions of the operation planning unit 22.

[0187] The sensor fusion processor 160 acquires sensor data from multiple sensor groups 101, 102, and 10n, fuses this sensor data, and generates an environmental model. The planning processor 330 uses this environmental model to derive driving behavior. Here, the planning processor 330 provides information on the derived hypothetical driving behavior. The hypothetical driving behavior is a candidate for the driving behavior to be executed.

[0188] The guideline processor 300 evaluates the driving behavior provided by the planning processor 330 using a rule set. Specifically, each guideline 311, 312, and 31n uses sensor data from paired sensor groups 101, 102, and 10n, a rule set, and a scenario structure to determine whether the driving behavior violates the rules of the rule set. Each guideline 311, 312, and 31n outputs a degree of violation or a sequence of violations to the guideline integrator 321, similar to the first embodiment. The guideline integrator 321 integrates the degree of violation or the sequence of violations input from each guideline 311, 312, and 31n. The guideline integrator 321 outputs the integrated degree of violation or the sequence of violations to the planning processor 330 as the result of the rule evaluation for the driving behavior. Based on this evaluation result, the planning processor 330 determines the final driving behavior of vehicle 1.

[0189] Here, the planning processor 330 may have the guideline processor 300 compare multiple driving behaviors. In this case, the guideline processor 300 may calculate a separate degree of violation or a series of violations for each driving behavior and output the difference in performance of each driving behavior to the planning processor 330 as an evaluation result.

[0190] Next, an example of a processing method for realizing the driving function will be explained using the flowchart in Figure 13. The series of processes shown in steps S21 to S25 are executed by the driving system 2 at predetermined intervals or based on predetermined triggers. Specifically, the series of processes may be executed at predetermined intervals when the automatic driving mode is managed to automatic driving level 3 or higher. As another specific example, the series of processes may be executed at predetermined intervals when the automatic driving mode is managed to automatic driving level 2 or higher.

[0191] In S21, the sensor fusion processor 160 fuses the sensor data from multiple sensor groups 101, 102, and 10n. After processing in S21, the process proceeds to S22.

[0192] In S22, the planning processor 330 calculates a provisional driving action. After processing in S22, the process proceeds to S23.

[0193] In S23, the guideline processor 300 evaluates the rules against the hypothetical driving behavior calculated in S22. After processing in S23, the process proceeds to S24.

[0194] In S24, the planning processor 330 refers to the evaluation results from S23 and determines the final driving action. After processing in S24, the process proceeds to S25.

[0195] In S25, at least one of the guideline processor 300 and the planning processor 330 generates recording data and outputs the recording data to the recording device 55. The recording device 55 stores the recording data in the memory 55a. The series of processes ends in S25.

[0196] According to the fourth embodiment described above, guidelines 311, 312, and 31n output individual evaluation results regarding the strategic guidelines for a hypothetical driving action. Then, each individual evaluation result for the hypothetical driving action is integrated to output an integrated evaluation result. The final driving action is determined by referring to this integrated evaluation result for the hypothetical driving action. In this way, the rule set can be used for a monitoring function to determine whether the driving plan is appropriate, thereby increasing the safety of the driving system 2.

[0197] In the fourth embodiment, guidelines 311, 312, and 31n correspond to individual evaluation units. Guideline integration 321 corresponds to integrated evaluation unit.

[0198] (Fifth embodiment) As shown in Figure 14, the fifth embodiment is a modification of the first embodiment. The fifth embodiment will be described focusing on the differences from the first embodiment.

[0199] In the fifth embodiment, the generated recording data (for example, relevant data related to accident investigation) may be transmitted to an external system 96 via communication through the communication system 43 (for example, V2X communication) and stored in the storage medium 98 of the external system 96. When transmitting the recording data to the external system 96, it may be generated and transmitted as encoded data that conforms to a specific format, such as an SDM (Safety Driving Model) message. The transmission to the external system 96 does not have to be a direct transmission of radio waves from the vehicle 1 to the external system 96, but may be a transmission using a relay terminal such as a roadside unit and a network.

[0200] The external system 96 includes a dedicated computer 97 having at least one memory 97a and one processor 97b, and at least one large-capacity storage medium 98. In the dedicated computer 97, the memory 97a may be at least one type of non-transitional tangible storage medium, such as semiconductor memory, magnetic media, and optical media, which non-temporarily stores programs and data that can be read by the processor 97b. Furthermore, the memory 97a may also include a rewritable volatile storage medium, such as RAM (Random Access Memory). The processor 97b includes at least one type as a core, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and RISC (Reduced Instruction Set Computer)-CPU. The storage medium 98 may be at least one type of non-transitional tangible storage medium, such as semiconductor memory, magnetic media, and optical media.

[0201] The external system 96 decodes the received message. The external system 96 may then sequentially record the recorded data in a memory area reserved within the storage medium 98 for each vehicle. The external system 96 may also collect recorded data from a large number of vehicles traveling on the road and sequentially record the recorded data in a common memory area.

[0202] The accumulated data may be used as big data for developing the road network. For example, the external system 96 identifies accident-prone locations from the accumulated data and further identifies the rules that are frequently violated at those accident-prone locations. This makes it possible to modify the road structure at accident-prone locations to make it less likely for violations of the identified rules to occur.

[0203] The accumulated data may be used to verify and validate the driving system 2 or the safety model on which it is based, in order to realize an efficient SOTIF process. For example, the driving system 2 can be improved by analyzing the causal relationship between sensor data, the violation degree calculated accordingly, and the driving behavior of vehicle 1. Improvements to the driving system 2 include at least one of the following: improvement of the rule evaluation algorithm and improvement of the rule set. Improvements to the rule set include at least one of the following: modification of the rules themselves and modification of the priority structure.

[0204] According to the fifth embodiment described above, data is generated that links individual evaluation results with the integrated evaluation results. This data is transmitted to an external system 96 located outside of vehicle 1 via a communication system 43 installed in vehicle 1. By transmitting the evaluation results to an external system in a consolidated manner, it becomes easier to perform post-event verification and validation even if data located inside vehicle 1 is damaged due to an accident or the like. Furthermore, it becomes easier for the external system 96 to aggregate and utilize data from multiple vehicles.

[0205] (Other embodiments) Although several embodiments have been described above, this disclosure is not intended to be limited to those embodiments, and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0206] In another embodiment, the recorded data of the second to fourth embodiments may be transmitted to an external system 96 via communication through the communication system 43 (e.g., V2X communication), similar to the fifth embodiment, and stored in the storage medium 98 of the external system 96.

[0207] In another embodiment, the guideline processor 200, rule DB 58, and scenario DB 59 may be installed in a vehicle driven by a driver. For example, the degree of violation or the sequence of violations output by the guideline processor 200 may be recorded as recorded data by the recording device 55, and this recorded data may be used to evaluate the manual driving.

[0208] Alternatively, for example, the rule evaluation results from the guideline processor 200 may be presented to the driver during manual operation by an information-presenting HMI device 70. In this example, information presentation for rules with a violation degree of 0 may be omitted, and only information presentation for rules with a violation degree of a predetermined threshold or higher may be provided. The threshold may be 0.5 or 1. In this way, it is preferable to select which rules to present information on, taking into consideration the inconvenience the driver may feel.

[0209] The control unit and method described herein may be implemented by a dedicated computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the apparatus and method described herein may be implemented by a dedicated hardware logic circuit. Alternatively, the apparatus and method described herein may be implemented by one or more dedicated computers comprising a combination of a processor that executes a computer program and one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium.

[0210] (Explanation of terms) Terms related to this disclosure are defined below. This definition is included in embodiments of this disclosure.

[0211] A road user may be a person using the road, including sidewalks and other adjacent spaces. Road users may also include pedestrians, cyclists, other VRUs, and vehicles (e.g., human-driven cars, vehicles equipped with autonomous driving systems).

[0212] A dynamic driving task (DDT) may be a real-time operational and tactical function for controlling a vehicle in traffic.

[0213] An automated driving system may be a set of hardware and software capable of continuously performing the entire DDT, regardless of whether it is limited to a specific operational design domain.

[0214] SOTIF (safety of the intended functionality) may be the absence of undue risk resulting from insufficient functionality of the intended function or its implementation.

[0215] A driving policy may be a set of strategies and rules that define control actions at the vehicle level.

[0216] A safety-relevant object may be any dynamic or static object that may be related to the safety performance of the DDT.

[0217] A scenario may be a description of the temporal relationships between several scenes within a series of scenes, including goals and values ​​in a specific situation influenced by actions and events. A scenario may also be a description of a continuous time series of activities integrating the subject vehicle, all its external environment, and their interactions in the process of performing a specific driving task.

[0218] A triggering condition may be a specific condition in a scenario that acts as a trigger for a subsequent system response that contributes to the failure to prevent, detect, or mitigate dangerous behavior or reasonably foreseeable indirect misuse.

[0219] A strategic guideline may be at least one state and at least one appropriate driving action associated with that state. A strategic guideline is broadly used to indicate any expression, explanation, description, definition, or logical relationship derived from one or more fundamental principles. A strategic guideline may be synonymous with a logical expression.

[0220] A hazardous situation may be an increased risk of potential breaches of the safety envelope, and also represents the increased risk level present in the DDT.

[0221] A safety envelope may be a set of limitations and conditions designed to ensure that an (automated) driving system operates as a constraint or control in order to maintain operations within an acceptable level of risk. A safety envelope may be a general concept that can be used to address all principles to which a driving policy may adhere, under which a vehicle operated by an (automated) driving system may have one or more boundaries around it.

[0222] (Disclosure of technical ideas) This specification discloses several technical concepts, as listed in the following paragraphs. Some paragraphs are written in a multiple dependent form, where subsequent paragraphs alternately refer to preceding paragraphs. These paragraphs written in a multiple dependent form define several technical concepts.

[0223] <Technical philosophy 1> A processing system that performs processing related to the operation of a vehicle (1), A plurality of individual evaluation units that output individual evaluation results regarding strategic guidelines based on sensor data, wherein at least a portion of the sensor data output sources differ from each other (211, 212, 21n, 311, 312, 31n), An integrated evaluation unit (221, 321) integrates each of the individual evaluation results and outputs an integrated evaluation result, A processing system comprising: an operation planning unit (22) that plans operation actions based on the evaluation results after integration.

[0224] <Technical philosophy 2> Each of the individual evaluation units outputs a matrix of violation metrics, which are the individual evaluation results for a rule set containing multiple rules, according to the processing system described in Technical Concept 1.

[0225] <Technical philosophy 3> The processing system according to technical concept 2, wherein each integrated evaluation unit generates a matrix of integrated violation metrics based on the matrix of multiple violation metrics output from each individual evaluation unit, and outputs it as the integrated evaluation result.

[0226] <Technical philosophy 4> A processing system according to technical concept 2 or 3, which identifies a group of sensors that have failed or have falsely detected based on a matrix of each of the aforementioned violation metrics.

[0227] <Technical philosophy 5> The aforementioned rule set is provided in common among the multiple individual evaluation units, Each of the aforementioned individual evaluation units evaluates a plurality of rules common to each other based on the rule set, in a processing system according to any one of technical ideas 2 to 4.

[0228] <Technical philosophy 6> The processing system according to technical concept 5, wherein each of the individual evaluation units performs an evaluation using a different algorithm corresponding to the difference in the source of the sensor data output for the same rule.

[0229] <Technical philosophy 7> The processing system according to technical concept 5, wherein each of the individual evaluation units performs an evaluation for the same rule using the same algorithm and different parameters according to the differences in the source of the sensor data output.

[0230] <Technical philosophy 8> The aforementioned rule set is provided in common among the multiple individual evaluation units, Each of the individual evaluation units evaluates the rules that differ from each other in part from the other, based on the rule set, by excluding some of the rules included in the rule set from evaluation depending on the difference in the source of the sensor data, as described in any one of the technical ideas 2 to 4.

[0231] <Technical philosophy 9> A processing system according to any one of the technical ideas 1 to 8, which generates associated data of the individual evaluation results and the integrated evaluation results and stores it in a storage medium (55a).

[0232] <Technical Thought 10> A processing system according to any one of technical ideas 1 to 9, which generates associated data of the individual evaluation results and the integrated evaluation results, and transmits it to an external system (96) located outside the vehicle via a communication system (43) mounted on the vehicle.

[0233] <Technical Thought 11> The processing system described in any one of the technical concepts 1 to 10, wherein the multiple individual evaluation units are implemented by a single common processor (200, 300).

[0234] <Technical Thought 12> The processing system according to any one of the technical concepts 1 to 10, wherein each of the multiple individual evaluation units is implemented by a separate processor (201, 202, 20n) that corresponds to it individually.

[0235] <Technical Thought 13> The aforementioned driving planning unit derives a provisional driving action, The individual evaluation unit outputs individual evaluation results regarding strategic guidelines for the hypothetical driving behavior, The integrated evaluation unit integrates the individual evaluation results for the provisional driving actions and outputs the integrated evaluation result. The operation planning unit is a processing system according to any one of the technical concepts 1 to 12, which determines the final operation by referring to the integrated evaluation results of the provisional operation.

[0236] <Technical Thought 14> A processing system that performs processing related to the operation of a vehicle (1), A plurality of individual evaluation units that output individual evaluation results regarding strategic guidelines based on sensor data, wherein at least a portion of the output sources of the sensor data differ from each other (211, 212, 21n), Each of the individual evaluation units is provided to correspond to a plurality of individual driving planning units (251, 252, 25n) which plan individual driving actions based on the individual evaluation results output by the pair of individual evaluation units, A processing system comprising: an integrated driving plan unit (261) that integrates each of the individual driving actions and plans the integrated driving action.

[0237] <Technical Thought 15> A processing system that performs processing related to the operation of a vehicle (1), Guideline processor (200), Equipped with a planning processor (230), The aforementioned guideline processor, A function that outputs individual evaluation results regarding strategic guidelines based on sensor data, comprising multiple guideline functions (211, 212, 21n, 311, 312, 31n) in which at least a portion of the sensor data output sources differ from each other, A guideline integration function (221,321) is implemented that integrates each of the aforementioned individual evaluation results and outputs the integrated evaluation result. The aforementioned planning processor, A processing system that outputs a driving action plan in response to the input of the integrated evaluation results.

[0238] This technological concept allows for improved traceability of planned driving behavior.

[0239] <Technical Thought 16> A processing system that performs processing related to the operation of a vehicle (1), Multiple guideline processors (211, 212, 21n, 220), Equipped with a planning processor (230), Multiple of the aforementioned guideline processors, A function that outputs individual evaluation results regarding strategic guidelines based on sensor data, comprising individual processors that each implement a plurality of guideline functions (211, 212, 21n, 311, 312, 31n) in which at least a portion of the sensor data output sources differ among the plurality of guideline processors, It has a guideline integration function (221, 321) that integrates each of the individual evaluation results and outputs the integrated evaluation result, and an integrated processor that realizes this, The aforementioned planning processor, A processing system that outputs a driving action plan in response to the input of the integrated evaluation results.

[0240] This technological concept allows for improved traceability of planned driving behavior.

[0241] <Technical Thought 17> A processing system that performs processing related to the operation of a vehicle (1), Multiple individual processors (211, 212, 21n), Equipped with an integrated processor (260), Each of the aforementioned individual processors A function that outputs individual evaluation results regarding strategic guidelines based on sensor data, comprising multiple guideline functions (211, 212, 21n) in which at least a portion of the sensor data output sources differ among the multiple individual processors, An individual planning function (251, 252, 25n) that outputs an individual driving action plan based on the individual evaluation results, The integrated processor, A processing system that integrates each of the individual driving actions and outputs an integrated driving action plan.

[0242] According to this technical idea, the traceability of the planned driving action can be improved.

[0243] <Technical Idea 18> A method of executing processing related to a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220), comprising: Obtaining individual sensor data from a plurality of sensor groups (101, 102, 10n) respectively; Based on the individual sensor data, individually executing individual evaluations regarding strategic guidelines in parallel; Integrating the results of each of the individual evaluations and outputting an integrated evaluation result.

[0244] According to this technical idea, the traceability in processing related to the vehicle can be improved.

[0245] <Technical Idea 19> A method of executing processing related to a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220, 241, 242, 24n), comprising: Referring to the scenario structure stored in the scenario database (59) to identify an unknown scenario that the vehicle is encountering; Evaluating the rules defined in the rule set in the unknown scenario based on the rule set stored in the rule database (58) and the identification result.

[0246] According to this technical idea, it becomes possible to provide feedback on an unknown scenario that the vehicle has encountered.

[0247] <Technical idea 20> A method of executing processing related to a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220, 241, 242, 24n), comprising: Identifying an unknown scenario encountered by the vehicle by referring to a scenario structure stored in a scenario database (59);

[0248] Identifying, based on a rule set stored in a rule database (58) and the identification result, a rule that is difficult for the vehicle to follow among the rules defined in the rule set in the unknown scenario.

[0248] According to this technical idea, it becomes possible to provide feedback on an unknown scenario encountered by the vehicle.

[0249] <Technical idea 21> A recording medium for storing data related to the driving behavior of a vehicle (1), comprising: The value of a violation metric associated with a driving behavior that violates a rule statement defined in a rule set; The sensor data used for calculating the violation metric; And storing them in association with each other.

[0250] According to this technical idea, the traceability of the driving behavior of the vehicle can be improved.

[0251] <Technical idea 22> A method of generating data related to the driving behavior of a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220, 241, 242, 24n), comprising: Calculating, using sensor data, the value of a violation metric associated with a driving behavior that violates a rule statement defined in a rule set; A method for generating data stored in a dedicated data format such that the value of the violation metric and the sensor data used to calculate the violation metric are associated.

[0252] This technological concept can improve traceability of vehicle driving behavior.

[0253] <Technical Thought 23> A recording medium for storing data relating to the driving behavior of a vehicle (1), The values ​​of violation metrics associated with driving behavior that violates the rule statements defined in the rule set, which are calculated using sensor data from which at least some output sources differ from one another, The value of the integrated violation metric obtained by integrating the values ​​of the multiple violation metrics, A storage medium that stores information in association with other information.

[0254] This technological concept can improve traceability of vehicle driving behavior.

[0255] <Technical Thought 24> A method for generating data relating to the driving behavior of a vehicle (1) using at least one processor (51b, 200, 201, 202, 20n, 220), The calculation of multiple violation metric values ​​associated with driving behavior that violates rule statements defined in the rule set, each calculated using sensor data from which at least some of the output sources differ from one another, The values ​​of the aforementioned multiple violation metrics are integrated, and the value of the integrated violation metric is calculated. A method for generating data stored in a dedicated data format such that the values ​​of the multiple violation metrics before integration are associated with the values ​​of the violation metrics after integration.

[0256] This technological concept can improve traceability of vehicle driving behavior.

[0257] <Technical Thought 25> A system comprising at least one processor (97b) and at least one storage medium (98) for aggregating information of multiple vehicles, The aforementioned at least one processor is A message is received from the vehicle, which includes the value of a violation metric associated with a driving behavior that violates a rule statement defined in the rule set, and the sensor data used to calculate the violation metric. A system for storing the value of the violation metric and the sensor data in the at least one storage medium.

[0258] This technological concept can improve traceability of vehicle driving behavior.

[0259] <Technical Thought 26> A driving system that performs a dynamic driving task for a vehicle (1), Multiple sensors (40) installed on the vehicle are classified and configured into multiple sensor groups (101, 102, 10n), each of which provides sensor data. A set of rules that implements a priority structure for a series of rules arranged based on relative importance is stored in a rule database (58), A scenario database (59) that stores a scenario structure including multiple scenarios showing the vehicle, the external environment, and their interactions in the process of performing the aforementioned dynamic driving task, At least one processor (51b, 200, 201, 202, 20n, 220, 230), A device comprising at least one recording medium (55a), The aforementioned at least one processor is The respective sensor data is acquired from each of the sensor groups. Access the rule database to obtain the rule set, Access the scenario database to obtain the scenario structure, Based on the sensor data, the rule set, and the scenario structure, calculate a degree of violation of the rule, which is a plurality of degrees of violation calculated using sensor data from different sensor groups that are the sources of output, Integrate the plurality of degrees of violation to calculate an integrated degree of violation, Execute the dynamic driving task based on the integrated degree of violation, A driving system that records the plurality of degrees of violation before integration in at least one storage medium.

[0260] According to this technical idea, the traceability of the driving system can be improved.

[0261] <Technical Idea 27> A driving method for a vehicle (1) that executes a dynamic driving task, comprising: Obtain sensor data respectively provided from a plurality of sensor groups classified and configured by a plurality of sensors (40) provided in the vehicle, Obtain a rule set from a rule database (58) that implements a priority structure for a series of rules arranged based on relative importance, Obtain a scenario structure including a plurality of scenarios indicating the vehicle, the external environment, and their interactions in the process of executing the dynamic driving task from a scenario database (59), Based on the sensor data, the rule set, and the scenario structure, calculate a degree of violation of the rule, which is a plurality of degrees of violation calculated using sensor data from different sensor groups that are the sources of output, Integrate the plurality of degrees of violation to calculate an integrated degree of violation, Execute the dynamic driving task based on the integrated degree of violation, A driving method for a vehicle that records the plurality of degrees of violation before integration in at least one storage medium (55a).

[0262] This technological concept can improve traceability for the performance of dynamic driving tasks of vehicles.

[0263] <Technical Thought 28> A method for generating individual evaluation results regarding strategic guidelines, Sensor data is acquired from multiple sensor groups installed on the vehicle. Based on the sensor data acquired from each sensor group, an individual evaluation unit provided for each sensor group generates a matrix of violation metrics, which is an individual evaluation result for a rule set containing multiple rules. Based on the violation metric matrix calculated by each individual evaluation unit, a matrix of the integrated violation metric, which is the integrated evaluation result, is generated. A method for generating individual evaluation results for strategic guidelines, which involves recording a matrix of the aforementioned individual violation metrics on a recording medium.

[0264] This technical approach can improve the traceability of evaluation results related to strategic guidelines.

Claims

1. A processing system that performs processing related to the operation of a vehicle (1), A plurality of individual evaluation units that evaluate each rule in a rule set containing multiple rules relating to traffic laws using sensor data and output individual evaluation results for the rule set, wherein the plurality of individual evaluation units (211, 212, 21n, 311, 312, 31n) have at least a portion of the sensor data output sources that differ from each other, An integrated evaluation unit (221, 321) integrates each of the individual evaluation results and outputs an integrated evaluation result, A processing system comprising: a driving planning unit (22) that plans driving behavior by deriving driving behavior that minimizes violations of the rules based on the evaluation results after integration.

2. Each of the individual evaluation units outputs a matrix of violation metrics indicating the degree of violation for each rule as an individual evaluation result. The processing system according to claim 1, wherein the integrated evaluation unit integrates the violation metrics for the same rule in the matrix of multiple violation metrics output from each of the individual evaluation units using an integration function, generates a matrix of violation metrics after integration, and outputs it as the evaluation result after integration.

3. The processing system according to claim 2, which identifies the source of the sensor data output in which a failure or false detection has occurred, based on the fact that the absolute value of the difference between the violation metric calculated by each of the individual evaluation units and the integrated violation metric calculated by the integrated evaluation unit is greater than or equal to a detection threshold.

4. The aforementioned rule set is provided in common among the multiple individual evaluation units, The processing system according to any one of claims 1 to 3, wherein each of the individual evaluation units evaluates a plurality of rules common to each other based on the rule set.

5. The processing system according to claim 4, wherein each of the individual evaluation units performs an evaluation using a different algorithm corresponding to the difference in the source of the sensor data output for the same rule.

6. The processing system according to claim 4, wherein each of the individual evaluation units performs an evaluation for the same rule using the same algorithm and different parameters corresponding to the differences in the source of the sensor data output.

7. The aforementioned rule set is provided in common among the multiple individual evaluation units, The processing system according to any one of claims 1 to 3, wherein each individual evaluation unit evaluates the rules that differ from each other in part from the plurality of rules included in the rule set, based on the rule set, by excluding some of the rules included in the rule set from evaluation in accordance with the differences in the source of the sensor data output.

8. The processing system according to claim 1, which generates associated data of the individual evaluation results and the integrated evaluation results and stores it in a storage medium (55a).

9. The processing system according to claim 1, which generates associated data of the individual evaluation results and the integrated evaluation results, and transmits it to an external system (96) located outside the vehicle via a communication system (43) mounted on the vehicle.

10. The processing system according to claim 1, wherein the multiple individual evaluation units are implemented by a single common processor (200, 300).

11. The processing system according to claim 1, wherein each of the multiple individual evaluation units is implemented by a separate processor (201, 202, 20n) that corresponds to it individually.

12. The aforementioned driving planning unit derives a provisional driving action, The individual evaluation unit outputs individual evaluation results regarding the rule set for the hypothetical driving behavior, The integrated evaluation unit integrates the individual evaluation results for the provisional driving actions and outputs the integrated evaluation result. The processing system according to claim 1, wherein the operation planning unit determines the final operation by referring to the integrated evaluation results of the provisional operation.

13. A processing system that performs processing related to the operation of a vehicle (1), A plurality of individual evaluation units that evaluate each rule in a rule set containing multiple rules relating to traffic laws using sensor data and output individual evaluation results for the rule set, wherein the plurality of individual evaluation units (211, 212, 21n) have at least a portion of the sensor data output sources that differ from each other, Each of the individual evaluation units is provided to correspond to a plurality of individual driving planning units (251, 252, 25n) that plan individual driving actions by deriving driving actions that minimize violations of the rules based on the individual evaluation results output by the paired individual evaluation units, A processing system comprising: an integrated driving plan unit (261) that integrates each of the individual driving actions and plans the integrated driving action.

Images