Processing system

Abstract

This processing system (50) executes processing related to driving a vehicle (1). The processing system (50) comprises: a plurality of guidelines (211, 212, 21n) by which individual evaluation results pertaining to strategic guidelines are outputted on the basis of sensor data, at least some of the sensor data output sources differing from each other; guideline integration (221) by which the individual evaluation results are integrated and the integrated evaluation result is outputted; and an operation planning unit (22) that plans operation actions on the basis of the evaluation results after the integration.

Description

Processing System CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2022-187493 filed in Japan on November 24, 2022, and the contents of the original application are incorporated by reference in their entirety.

[0002] TECHNICAL FIELD This disclosure relates to the operation of vehicles.

[0003] In Patent Document 1, in order to plan driving behavior, a guideline processor acquires sensor data from a plurality of sensors and performs evaluation regarding strategic guidelines based on the sensor data.

[0004] US Patent Application Publication No. 2021 / 0356962

[0005] However, in a configuration in which sensor data from all sensors are evaluated collectively, as in Patent Document 1, it is difficult to identify the causal relationship between the evaluation results of the derived driving behavior or strategic guideline and the multiple sensors used therefor. In particular, when the evaluation of the strategic guideline is performed using artificial intelligence or the like, the difficulty of verifying it becomes significantly higher. Thus, there is room for improvement in the traceability of planned driving behavior.

[0006] One of the objectives of the disclosure of this specification is to provide a processing system that improves traceability of planned driving behavior.

[0007] The aspect disclosed herein is a processing system that performs processing related to vehicle driving, and includes: a plurality of individual evaluation units that output individual evaluation results related to strategic guidelines based on sensor data, where the individual evaluation units have at least some of the sensor data output sources that are different from each other; an integrated evaluation unit that integrates the individual evaluation results and outputs the integrated evaluation result; and a driving planning unit that plans driving behavior based on the integrated evaluation result.

[0008] Another disclosed aspect is a processing system that executes processing related to vehicle driving, comprising: a plurality of individual evaluation units that output individual evaluation results related to strategic guidelines based on sensor data, where the plurality of individual evaluation units have at least some of the sensor data output sources that are different from each other; a plurality of individual driving planning units that are provided individually corresponding to each individual evaluation unit and plan individual driving actions based on the individual evaluation results output by the paired individual evaluation unit; and an integrated driving planning unit that integrates each individual driving action and plans the integrated driving action.

[0009] According to these aspects, evaluations of the strategic guidelines are performed individually, with at least some of the sensor data output sources being different from each other. This process allows the final driving behavior and the causal relationship with the sensors to be broken down into individual evaluation results and analyzed. This improves the traceability of the planned driving behavior.

[0010] Note that the symbols in parentheses included in the claims etc. are intended to exemplify the correspondence with the parts of the embodiments described below, and are not intended to limit the technical scope.

[0011] Schematic diagram of a driving system. Diagram showing a schematic diagram of the hardware configuration of the driving system. Diagram of the hardware configuration of the driving system. Diagram of the software configuration of the driving system. Diagram showing an example of rule relationships. Diagram showing an example of rule relationships. Diagram showing an example of rule relationships. Diagram showing an example of a rule set implementation. Flowchart showing an example of a processing method. Diagram showing an example of a rule set implementation. Diagram showing an example of a rule set implementation. Flowchart showing an example of a processing method. Diagram showing an example of a rule set implementation.

[0012] Hereinafter, several embodiments will be described with reference to the drawings. Note that corresponding components in each embodiment are given the same reference numerals, and redundant description may be omitted. When only a portion of the configuration is described in each embodiment, the configuration of another embodiment described previously can be applied to the remaining portion of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of several embodiments can also be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.

[0013] In the following embodiments, the contents of “Safety First for Automated Driving,” Tech. Rep., 2019, by Aptiv, Audi, Baidu, BMW, Continental, Daimler, FCA, hereby, Infineon, Intel, and Volkswagen, are incorporated by reference in their entirety.

[0014] (First embodiment) A driving system 2 of the first embodiment realizes functions related to driving a moving object. Part or all of the driving system 2 is mounted on the moving object. The moving object that is the target of processing by the driving system 2 is a vehicle 1. This vehicle 1 may be referred to as a subject vehicle, a host vehicle, or the like. The vehicle 1 may be configured to be able to communicate with other vehicles directly or indirectly via a communication infrastructure. The other vehicles may be referred to as target vehicles.

[0015] The vehicle 1 may be a road user capable of performing manual driving, such as a car or truck. The vehicle 1 may also be capable of performing automated driving. Driving is classified into levels according to the extent to which the driver performs all dynamic driving tasks (DDTs). Autonomous driving levels are specified, for example, in SAE J3016. At levels 0 to 2, the driver performs some or all of the DDTs. Levels 0 to 2 may be classified as so-called manual driving. Level 0 indicates that driving is not automated. Level 1 indicates that the driver is assisted by the driving system 2. Level 2 indicates that driving is partially automated.

[0016] At levels 3 and above, the driving system 2 performs all of the DDT while engaged. Levels 3 to 5 may be classified as so-called automated driving. Systems capable of driving at levels 3 and above may be called automated driving systems. Vehicles equipped with automated driving systems or vehicles capable of driving at levels 3 and above may be called automated vehicles (AVs). Level 3 indicates that driving is conditionally automated. Level 4 indicates that driving is highly automated. Level 5 indicates that driving is fully automated.

[0017] Furthermore, a driving system 2 that is unable to perform driving at level 3 or above but can perform driving at least at levels 1 and 2 may be referred to as a driving assistance system. Hereinafter, when there is little need to specify the achievable level of autonomous driving, the autonomous driving system or driving system may be referred to simply as the driving system 2.

[0018] <Overview of Driving System> The architecture of the driving system 2 is selected so as to enable an efficient SOTIF (safety of the intended functionality) process. For example, the architecture of the driving system 2 may be configured based on the sense-plan-act model. The sense-plan-act model comprises a sense element, a plan element, and an act element as main system elements. The sense element, plan element, and act element interact with each other. Here, sense may be replaced with perception, plan with judgment, and act with control, respectively.

[0019] As shown in Fig. 1, at the functional level (in other words, from a functional perspective), such a driving system 2 implements a recognition function, a judgment function, and a control function. As shown in Fig. 2, at the technical level (in other words, from a technical perspective), at least a plurality of sensors 40 corresponding to the recognition function, at least one processing system 50 corresponding to the judgment function, and a plurality of motion actuators 60 corresponding to the control function are implemented.

[0020] In detail, a recognition unit 10 may be constructed in the driving system 2 as a functional block that realizes a recognition function, mainly consisting of a plurality of sensors 40, a processing system that processes information detected by the plurality of sensors 40, and a processing system that generates an environmental model based on information from the plurality of sensors 40. A judgment unit 20 may be constructed in the driving system 2 as a functional block that realizes a judgment function, mainly consisting of a processing system 50. A control unit 30 may be constructed in the driving system 2 as a functional block that realizes a control function, mainly consisting of a plurality of movement actuators 60 and at least one processing system that outputs operation signals for the plurality of movement actuators 60.

[0021] Here, the recognition unit 10 may be realized in the form of a recognition system 10a as a subsystem provided so as to be distinguishable from the determination unit 20 and the control unit 30. The determination unit 20 may be realized in the form of a determination system 20a as a subsystem provided so as to be distinguishable from the recognition unit 10 and the control unit 30. The control unit 30 may be realized in the form of a control system 30a as a subsystem provided so as to be distinguishable from the recognition unit 10 and the determination unit 20. The recognition system 10a, the determination system 20a, and the control system 30a may constitute components independent of each other.

[0022] Furthermore, a plurality of HMI (Human Machine Interface) devices 70 may be mounted on the vehicle 1. The HMI devices 70 realize human-machine interaction, which is interaction between the occupants (including the driver) of the vehicle 1 and the driving system 2. A portion of the plurality of HMI devices 70 that realizes an operation input function by the occupants may be part of the recognition unit 10. A portion of the plurality of HMI devices 70 that realizes an information presentation function may be part of the control unit 30. On the other hand, the function realized by the HMI device 70 may be positioned as a function independent of the recognition function, the judgment function, and the control function.

[0023] The recognition unit 10 is responsible for recognition functions, including localization (e.g., location estimation) of road users such as the vehicle 1 and other vehicles. The recognition unit 10 detects the external environment, internal environment, vehicle state, and the state of the driving system 2 of the vehicle 1. The recognition unit 10 fuses the detected information to generate an environmental model. The environmental model may also be referred to as a world model. The determination unit 20 applies the objective and driving policy to the environmental model generated by the recognition unit 10 to derive a control action. The control unit 30 executes the control action derived by the determination unit 20.

[0024] <Physical Architecture Overview> An example of the physical architecture of the driving system 2 will be described using Figure 2. The driving system 2 includes a plurality of sensors 40, a plurality of motion actuators 60, a plurality of HMI devices 70, and at least one processing system 50. These components can communicate with each other via one or both of wireless and wired connections. These components may also be able to communicate with each other through an in-vehicle network such as CAN (registered trademark). These components will be described in further detail using Figure 3.

[0025] The multiple sensors 40 include one or more external environment sensors 41. The multiple sensors 40 may include at least one of one or more internal environment sensors 42, one or more communication systems 43, and a map database (DB) 44. When the sensor 40 is interpreted in a narrow sense to refer to the external environment sensor 41, the internal environment sensor 42, the communication system 43, and the map database 44 may be positioned as components separate from the sensor 40 that corresponds to the technology level of the recognition function.

[0026] The external environment sensor 41 may detect targets present in the external environment of the vehicle 1. Target detection type external environment sensors 41 include, for example, a camera, a LiDAR (Light Detection and Ranging / Laser imaging Detection and Ranging) laser radar, a millimeter wave radar, an ultrasonic sonar, etc. Typically, a combination of multiple types of external environment sensors 41 may be implemented to monitor the front, sides, and rear directions of the vehicle 1.

[0027] As an example of the installation of the external environment sensor 41, multiple cameras (e.g., 11 cameras) configured to monitor each of the directions in front, front-side, side, rear-side, and rear of the vehicle 1 may be installed on the vehicle 1.

[0028] As another example of installation, vehicle 1 may be equipped with multiple cameras (e.g., four cameras) configured to monitor the front, sides, and rear of vehicle 1, respectively, multiple millimeter-wave radars (e.g., five millimeter-wave radars) configured to monitor the front, front-side, sides, and rear of vehicle 1, respectively, and a LiDAR configured to monitor the front of vehicle 1.

[0029] Furthermore, the external environment sensor 41 may detect atmospheric conditions and weather conditions in the environment outside the vehicle 1. The condition detection type external environment sensor 41 is, for example, an outside air temperature sensor, a temperature sensor, a raindrop sensor, or the like.

[0030] The internal environment sensor 42 may detect a specific physical quantity related to vehicle motion (hereinafter referred to as a motion physical quantity) in the internal environment of the vehicle 1. The motion physical quantity detection type internal environment sensor 42 is, for example, a speed sensor, an acceleration sensor, a gyro sensor, etc. The internal environment sensor 42 may detect the state of an occupant in the internal environment of the vehicle 1. The occupant detection type internal environment sensor 42 is, for example, an actuator sensor, a driver monitoring sensor and its system, a biological sensor, a seating sensor, an in-vehicle equipment sensor, etc. Here, the actuator sensor in particular is, for example, an accelerator sensor, a brake sensor, a steering sensor, etc., which detect the state of an occupant's operation of a motion actuator 60 related to the motion control of the vehicle 1.

[0031] The communication system 43 obtains communication data usable in the driving system 2 via wireless communication. The communication system 43 may receive positioning signals from artificial satellites of a global navigation satellite system (GNSS) that exist in the external environment of the vehicle 1. The positioning type communication device in the communication system 43 is, for example, a GNSS receiver.

[0032] The communication system 43 may transmit and receive communication signals to and from an external system 96 present in the external environment of the vehicle 1. Examples of V2X-type communication devices in the communication system 43 include dedicated short range communications (DSRC) communication devices, cellular V2X (C-V2X) communication devices, etc. Examples of communication with a V2X system present in the external environment of the vehicle 1 include communication with a communication system of another vehicle (V2V), communication with infrastructure equipment such as a communication device installed in a traffic light or a roadside device (V2I), communication with a mobile terminal of a pedestrian (V2P), and communication with a network such as a cloud server (V2N). The architecture of V2X communication, including V2I communication, may be an architecture specified in ISO 21217, ETSI TS 102 940-943, IEEE 1609, etc.

[0033] Furthermore, the communication system 43 may transmit and receive communication signals to and from a mobile terminal 91, such as a smartphone, present inside the vehicle 1. Examples of terminal communication type communication devices in the communication system 43 include Bluetooth (registered trademark) devices, Wi-Fi (registered trademark) devices, and infrared communication devices.

[0034] The map DB 44 is a database that stores map data available to the driving system 2. The map DB 44 includes at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The map DB 44 may include a database of a navigation unit that navigates the vehicle 1 along a route to a destination. The map DB 44 may include a database of probe data (PD) maps generated using probe data (PD) collected from each vehicle. The map DB 44 may include a database of high-precision maps with a high level of accuracy that are primarily used in automated driving systems. The map DB 44 may also include a database of parking lot maps that include detailed parking lot information, such as parking space information, that is used in automated parking or parking assistance applications.

[0035] The map DB 44 suitable for the driving system 2 acquires and stores the latest map data, for example, by communicating with a map server via a V2X communication system 43. The map data is data representing the external environment of the vehicle 1, and is converted into two-dimensional or three-dimensional data. The map data may include road data representing at least one of the position coordinates, shape, road surface condition, and standard running route of a road structure. The map data may also include marking data representing at least one of the position coordinates and shape of road signs, road markings, and lane markings attached to a road. The marking data included in the map data may represent landmarks such as traffic signs, arrow markings, lane markings, stop lines, directional signs, landmark beacons, business signs, and changes in road line patterns. The map data may also include structure data representing at least one of the position coordinates and shape of buildings and traffic lights facing the road. The marking data included in the map data may represent landmarks such as street lights, road edges, reflectors, and poles.

[0036] The motion actuator 60 can control vehicle motion based on an input control signal. The drive-type motion actuator 60 is, for example, a power train including at least one of an internal combustion engine, a drive motor, etc. The braking-type motion actuator 60 is, for example, a brake actuator. The steering-type motion actuator 60 is, for example, a steering.

[0037] The HMI device 70 may be an operation input device that can input operations by the driver in order to transmit the will or intention of the occupants, including the driver of the vehicle 1, to the driving system 2. Examples of the operation input type HMI device 70 include an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a turn signal lever, a mechanical switch, and a touch panel of a navigation unit. Of these, the accelerator pedal controls the power train as the motion actuator 60. The brake pedal controls a brake actuator as the motion actuator 60. The steering wheel controls a steering actuator as the motion actuator 60.

[0038] The HMI device 70 may be an information presentation device that presents information such as visual information, auditory information, and cutaneous information to occupants including the driver of the vehicle 1. Examples of the visual information presentation type HMI device 70 include a combination meter, a graphic meter, a navigation unit, a center information display (CID), a head-up display (HUD), and an illumination unit. Examples of the auditory information presentation type HMI device 70 include a speaker and a buzzer. Examples of the cutaneous information presentation type HMI device 70 include a steering wheel vibration unit, a driver's seat vibration unit, a steering wheel reaction force unit, an accelerator pedal reaction force unit, a brake pedal reaction force unit, an air conditioning unit, and the like.

[0039] The HMI device 70 may also realize an HMI function linked to a mobile terminal such as a smartphone by communicating with the terminal via the communication system 43. For example, the HMI device 70 may present information acquired from the smartphone to passengers including the driver. Also, for example, operation input to a smartphone may be used as an alternative means for operation input to the HMI device 70.

[0040] At least one processing system 50 is provided. For example, the processing system 50 may be an integrated processing system that integrally executes processing related to the recognition function, processing related to the judgment function, and processing related to the control function. In this case, the integrated processing system 50 may further execute processing related to the HMI device 70, or a processing system dedicated to the HMI may be provided separately. For example, the processing system dedicated to the HMI may be an integrated cockpit system that integrally executes processing related to each HMI device.

[0041] For example, the processing system 50 may be configured to have at least one processing unit corresponding to processing related to the recognition function, at least one processing unit corresponding to processing related to the judgment function, and at least one processing unit corresponding to processing related to the control function.

[0042] The processing system 50 has a communication interface to the outside and is connected to at least one type of element related to processing by the processing system 50, such as the sensor 40, the motion actuator 60, and the HMI device 70, via at least one type of interface, such as a LAN (Local Area Network), a wire harness, an internal bus, and a wireless communication circuit.

[0043] The processing system 50 includes at least one dedicated computer 51. The processing system 50 may combine multiple dedicated computers 51 to realize functions such as recognition functions, judgment functions, and control functions.

[0044] For example, the dedicated computer 51 constituting the processing system 50 may be an integrated ECU that integrates the driving functions of the host vehicle 1. The dedicated computer 51 constituting the processing system 50 may be a determination ECU that determines DDT. The dedicated computer 51 constituting the processing system 50 may be a monitoring ECU that monitors the driving of the vehicle. The dedicated computer 51 constituting the processing system 50 may be an evaluation ECU that evaluates the driving of the vehicle. The dedicated computer 51 constituting the processing system 50 may be a navigation ECU that navigates the driving route of the host vehicle 1.

[0045] Furthermore, the dedicated computer 51 constituting the processing system 50 may be a locator ECU that estimates the position of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an image processing ECU that processes image data detected by the external environment sensor 41. The dedicated computer 51 constituting the processing system 50 may be an actuator ECU that controls the motion actuator 60 of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an HCU (HMI Control Unit) that comprehensively controls the HMI device 70. The dedicated computer 51 constituting the processing system 50 may be at least one external computer that constitutes an external center or mobile terminal that can communicate via the communication system 43, for example.

[0046] The dedicated computer 51 constituting the processing system 50 has at least one memory 51a and one processor 51b. The memory 51a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores programs and data that can be read by the processor 51b. Furthermore, the memory 51a may be a rewritable volatile storage medium, such as a random access memory (RAM). The processor 51b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0047] The dedicated computer 51 constituting the processing system 50 may be a SoC (System on a Chip) that integrates memory, a processor, and an interface on a single chip, or may have a SoC as a component of the dedicated computer 51.

[0048] Furthermore, the processing system 50 may include at least one database for executing the dynamic driving task. The database may include at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, and an interface for accessing the storage medium. The database may be a scenario database (hereinafter referred to as a scenario DB) 59, which will be described in detail below. The database may be a rule database (hereinafter referred to as a rule DB) 58, which will be described in detail below. At least one of the scenario DB 59 and the rule DB 58 may not be provided in the processing system 50, but may be provided independently of the other systems 10a, 20a, and 30a in the driving system 2. At least one of the scenario DB 59 and the rule DB 58 may be provided in an external system 96 and configured to be accessible from the processing system 50 via the communication system 43.

[0049] The processing system 50 may also include at least one recording device 55 that records at least one of the recognition information, judgment information, and control information of the driving system 2. The recording device 55 may include at least one memory 55a and an interface 55b for writing data to the memory 55a. The memory 55a may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium.

[0050] At least one of the memories 55a may be mounted on the board in a form that is not easily detachable or replaceable, such as an embedded multi-media card (eMMC) using a flash memory, etc. At least one of the memories 55a may be detachable and replaceable from the recording device 55, such as an SD card, etc.

[0051] The recording device 55 may have a function of selecting information to be recorded from the recognition information, judgment information, and control information. In this case, the recording device 55 may have a dedicated computer 55c. A processor provided in the recording device 55 may temporarily store information in RAM or the like. The processor may select information to be recorded from the temporarily stored information and save the selected information in the memory 51a.

[0052] The recording device 55 may access the memory 55a and perform recording in accordance with a data write command from the recognition system 10a, the determination system 20a, or the control system 30a. The recording device 55 may determine information flowing through the in-vehicle network, and access the memory 55a and perform recording based on the judgment of a processor provided in the recording device 55.

[0053] The recording device 55 may not be provided in the processing system 50, but may be provided independently of the other systems 10a, 20a, and 30a in the operation system 2. The recording device 55 may be provided in an external system 96 and configured to be accessible from the processing system 50 via the communication system 43.

[0054] <Outline of Logical Architecture> Next, an example of a logical architecture in the driving system 2 will be described with reference to Fig. 4. The recognition unit 10 may include an environment recognition unit 11, a self-position recognition unit 12, and an internal recognition unit 13 as sub-blocks that further classify recognition functions.

[0055] The environment recognition unit 11 individually processes information (sometimes referred to as sensor data) related to the external environment acquired from each sensor 40, and realizes a function of recognizing the external environment including targets, other road users, etc. The environment recognition unit 11 individually processes detection data detected by each external environment sensor 41. The detection data may be detection data provided by, for example, millimeter-wave radar, sonar, LiDAR, etc. The environment recognition unit 11 may generate relative position data including the direction, size, and distance of an object relative to the vehicle 1 from the raw data detected by the external environment sensors 41.

[0056] The detection data may be image data provided by, for example, a camera, LiDAR, or the like. The environment recognition unit 11 processes the image data and extracts objects reflected within the angle of view of the image. The object extraction may include estimating the direction, size, and distance of the object relative to the vehicle 1. The object extraction may also include classifying the object using, for example, semantic segmentation.

[0057] Furthermore, the environment recognition unit 11 processes information acquired through the V2X function of the communication system 43. The environment recognition unit 11 processes information acquired from the map DB 44.

[0058] The environment recognition unit 11 may be further divided into a plurality of sensor recognition units each optimized for one sensor group. When a sensor recognition unit is associated with recognizing information from one sensor group, the sensor recognition unit may fuse information from the one sensor group.

[0059] The self-location recognition unit 12 performs localization of the vehicle 1. The self-location recognition unit 12 acquires global position data of the vehicle 1 from the communication system 43 (e.g., a GNSS receiver). In addition, the self-location recognition unit 12 may acquire position information of targets extracted by the environment recognition unit 11. The self-location recognition unit 12 also acquires map information from the map DB 44. The self-location recognition unit 12 integrates this information to estimate the position of the vehicle 1 on the map.

[0060] The internal recognition unit 13 processes the detection data detected by each internal environment sensor 42 and realizes a function of recognizing the vehicle state. The vehicle state may include the state of the physical quantities of motion of the vehicle 1 detected by a speed sensor, an acceleration sensor, a gyro sensor, etc. The vehicle state may also include at least one of the states of the occupants including the driver, the operation state of the driver with respect to the motion actuator 60, and the switch state of the HMI device 70.

[0061] The determination unit 20 may include a prediction unit 21, an operation planning unit 22, and a mode management unit 23 as sub-blocks that further classify the determination function.

[0062] The prediction unit 21 acquires information on the external environment recognized by the environment recognition unit 11 and the self-position recognition unit 12, the vehicle state recognized by the internal recognition unit 13, etc. The prediction unit 21 may interpret the environment based on the acquired information and estimate the current situation of the vehicle 1. The situation here may be an operational situation or may include the operational situation.

[0063] The prediction unit 21 may interpret the environment and predict the behavior of objects such as other road users. The objects may be safety-relevant objects. The behavior prediction may include at least one of the following: prediction of the object's speed, prediction of the object's acceleration, and prediction of the object's trajectory. The behavior prediction may be performed based on reasonably foreseeable assumptions.

[0064] The prediction unit 21 may interpret the environment and make a determination regarding a scenario in which the vehicle 1 is currently placed. The determination regarding the scenario may be to select at least one scenario in which the vehicle 1 is currently placed from a catalog of scenarios constructed in the scenario DB 59. The prediction unit 21 may interpret the environment or predict potential dangers based on the selected scenario.

[0065] Furthermore, the prediction unit 21 may estimate the driver's intention based on the predicted behavior, the predicted potential danger, and the acquired vehicle state.

[0066] The driving planning unit 22 plans autonomous driving of the vehicle 1 based on at least one of the estimated information of the vehicle 1's position on a map by the self-position recognition unit 12, the prediction information and driver intention estimation information by the prediction unit 21, and the function constraint information by the mode management unit 23.

[0067] The driving planner 22 realizes a route planning function, a behavior planning function, and a trajectory planning function. The route planning function is a function of planning at least one of a route to a destination and a mid-range lane plan based on estimated information about the position of the vehicle 1 on a map. The route planning function may further include a function of determining at least one of a lane change request and a deceleration request based on the mid-range lane plan. Here, the route planning function may be a mission / route planning function in a strategic function, and may be a function of outputting a mission plan and a route plan.

[0068] The behavior planning function is a function that plans the behavior of the vehicle 1 based on at least one of the route to the destination planned by the route planning function, the mid-distance lane plan, the lane change request and the deceleration request, the prediction information and the driver's intention estimation information by the prediction unit 21, and the function constraint information by the mode management unit 23. The behavior planning function may include a function that generates conditions related to state transitions of the vehicle 1. The conditions related to state transitions of the vehicle 1 may correspond to triggering conditions. The behavior planning function may include a function that determines state transitions of an application that realizes DDT and further state transitions of driving actions based on the conditions. The behavior planning function may include a function that determines longitudinal constraints on the path of the vehicle 1 and lateral constraints on the path of the vehicle 1 based on the state transition information. The behavior planning function may be a tactical behavior plan in the DDT function and may output tactical behavior.

[0069] The trajectory planning function is a function that plans a driving trajectory of the vehicle 1 based on the judgment information by the prediction unit 21, longitudinal constraints on the path of the vehicle 1, and lateral constraints on the path of the vehicle 1. The trajectory planning function may include a function that generates a path plan. The path plan may include a speed plan, or the speed plan may be generated as a plan independent of the path plan. The trajectory planning function may include a function that generates multiple path plans and selects an optimal path plan from the multiple path plans, or a function that switches between path plans. The trajectory planning function may further include a function that generates backup data of the generated path plan. The trajectory planning function may be a trajectory planning function in the DDT function, and may output a trajectory plan.

[0070] The mode management unit 23 monitors the driving system 2 and sets constraints on driving-related functions. The mode management unit 23 may manage the autonomous driving mode, for example, the autonomous driving level. The management of the autonomous driving level may include management of switching between manual driving and autonomous driving, i.e., management of transfer of authority between the driver and the driving system 2, in other words, takeover. The mode management unit 23 may monitor the status of subsystems related to the driving system 2 and determine system malfunctions (e.g., errors, unstable operation states, system failures, and malfunctions). The mode management unit 23 may determine a mode based on the driver's intention based on driver's intention estimation information generated by the internal recognition unit 13. The mode management unit 23 may set constraints on driving-related functions based on at least one of the system malfunction determination result, the mode determination result, the vehicle state determined by the internal recognition unit 13, the sensor abnormality (or sensor failure) signal output from the sensor 40, application state transition information and trajectory planning determined by the driving planner 22, etc.

[0071] Furthermore, the mode management unit 23 may have a comprehensive function of determining, in addition to constraints on driving functions, longitudinal constraints on the path of the vehicle 1 and lateral constraints on the path of the vehicle 1. In this case, the operation planning unit 22 plans behavior and trajectories in accordance with the constraints determined by the mode management unit 23.

[0072] The control unit 30 may include a motion control unit 31 and an HMI output unit 71 as sub-blocks that further classify the control functions. The motion control unit 31 controls the motion of the vehicle 1 based on the trajectory plan (e.g., a path plan and a speed plan) acquired from the driving plan unit 22. Specifically, the motion control unit 31 generates accelerator request information, shift request information, brake request information, and steering request information according to the trajectory plan, and outputs them to the motion actuator 60.

[0073] Here, the motion control unit 31 can directly obtain the vehicle state recognized by the recognition unit 10 (particularly the internal recognition unit 13), such as at least one of the current speed, acceleration, and yaw rate of the vehicle 1, from the recognition unit 10 and reflect this in the motion control of the vehicle 1.

[0074] The HMI output unit 71 outputs information related to the HMI based on at least one of prediction information and driver intention estimation information from the prediction unit 21, application state transition information and trajectory planning from the driving planner 22, and function constraint information from the mode manager 23. The HMI output unit 71 may manage vehicle interactions. The HMI output unit 71 may generate a notification request based on the management state of the vehicle interactions and control an information presentation function of the HMI device 70. Furthermore, the HMI output unit 71 may generate control requests for wipers, a sensor washing device, headlights, and an air conditioner based on the management state of the vehicle interactions and control these devices.

[0075] <Strategic Guidelines> The determination unit 20 or the driving planner 22 can realize their functions in accordance with strategic guidelines based on driving policies. A set of strategic guidelines is obtained by analyzing basic principles. The set of strategic guidelines can be implemented in the driving system 2 as one or more databases. The set of strategic guidelines may be a rule set. The rule set may be, for example, rulebooks. The rules included in the rule set may be defined to include traffic laws, safety rules, ethical rules, and local cultural rules.

[0076] The strategic guideline is expressed by the following first to fourth items. The first item is one or more conditions. The second item is one or more actions associated with one or more conditions. The third item is a strategic factor associated with the condition and the appropriate action. The strategic factors are, for example, distance, speed, acceleration, deceleration, direction, time, temperature, season, chemical concentration, area, height, and weight. The fourth item is a deviation metric that quantitatively evaluates deviation from appropriate action during machine operation. The deviation metric may be or may include a violation metric.

[0077] The ground rules may include laws, regulations, etc., and may also include combinations thereof. The ground rules may include preferences that are not influenced by laws, regulations, etc. The ground rules may include exercise behavior based on past experience. The ground rules may include characterization of the exercise environment. The ground rules may include ethical concerns.

[0078] The basic principles may also include human feedback regarding the automated driving system. The basic principles may include feedback of at least one of comfort and predictability from vehicle occupants or other road users. Predictability may indicate a reasonably foreseeable range. Predictability feedback may be feedback based on a reasonably foreseeable range. The basic principles may also include rules.

[0079] The strategic guidelines may include logical relationships of clear, systematic, and comprehensive fundamental principles or rules for corresponding movement behaviors. The fundamental principles or rules may have quantitative measures.

[0080] <Scenario> The driving system 2 may adopt a scenario-based approach to perform or evaluate a dynamic driving task. The processes required to perform a dynamic driving task in automated driving are classified into disturbances in the recognition element, disturbances in the judgment element, and disturbances in the control element, which are based on different physical principles. The root causes that affect the processing results in each element are structured as a scenario structure.

[0081] Disturbances in the recognition element are called perception disturbances. Perception disturbances are disturbances that indicate a state in which the recognition unit 10 is unable to correctly recognize danger due to internal or external factors of the sensor 40 and the host vehicle 1. Internal factors include instability related to variations in the installation or manufacturing of sensors such as the external environment sensor 41, tilting of the vehicle due to uneven loads that change the direction of the sensor, and shielding of the sensor due to parts installed on the outside of the vehicle. External factors include fogging or dirt on the sensor. The physical principles of perception disturbances are based on the sensor mechanism of each sensor.

[0082] The disturbance in the judgment element is a traffic disturbance. The traffic disturbance is a disturbance that indicates a potentially dangerous traffic situation that occurs as a result of a combination of the road geometry, the behavior of the vehicle 1, and the positions and behaviors of surrounding vehicles. The physical principles of traffic disturbances are based on a geometric perspective and the actions of road users.

[0083] Disturbances in the control elements are vehicle disturbances. Vehicle disturbances may also be referred to as control disturbances. Vehicle disturbances are disturbances that indicate a situation in which the vehicle 1 may be unable to control its own dynamics due to internal or external factors. Internal factors include, for example, the total weight and weight balance of the vehicle 1. External factors include, for example, road surface irregularities, slopes, wind, etc. The physical principles of vehicle disturbances are based on the mechanical effects input to the tires and the vehicle body.

[0084] In order to deal with a collision of the vehicle 1 with another road user or a structure as a risk in a dynamic driving task of automated driving, a traffic disturbance scenario system in which traffic disturbance scenarios are systematized is used as one of the scenario structures. For the traffic disturbance scenario system, a reasonably foreseeable range or a reasonably foreseeable boundary can be defined, and an avoidable range or a avoidable boundary can be defined.

[0085] The avoidable range or boundary can be defined, for example, by defining and modeling the performance of a competent and careful human driver, which can be defined in three elements: perception, judgment, and control.

[0086] For example, the scenario structure may be stored in a scenario DB 59. The scenario DB 59 may store a plurality of scenarios including at least one of a functional scenario, a logical scenario, and a concrete scenario. A functional scenario defines a top-level qualitative scenario structure. A logical scenario is a scenario in which a quantitative parameter range is assigned to a structured functional scenario. A concrete scenario defines a boundary of a safety determination that distinguishes between a safe state and an unsafe state.

[0087] An unsafe state is, for example, a hazardous situation. A range corresponding to a safe state may be referred to as a safe range, and a range corresponding to an unsafe state may be referred to as an unsafe range. Furthermore, a condition that contributes to unsafe behavior of the vehicle 1 in a scenario or an inability to prevent, detect, and mitigate reasonably foreseeable misuse may be a trigger condition.

[0088] Scenarios can be classified as known or unknown, and as dangerous or non-dangerous, i.e., known dangerous scenarios, known non-dangerous scenarios, unknown dangerous scenarios, and unknown non-dangerous scenarios.

[0089] Rule Sets A rule set is a data structure that implements a priority structure for a set of rules arranged based on relative importance. For any particular rule in the priority structure, a rule with a higher priority is more important than a rule with a lower priority. The priority structure may be one of a hierarchical structure, a non-hierarchical structure, and a hybrid priority structure. A hierarchical structure may, for example, be a structure indicating a pre-order for various degrees of rule violation. A non-hierarchical structure may, for example, be a weighting system for rules. A rule set may include a subset of rules. The subset of rules may be hierarchical.

[0090] The relationship between two rules in a rule set can be defined as shown by the directed graphs in Figures 5-7. Figure 5 shows that rule A is more important than rule B. Figure 6 shows that rule A and rule B are not comparable. Figure 7 shows that rule A and rule B are equally important.

[0091] Furthermore, the rule set can be customized and implemented as follows. For example, multiple rules can be combined into one rule. Specifically, if there is a rule α that has a higher priority than rules β and γ, and it is indicated that rules β and γ cannot be compared, rules β and γ may be combined into one rule.

[0092] Furthermore, for example, it is possible to clarify the relationship between two rules whose priority relationship is not clear by adding another perspective. Specifically, if there is rule α that has a higher priority than rules β and γ, and the relationship between rules β and γ is not clear, adding another perspective may make it clear that rule γ has a higher priority than rule β.

[0093] Furthermore, for example, new elements can be taken into consideration by adding rules. Specifically, if there is a rule α that has a higher priority than rules β and γ, and the relationship between rules β and γ is not clear, adding a rule δ that has a higher priority than rules β and γ can improve the priority structure.

[0094] The rule set may be implemented in the form described below to facilitate verification and validation of SOTIF.

[0095] The rule set may be configured as hardware independent from a module for operation planning, for example, the operation planning unit 22. For example, the rule set may be stored in a rule DB 58 provided independently from the dedicated computer 51 that realizes the operation planning unit 22 in the processing system 50. By separating the rule set from a module that is a black box, such as artificial intelligence, it is possible to achieve traceability between the rules defined in the rule set and traffic laws, etc.

[0096] The rule set may be implemented in a form that facilitates verification against known scenarios. For example, the deviation metric or violation metric of each rule against known scenarios may be stored in a recording device 55 or the like. This facilitates scoring the judgment system 20a and the performance of the entire driving system 2. By separating the rule set from the driving plan module as described above, it becomes easy to verify the quality of the specifications of the rule set itself.

[0097] The rule set may be implemented in a form that facilitates validation against unknown scenarios. For example, when the vehicle 1 encounters an unknown, dangerous scenario, the vehicle's motion behavior or actual planning process may be decomposed into rules, allowing poor performance by the driving system 2 to be associated with a rule violation. For example, this may allow rules that the vehicle 1 finds difficult to follow to be identified and fed back to the driving system 2. At least some of the processing or validation in this feedback loop may be performed within the driving system 2 or the processing system 50, or may be aggregated and performed by the external system 96.

[0098] Each rule in the rule set is implemented so that it can be evaluated using metrics such as deviation metrics, violation metrics, etc. These metrics may be functions of strategic factors related to one or both of the state of the strategic guideline and the appropriate action. These metrics may be weighted sums of the strategic factors. These metrics may be the strategic factors themselves or proportional to the strategic factors. These metrics may be inversely proportional to the strategic factors. These metrics may be probability functions of the strategic factors. These metrics may also include at least one of energy consumption, time loss, and financial loss.

[0099] Additionally, a violation metric is a representation of the futility associated with driving behavior that violates a rule statement specified in a rule set. The violation metric may be a value that determines the degree of violation using empirical evidence. The empirical evidence may include crowd-sourced data on what humans consider reasonable, driver preferences, experiments measuring driver parameters, and studies from law enforcement or other authorities.

[0100] The driving behavior referred to here may not necessarily be limited to a trajectory, but may be. In this case, the deviation metric or violation metric includes longitudinal and lateral distance metrics. That is, if the vehicle 1 does not maintain the appropriate longitudinal and lateral distance metrics, the rule is deemed to be violated. The distance metric referred to here may correspond to or correspond to a safety envelope, a safety distance, etc. The distance metric may be expressed as an inverse function of distance.

[0101] <Implementation Example> An example of implementing a rule set in the operation system 2 will be described in more detail below with reference to Fig. 8. In this example, the rule set is used for pre-processing of an operation plan. The operation system 2 implementing the rule set includes a plurality of sensor groups 101, 102, 10n, a rule DB 58, a scenario DB 59, a guideline processor 200, and a planning processor 230. The guideline processor 200 executes a computer program to realize guidelines 211, 212, 21n, which are guidelines individually corresponding to the sensor groups 101, 102, 10n and are the same number as the sensor groups 101, 102, 10n, and a guideline integration 221 that integrates the same number of guidelines 211, 212, 21n.

[0102] Here, the functions realized by the guideline processor 200 may correspond to at least some of the functions of the prediction unit 21. The functions realized by the planning processor 230 may correspond to at least some of the functions of the operation planner 22. Each of the processors 200, 230 is a specific implementation example of at least one of the processors 51b described above. Each of the processors 200, 230 may be mainly configured using an independent semiconductor chip. The guideline processor 200 and the planning processor 230 may be implemented on a common substrate. The guideline processor 200 and the planning processor 230 may be implemented on separate substrates.

[0103] The multiple sensor groups 101, 102, and 10n are configured by classifying the multiple sensors 40 mounted on the vehicle 1 into multiple groups. The multiple sensors 40 here may include an external environment sensor 41 and may further include a communication system 43 and a map DB 44. The number of sensor groups may be any number equal to or greater than two, but setting the number to an odd number equal to or greater than three makes it easier to perform majority voting, extract a median, and the like in the integration process described below. One sensor group may include one or more sensors. Furthermore, some sensors may be shared between the sensor groups, such as when the sensor group 101 includes sensors A and B and the sensor group 102 includes sensors B and C.

[0104] The multiple sensor groups 101, 102, 10n may be classified according to the type of sensor 40, according to the direction monitored by the sensor 40, or according to the coordinate system employed (or detected) by the sensor 40. Furthermore, any other suitable classification method may be employed.

[0105] Classification according to type is, for example, classification according to the type of sensor 40. Classification according to type may simplify the sensor fusion process in each of the sensor groups 101, 102, and 10n. Furthermore, the characteristics of each of the sensor groups 101, 102, and 10n, such as favorable and unfavorable situations, are clarified. This makes it easy to define a policy for applying a rule set to the sensor data of each of the sensor groups 101, 102, and 10n.

[0106] In the classification example by type, the first sensor group includes a plurality of cameras arranged to monitor the front, sides, and rear of the vehicle 1. The second sensor group includes a plurality of millimeter-wave radars arranged to monitor the front, front-side, sides, and rear of the vehicle 1. The third sensor group includes LiDAR arranged to monitor the front, sides, and rear of the vehicle 1. The fourth sensor group includes a map DB 44 and a communication system 43.

[0107] Classification according to direction is, for example, classification by monitoring direction, or classification in which one sensor group covers all directions (360 degrees) around the vehicle 1. Classification by monitoring direction makes it easy to select rules to be applied by each guideline depending on the scenario. On the other hand, classification so that one sensor group covers all directions around the vehicle 1 increases redundancy because even if one sensor group stops functioning due to a failure or the like, rule sets related to each direction can be applied to the sensor data of other sensor groups.

[0108] In the example of classification by direction, the first sensor group includes a camera, millimeter wave radar, and LiDAR arranged to monitor the area ahead of the vehicle 1. The second sensor group includes a camera and millimeter wave radar arranged to monitor the area to the side of the vehicle 1. The third sensor group includes a camera and millimeter wave radar arranged to monitor the area behind the vehicle 1.

[0109] As an example of a classification that covers all directions, the first sensor group is configured to include a camera arranged to monitor the front of the vehicle 1 and a millimeter-wave radar arranged to monitor the sides and rear of the vehicle 1. The second sensor group is a LiDAR arranged to monitor the front of the vehicle 1 and a camera configured to monitor the sides and rear of the vehicle 1. The third sensor group is configured to combine a millimeter-wave radar configured to monitor the front and front-side of the vehicle 1 with a map DB 44 and a communication system 43 that can also obtain information on the rear, etc.

[0110] Each of the sensor groups 101, 102, and 10n belongs to the recognition system 10a and realizes the functions of the recognition unit 10. Each of the sensor groups 101, 102, and 10n outputs sensor data to a corresponding pair of guide lines 211, 212, and 21n. That is, each of the guide lines 211, 212, and 21n receives sensor data from different output sources.

[0111] Each guideline 211, 212, 21n is an evaluator that evaluates a rule based on sensor data from a corresponding pair of sensor groups 101, 102, 10n, a rule set stored in the rule DB 58, and scenario data stored in the scenario DB 59. The guideline 211, 212, 21n is configured to execute processing in accordance with the strategic guideline. The guideline 211, 212, 21n here may refer to a guideline for driving behavior.

[0112] The rule DB 58 stores a rule set in a format that can be read by the processor 200 by a computer program that implements each of the guidelines 211, 212, and 21n.

[0113] The scenario DB 59 stores a scenario structure in a form that can be read by the processor 200 by a computer program in each guideline 211, 212, 21n. Each guideline 211, 212, 21n may recognize the environment in which the vehicle 1 is located based on sensor data input from the sensor groups 101, 102, 10n that form a pair with the guideline 211, 212, 21n. In recognizing the environment, each guideline 211, 212, 21n may refer to the scenario structure and select a scenario that the vehicle 1 is encountering. The selected scenario may be a single scenario or a combination of multiple scenarios. Each guideline 211, 212, 21n may select different scenarios in parallel and proceed with processing.

[0114] Each guideline 211, 212, 21n may specify a known hazardous scenario, a known non-hazardous scenario, an unknown hazardous scenario, and an unknown non-hazardous scenario in selecting a scenario. Each guideline 211, 212, 21n may evaluate rules by referencing a scenario structure, selecting and identifying a scenario. For example, in a known hazardous scenario, rules related to that risk factor should be evaluated negatively (as violated).

[0115] The first guideline 211 calculates a first violation degree sequence for a rule sequence corresponding to a rule set using the first sensor group 101. The second guideline 212 calculates a second violation degree sequence for a rule sequence corresponding to a rule set using the second sensor group 102. The kth guideline calculates a kth violation degree sequence for a rule sequence corresponding to a rule set using the kth sensor group. ｓ The guideline 21n is the nth rule sequence corresponding to the rule set. ｓ Using the sensor group 10n, ｓ Calculate the violation degree sequence, where n ｓ is the total number of sensor groups, where k=1, 2, n ｓ is.

[0116] The rule sequence is expressed by, for example, the following formula 1.

[0117] Here, n ｒ is the total number of rules stored in the rule set. When the same rule is evaluated between the guidelines 211, 212, and 21n, the rule sequence is ｒ Alternatively, when evaluating rules that are partially different between the guidelines 211, 212, and 21n, the ｓ row n ｒ The matrix may be expressed as a matrix of one row and multiple columns. The concept of matrix here is understood to include a configuration of one row and multiple columns and a configuration of multiple rows and one column. When simply referring to a column, the term column is understood to include a matrix of one row and multiple columns and a configuration of multiple rows and multiple columns.

[0118] The violation degree sequence output by the kth guideline to the guideline integration 221 is expressed by, for example, the following Equation 2.

[0119] The violation score column is data in which the violation score for each rule is matrixed. The violation score column can also be considered data in which the values ​​of violation metrics are matrixed, i.e., a matrix of violation metrics. The violation score indicates the evaluation result of the rule as a numerical value. The violation score is 0 when the rule is fully complied with. The violation score is 1 when the rule is completely violated. Each guideline may be configured to output either 0 or 1 as the violation score, or any value in the range from 0 to 1. For example, an intermediate value such as 0.5 may be output as the violation score. For example, an intermediate value may indicate that it is impossible to determine whether the rule is violated due to a decrease in reliability or insufficient sensor data. For example, an intermediate value may indicate a provisional evaluation result of the rule for an unknown scenario that cannot be adequately addressed by the current guideline specifications. The violation score may be an example of a specific implementation of a deviation metric or a violation metric.

[0120] The evaluation of the rule, i.e., the calculation of Equation 2 in response to the input of Equation 1, may be realized solely by a computer program, or may be realized by a trained model using artificial intelligence.

[0121] The guideline integration 221 is an integrator that integrates the evaluation results of the rules according to the guidelines 211, 212, and 21n, and calculates the degree of violation after integration using an integration function that integrates the degree of violation.

[0122] The integration function is expressed by, for example, the following Equation 3.

[0123] The violation degree (violation degree sequence) after integration is expressed by, for example, the following Equation 4: ｒ is.

[0124] The integration function integrates the violation levels evaluated by each guideline 211, 212, 21n for the same rule. For example, the median of the violation levels evaluated by each guideline 211, 212, 21n may be used as the integrated violation level. When the median is used, if each guideline 211, 212, 21n outputs a clear value of 0 or 1, the integrated violation level will also be a clear value of 0 or 1. This prevents driving behaviors based on ambiguous evaluations from being derived in the driving plan, which is post-processing. Furthermore, the mode, average, or weighted average evaluated by each guideline 211, 212, 21n may be used as the integrated violation level.

[0125] Here, specific examples 1 and 2 of the calculation will be described. In specific example 1, the degree of violation against the rule is calculated from three sensor groups. It is assumed that there is a rule that "for a certain host vehicle's trajectory, there should be no objects on the path." Under this rule, each guideline outputs a violation degree of 0 if it is determined that there is no object on the path, and outputs a violation degree of 1 if it is determined that there is an object on the path.

[0126] Consider a case where there is no actual object on the path, the first sensor group mistakenly recognizes a ghost on the path as an object, and the second and third sensor groups do not recognize the ghost on the path. In this case, the first guideline outputs a violation level of 1, and the second and third guideline output a violation level of 0. The integrated guideline uses the median value, 0, as the violation level after integration.

[0127] Also, consider a case where an object is actually present on the path, but the first sensor group is unable to recognize the object due to a recognition disturbance or the like, while the second and third sensor groups are able to recognize the object on the path. In this case, the first guideline outputs a violation level of 0, and the second and third guideline output a violation level of 1. The integrated guideline uses the median value, 1, as the violation level after integration.

[0128] In the second example, the degree of violation of the rules is calculated from five sensors. ｌａｔ is the threshold d ０ There is a rule that states "The number of times the ...

[0129] In this rule, the degree of violation is expressed by the following formula 5.

[0130] The function of Equation 5 means that the maximum value among the values ​​listed in the parentheses is adopted. The value ρ output by Equation 5 may be normalized to take a range from 0 to 1.

[0131] Actual lateral distance d ｌａｔ is the threshold d ０ If only the first sensor group is within the lateral distance d ｌａｔ is the threshold d ０ Consider a case where the first guideline is erroneously detected as being greater than the horizontal distance d. In this case, the first guideline outputs a violation level of 0, and the other guidelines, 2 to 5, output violation levels according to their respective detection distances. The integrated guideline uses the median value as the violation level after integration, but four of the five guidelines, although with some error, are based on the horizontal distance d ｌａｔ is the threshold d ０ Therefore, the degree of violation after integration is determined to be smaller than the horizontal distance d ｌａｔ is the threshold d ０ The violation level is smaller than the threshold.

[0132] The planning processor 230 plans driving behavior based on the integrated violation level output from the guideline integration 221 , the rule set stored in the rule DB 58 , and the scenario data stored in the scenario DB 59 .

[0133] The planning processor 230 refers to the integrated violation level and derives driving actions that allow the vehicle 1 to avoid the violation. There may be cases where it is difficult for the vehicle 1 to avoid the violation. In such cases, the planning processor 230 derives driving actions that minimize the violation level. In minimizing the violation level, the priority structure in the rule set may be referred to.

[0134] The trajectory of the vehicle 1 is composed of a series of positions over time over the duration of the scenario. Thus, the planning processor 230 may derive a driving behavior by aggregating instantaneous violations over time. The aggregation may be, for example, accumulating violations over time. The derived driving behavior may depend on whether a rule was violated lightly over a long period of time or severely over a short period of time. The derived driving behavior may depend on at least one of an average violation over time and a maximum violation over time.

[0135] Here, the similarities and differences in rule evaluation between the guidelines 211, 212, and 21n will be described. In one method, the guidelines 211, 212, and 21n may be configured to evaluate a common rule among multiple rules in a rule set. A configuration that evaluates a common rule is particularly suitable for combination with a classification that covers all directions (360 degrees) around the vehicle using a single sensor group. Furthermore, because a common rule is evaluated, the integration between the guidelines significantly improves evaluation accuracy.

[0136] Each guideline 211, 212, 21n may be configured to evaluate only some of the rules in the rule set, rather than evaluating all of the rules stored in the rule DB 58. Furthermore, the rules evaluated may differ partially or entirely between the guidelines 211, 212, 21n. In this case, the rule DB 58 may additionally store information indicating to which guideline 211, 212, 21n each rule in the rule set applies.

[0137] If a guideline contains rules in a rule sequence that are not subject to evaluation, it is preferable that the violation degree of the rule not calculated by that guideline not return a valid numerical value. Not returning a valid numerical value may include returning an invalid numerical value or maintaining the initial state without setting a numerical value. For example, if valid numerical values ​​for violation degrees are expressed in the range from 0 to 1, the invalid numerical value may be a negative numerical value or a numerical value greater than 1. The initial state is, for example, a null value or a null character. If the violation degree for which a valid numerical value is not returned is included in the integration targets of the guideline integration 221, the guideline integration 221 considers the violation degree to be nonexistent (invalid) and calculates the post-integration violation degree from only the violation degrees of other guidelines.

[0138] The configuration for evaluating different rules among the guidelines 211, 212, and 21n is particularly suitable for combining classification by type or direction. For example, a group of sensors classified into specific sensor types may be made to evaluate rules related to favorable scenes corresponding to the specific sensor type, while excluding evaluation of rules related to unfavorable scenes. This can improve the accuracy of the final evaluation of the violation level before integration.

[0139] Furthermore, even when calculations are performed according to the same rule, the algorithms, parameters, etc. used in the guidelines 211, 212, 21n may differ from one another. Since the data format, coordinate system, dimension, resolution, reliability, error, timing delay effect, etc. of the sensor data input to the guidelines 211, 212, 21n may differ for each sensor group, algorithms, parameters, etc. adjusted in accordance with these factors may be adopted.

[0140] As a specific example, consider a case where a rule set is common to multiple guidelines 211, 212, and 21n. Ultrasonic sonar is suitable for detecting short-range objects. Therefore, guidelines corresponding to a sensor group primarily using ultrasonic sonar may be used only to calculate the degree of violation of rules targeting short-range objects, and not to calculate the degree of violation of rules targeting long-range objects. On the other hand, the communication system 43 is suitable for detecting long-range or blind spot information that cannot be detected by cameras, LiDAR, ultrasonic sonar, etc. Therefore, guidelines corresponding to a sensor group primarily using the communication system 43 may be used to calculate the degree of violation of rules targeting long-range and blind spot objects, and not to calculate the degree of violation of other rules. In other words, each guideline 211, 212, and 21n, based on a common rule set, excludes some of the multiple rules included in the rule set that differ from each other depending on the output source of the sensor data. In this way, each guideline 211, 212, and 21n may evaluate rules that differ from each other among the multiple rules.

[0141] <Handling of Related Data> The driving system 2 is configured to record related data sufficient for accident analysis. The related data may include calculation-related data of the guideline processor 200 and the planning processor 230. The guideline processor 200 and the planning processor 230 sequentially output the calculation-related data to the recording device 55. The recording device 55 sequentially stores the related data in the memory 55a.

[0142] The calculation-related data may include the data of the violation degree or violation degree sequence calculated by each guideline 211, 212, 21 n itself. The calculation-related data may further include the data of the integrated violation degree or violation degree sequence associated with the violation degree or violation degree sequence calculated by each guideline 211, 212, 21 n.

[0143] The calculation-related data is data associated with the data of the violation degree or violation degree sequence calculated by each guideline 211, 212, 21n, and may further include sensor data that was the basis for calculating the violation degree or violation degree sequence. If the sensor data includes camera data, images captured by the camera may be recorded. If the sensor data includes LiDAR data, point cloud data representing the reflection position of reflected light may be recorded.

[0144] The calculation-related data is data associated with the data of the violation degree or violation degree column calculated by each guideline 211, 212, 21n, and may further include selection information of the scenario on which the calculation of the violation degree or violation degree column was based.

[0145] The guideline processor 200 or another processor (e.g., an anomaly detection processor) 51b may further have a function of detecting a failure or erroneous detection of the sensor groups 101, 102, 10n based on the calculation-related data. A failure or erroneous detection of any sensor group 101, 102, 10n may be determined when the absolute value of the difference between the violation degree calculated using the sensor data of the sensor group 101, 102, 10n and the violation degree after integration adopted by the guideline integration 221 is equal to or greater than a detection threshold.

[0146] For example, consider a case where the violation levels output for three sensor groups are 0, 0.1, and 1, respectively, and the combined violation level is the median value of 0.1. If the detection threshold is set to 0.5, the sensor group that outputs a violation level of 1 is determined to have failed or detected incorrectly because the absolute value of the difference is 0.8 (>0.5).

[0147] In a configuration for determining such a failure or false detection, the determination result may be associated with the degree of violation or a sequence of degrees of violation calculated for each guideline, and may further be recorded.

[0148] In detecting a failure or false detection, the guideline processor 200 or another processor 51b may further detect a permanent failure of the sensor group or a temporary false detection due to the characteristics of the sensor group (for example, a difficult scene) in a classifiable manner. These classification results may be further recorded by associating the judgment result with a violation degree or a violation degree sequence calculated for each guideline.

[0149] The driving system 2 or the recording device 55 may generate, as recording data, at least one of the violation degree or violation degree sequence calculated by each guideline 211, 212, 21n, the rule sequence, the integrated violation degree or violation degree sequence, the sensor data of each sensor group 101, 102, 10n, scenario selection information for each guideline 211, 212, 21n, and the determination result of a failure or false detection of each sensor group 101, 102, 10n, so as to store the data in a dedicated data format for recording. At this time, only data related to a sensor group in which a failure or false detection has been detected may be generated and recorded among the data related to the multiple sensor groups 101, 102, 10n.

[0150] The determination result of a failure or erroneous detection may be used for various measures other than recording. For example, a restriction may be set so that a sensor group in which a failure or erroneous detection has been detected is restricted from being used in a function (e.g., an operation plan) of the operation system 2. The restriction may be set by the mode management unit 23.

[0151] Furthermore, for example, a notification of the presence of the sensor group 101, 102, 10n in which a failure or erroneous detection has been detected may be implemented. Specifically, the driving system 2 may present information about an abnormality in the sensor group 101, 102, 10n to the driver using the information presentation type HMI device 70. Furthermore, the driving system 2 may report the abnormality in the sensor group 101, 102, 10n via the communication system 43 to third parties such as the external system 96, a remote center, an operation management company of the vehicle 1, a seller of the vehicle 1, a vehicle manufacturer, a sensor manufacturer, other vehicles, an administrative agency that manages traffic infrastructure, and a certification body that certifies the safety of an automated driving system or the like.

[0152] <Processing Flow> Next, an example of a processing method for realizing a driving function will be described using the flowchart in Fig. 9. The series of processes shown in steps S11 to S15 is executed by the driving system 2 at predetermined time intervals or based on a predetermined trigger. As a specific example, the series of processes may be executed at predetermined time intervals when the autonomous driving mode is managed at autonomous driving level 3 or higher. As another specific example, the series of processes may be executed at predetermined time intervals when the autonomous driving mode is managed at autonomous driving level 2 or higher.

[0153] In S11, each guide line 211, 212, 21n acquires the latest sensor data from the sensor groups 101, 102, 10n that are paired with the guide line 211, 212, 21n. After the process of S11, the process proceeds to S12.

[0154] In S12, each of the guide lines 211, 212, and 21n acquires a rule from the rule DB 58 and evaluates the rule using the sensor data acquired in S1. After the process of S12, the process proceeds to S13.

[0155] In S13, the guideline integration 221 integrates the evaluation results of the rules in the guidelines 211, 212, and 21n, and outputs the integrated evaluation results to the planning processor 230. After the processing of S13, the process proceeds to S14.

[0156] In S14, the planning processor 230, based on the integrated evaluation results, plans the driving behavior of the vehicle 1. After processing in S14, the process proceeds to S15.

[0157] In S15, at least one of the guideline processor 200 and the planning processor 230 generates record data and outputs the record data to the recording device 55. The recording device 55 stores the record data in the memory 55a. The series of processes ends with S15.

[0158] According to the first embodiment described above, evaluations of the strategic guidelines are performed individually for each of the sensor data output sources, with at least some of the sensor data output sources being different from each other. This process allows the final driving behavior and the causal relationship with the sensors to be broken down into individual evaluation results for analysis. This improves the traceability of the planned driving behavior.

[0159] Furthermore, according to the first embodiment, each guideline 211, 212, 21n outputs a matrix of violation metrics, which are individual evaluation results for a rule set including multiple rules. By comparing multiple violation metric matrices individually output based on sensor data from at least some different sources, it becomes easier to identify the causal relationship between the violation metrics used to derive driving behavior and the sensors. Therefore, it is possible to improve the traceability of driving behavior.

[0160] Furthermore, according to the first embodiment, a matrix of integrated violation metrics is generated based on the matrix of multiple violation metrics output from each guideline 211, 212, 21n, and is output as an integrated evaluation result. Therefore, it is possible to derive driving behavior that appropriately reflects each sensor data.

[0161] Furthermore, according to the first embodiment, a group of sensors in which a failure or a false detection has occurred is identified based on the matrix of each violation metric, thereby improving the traceability of driving behavior.

[0162] Furthermore, according to the first embodiment, the multiple guidelines 211, 212, and 21 n evaluate multiple common rules based on a rule set that is shared among the multiple guidelines 211, 212, and 21 n. By sharing the evaluation targets, the integration process of the evaluation results can be performed easily and with high accuracy.

[0163] Furthermore, according to the first embodiment, the guidelines 211, 212, and 21n perform evaluations for the same rule using different algorithms according to the difference in the sensor data output source, thereby enabling rule evaluations to be performed appropriately for various types of sensors.

[0164] Furthermore, according to the first embodiment, the guidelines 211, 212, and 21n perform evaluation for the same rule using the same algorithm and different parameters according to the difference in the sensor data output source, thereby making it possible to perform rule evaluation appropriately according to the difference in the characteristics of the output source sensor.

[0165] Furthermore, according to the first embodiment, based on a rule set commonly provided among the multiple guidelines 211, 212, and 21n, some of the multiple rules included in the rule set are excluded from evaluation depending on differences in the sensor data output source. This allows for evaluation of rules that are partially different from each other among the multiple rules. By performing evaluations that are specialized for appropriate rules depending on differences in the characteristics of the output source sensors, individual evaluation results that are expected to be low in accuracy can be excluded. Since the accuracy of the individual evaluation results is improved, the validity of the integrated evaluation results can be improved.

[0166] Furthermore, according to the first embodiment, data associating the individual evaluation results with the integrated evaluation results is generated and stored in the memory 55 a as a storage medium. Storing the evaluation results together enables appropriate subsequent verification and validation.

[0167] Furthermore, according to the first embodiment, the multiple guidelines 211, 212, and 21n are realized by a single common processor 200. By using the common processor 200, it is not necessary to aggregate information such as individual evaluation results at the time of integration between devices, and therefore it is possible to execute the integration process with reduced delay.

[0168] In the first embodiment, the guidelines 211, 212, and 21n correspond to an individual evaluation unit, and the guideline integration 221 corresponds to an integrated evaluation unit.

[0169] Second Embodiment As shown in Fig. 10, the second embodiment is a modification of the first embodiment. The second embodiment will be described, focusing on the differences from the first embodiment.

[0170] The driving system 2 of the second embodiment includes a plurality of sensor groups 101, 102, and 10n, a plurality of guideline processors 201, 202, 20n, and 220, and a planning processor 230. The number of guideline processors 201, 202, 20n, and 220 provided is the total number of sensor groups 101, 102, and 10n plus one. Each guideline processor 201, 202, and 20n includes individual processors 201, 202, and 20n provided in the same number as the sensor groups 101, 102, and 10n, and one integrated processor 220.

[0171] Each individual processor 201, 202, 20n corresponds to one of the sensor groups 101, 102, 10n. Each individual processor 201, 202, 20n implements a guideline 211, 212, 21n, respectively, using sensor data input from the corresponding sensor group 101, 102, 10n by executing a computer program. The processing of the guideline 211, 212, 21n is the same as in the first embodiment.

[0172] The integration processor 220 acquires the violation degrees or violation degree sequences calculated by the individual processors 201, 202, and 20n, and integrates them by executing a computer program to realize the guideline integration 221. The process of the guideline integration 221 is the same as that in the first embodiment.

[0173] The multiple guideline processors 201, 202, 20n, and 220 and the planning processor 230 may be implemented on a common substrate. The multiple guideline processors 201, 202, 20n, and 220 and the planning processor 230 may be implemented on separate substrates. Furthermore, the multiple individual processors 201, 202, and 20n and the integrated processor 220 may be implemented on separate substrates. The multiple individual processors 201, 202, and 20n may be implemented on a common substrate, or may each be implemented on a separate substrate.

[0174] The rule DB 58 may be provided in common to the plurality of individual processors 201, 202, 20n, in which case each of the individual processors 201, 202, 20n accesses the common rule DB 58 and refers to the rule set.

[0175] On the other hand, multiple rule DBs 58 may be provided to correspond to the individual processors 201, 202, and 20n. In this configuration, a rule set can be optimized for each guideline 211, 212, and 21n. That is, it becomes possible to evaluate rules suitable for each sensor group 101, 102, and 10n in an appropriate manner.

[0176] According to the second embodiment described above, the multiple guidelines 211, 212, and 21n are implemented by separate processors 201, 202, and 20n, respectively. This allows evaluation to continue using the remaining processors even if an abnormality occurs in one of the processors 201, 202, and 20n. This improves the redundancy of the processing system 50.

[0177] Third Embodiment As shown in Fig. 11, the third embodiment is a modification of the first embodiment. The third embodiment will be described, focusing on the differences from the first embodiment.

[0178] The driving system 2 of the second embodiment includes a plurality of sensor groups 101, 102, and 10n, and a plurality of processors 241, 242, 24n, and 260. The number of processors 241, 242, 24n, and 260 is the total number of sensor groups 101, 102, and 10n plus one. Each processor 241, 242, and 24n includes the same number of individual processors 241, 242, and 24n as the number of sensor groups 101, 102, and 10n, and one integrated processor 260.

[0179] Each individual processor 201, 202, 20n corresponds to one of the sensor groups 101, 102, 10n. Each individual processor 201, 202, 20n realizes one guideline 211, 212, 21n and one planning 251, 252, 25n, respectively, by executing a computer program, using sensor data input from the sensor group 101, 102, 10n that is paired with the individual processor 201, 202, 20n.

[0180] Each of the plans 251, 252, and 25n is a planner that derives a driving action that allows the vehicle 1 to avoid a violation or minimize a violation by referring to the violation level or violation level sequence obtained from the corresponding pair of guidelines 211, 212, and 21n.

[0181] The processing of the guidelines 211, 212, and 21n is the same as in the first embodiment. On the other hand, the plannings 251, 252, and 25n are individually provided for the sensor groups 101, 102, and 10n and the guidelines 211, 212, and 21n. That is, each of the plannings 251, 252, and 25n derives a driving behavior and outputs it to the integrated processor 260.

[0182] The integrated processor 260 integrates the driving actions derived by the individual processors 241, 242, and 24n into one. For example, the integrated processor 260 may determine whether or not to change lanes by majority vote based on the driving actions derived by the individual processors 241, 242, and 24n.

[0183] According to the third embodiment described above, evaluations of the strategic guidelines are performed individually for each of the sensor data output sources, with at least some of the sensor data output sources being different from each other. This process allows the final driving behavior and the causal relationship with the sensors to be analyzed by breaking down the individual evaluation results. This improves the traceability of the planned driving behavior.

[0184] In the third embodiment, the guidelines 211, 212, and 21n correspond to an individual evaluation unit, the planning units 241, 242, and 24n correspond to an individual operation plan unit, and the planning integration unit 261 corresponds to an integrated operation plan unit.

[0185] 12 and 13, the fourth embodiment is a modification of the first embodiment. The fourth embodiment will be described, focusing on the differences from the first embodiment.

[0186] In the fourth embodiment, a rule set is used to check or monitor an operation plan. The operation system 2 includes a plurality of sensor groups 101, 102, 10n, a guideline processor 300, a sensor fusion processor 160, and a planning processor 330 (see FIG. 12 ). The functions realized by the guideline processor 300 may correspond to some of the functions of the prediction unit 21 and the operation planning unit 22. The functions realized by the sensor fusion processor 160 may correspond to some of the functions of the recognition unit 10. The functions realized by the planning processor 330 may correspond to at least some of the functions of the operation planning unit 22.

[0187] The sensor fusion processor 160 acquires sensor data from multiple sensor groups 101, 102, and 10n, fuses the sensor data, and generates an environmental model. The planning processor 330 uses the environmental model to derive driving behaviors. Here, the planning processor 330 provides information on the derived tentative driving behaviors. The tentative driving behaviors are candidates for driving behaviors to be executed.

[0188] The guideline processor 300 evaluates the driving behavior provided from the planning processor 330 using the rule set. Specifically, each guideline 311, 312, 31n uses the sensor data from its corresponding pair of sensor groups 101, 102, 10n, the rule set, and the scenario structure to determine whether the driving behavior violates the rules in the rule set. As in the first embodiment, each guideline 311, 312, 31n outputs a violation degree or a violation degree sequence to the guideline integration 321. The guideline integration 321 integrates the violation degrees or violation degree sequences input from each guideline 311, 312, 31n. The guideline integration 321 outputs the integrated violation degree or violation degree sequence to the planning processor 330 as an evaluation result of the rules for the driving behavior. The planning processor 330 determines the final driving behavior of the vehicle 1 based on this evaluation result.

[0189] Here, the planning processor 330 may cause the guideline processor 300 to compare a plurality of driving behaviors. In this case, the guideline processor 300 may calculate a separate violation level or a series of violation levels for each driving behavior, and output the performance difference between the driving behaviors to the planning processor 330 as an evaluation result.

[0190] Next, an example of a processing method for realizing the driving function will be described using the flowchart in Fig. 13. The series of processes shown in steps S21 to S25 is executed by the driving system 2 at predetermined time intervals or based on a predetermined trigger. As a specific example, the series of processes may be executed at predetermined time intervals when the autonomous driving mode is managed at autonomous driving level 3 or higher. As another specific example, the series of processes may be executed at predetermined time intervals when the autonomous driving mode is managed at autonomous driving level 2 or higher.

[0191] In S21, the sensor fusion processor 160 fuses the sensor data from the plurality of sensor groups 101, 102, and 10n. After the process of S21, the process proceeds to S22.

[0192] In S22, the planning processor 330 calculates a tentative driving behavior. After the process of S22, the process proceeds to S23.

[0193] In S23, the guideline processor 300 evaluates the rules against the tentative driving behavior calculated in S22. After the process of S23, the process proceeds to S24.

[0194] In S24, the planning processor 330 refers to the evaluation result of S23 and determines the final driving behavior. After the processing of S24, the process proceeds to S25.

[0195] In S25, at least one of the guideline processor 300 and the planning processor 330 generates record data and outputs the record data to the recording device 55. The recording device 55 stores the record data in the memory 55a. The series of processes ends with S25.

[0196] According to the fourth embodiment described above, the guidelines 311, 312, and 31n output individual evaluation results related to the strategic guidelines for the tentative driving actions. Then, the individual evaluation results for the tentative driving actions are integrated, and the integrated evaluation result is output. The final driving action is determined by referring to the integrated evaluation result for the tentative driving actions. In this way, the rule set can be used for the monitoring function of whether the driving plan is valid, thereby improving the safety of the driving system 2.

[0197] In the fourth embodiment, the guidelines 311, 312, and 31n correspond to an individual evaluation unit, and the guideline integration 321 corresponds to an integrated evaluation unit.

[0198] Fifth Embodiment As shown in Fig. 14, the fifth embodiment is a modification of the first embodiment. The fifth embodiment will be described, focusing on the differences from the first embodiment.

[0199] In the fifth embodiment, the generated record data (e.g., related data related to accident investigation) may be transmitted to the external system 96 by communication (e.g., V2X communication) through the communication system 43 and stored in a storage medium 98 of the external system 96. When transmitting the record data to the external system 96, the record data may be generated and transmitted as encoded data so as to conform to a specific format such as an SDM (Safety Driving Model) message. The transmission to the external system 96 does not have to be direct transmission of radio waves from the vehicle 1 to the external system 96, but may be transmission using a relay terminal such as a roadside device and a network.

[0200] The external system 96 includes a dedicated computer 97 having at least one memory 97a and at least one processor 97b, and at least one large-capacity storage medium 98. In the dedicated computer 97, the memory 97a may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores programs and data readable by the processor 97b. The memory 97a may further include a rewritable volatile storage medium, such as a random access memory (RAM). The processor 97b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU. The storage medium 98 may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium.

[0201] The external system 96 decodes the received message and may sequentially record the recorded data in a memory area reserved for each vehicle in the storage medium 98. The external system 96 may also collect recorded data from a number of vehicles traveling on a road and sequentially record the recorded data in a common memory area.

[0202] The accumulated data may be used as big data for improving the road network. For example, the external system 96 may identify accident-prone locations from the accumulated data and further identify rules that are frequently violated at the accident-prone locations. This makes it possible to improve the road structure at the accident-prone locations to make it less likely for violations of the identified rules to occur.

[0203] The accumulated data may be used to verify and validate the driving system 2 or the underlying safety model to enable an efficient SOTIF process. For example, the driving system 2 can be improved by analyzing the causal relationship between the sensor data, the violation degree calculated accordingly, and the driving behavior of the vehicle 1. The improvement of the driving system 2 includes at least one of improving the rule evaluation algorithm and improving the rule set. The improvement of the rule set includes at least one of changing the rules themselves and changing the priority structure.

[0204] According to the fifth embodiment described above, data is generated that associates individual evaluation results with integrated evaluation results. This data is transmitted to an external system 96 located outside the vehicle 1 via the communication system 43 installed in the vehicle 1. By transmitting the evaluation results collectively to the outside, even if the data stored in the vehicle 1 is damaged due to an accident or the like, subsequent verification and validation can be easily performed. Furthermore, it becomes easy for the external system 96 to aggregate and utilize data from multiple vehicles.

[0205] (Other Embodiments) Although multiple embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0206] In another embodiment, the recorded data of the second to fourth embodiments may be transmitted to an external system 96 by communication through the communication system 43 (e.g., V2X communication) and stored in a storage medium 98 of the external system 96, as in the fifth embodiment.

[0207] In another embodiment, the guideline processor 200, the rule DB 58, and the scenario DB 59 may be installed in a vehicle that is manually driven by a driver. For example, the violation level or the sequence of violation levels output by the guideline processor 200 may be recorded as record data by the recording device 55, and the record data may be used to evaluate manual driving.

[0208] Also, for example, the rule evaluation results by the guideline processor 200 may be presented to the driver during manual driving by the information presentation type HMI device 70. In this example, presentation of information about rules with a violation degree of 0 may be omitted, and only information about rules with a violation degree equal to or greater than a predetermined threshold may be presented. The threshold may be 0.5 or 1. In this way, it is preferable to select rule violations for which information is presented, taking into consideration the annoyance felt by the driver.

[0209] The controller and methods described herein may be implemented by a special-purpose computer comprising a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and methods described herein may be implemented by special-purpose hardware logic circuitry. Alternatively, the apparatus and methods described herein may be implemented by one or more special-purpose computers comprising a processor executing a computer program in combination with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0210] (Explanation of Terms) Terms related to the present disclosure are explained below. This explanation is included in the embodiments of the present disclosure.

[0211] A road user may be a person using a road, including sidewalks and other adjacent spaces. Road users may include pedestrians, cyclists, other VRUs, and vehicles (e.g., human-driven automobiles, vehicles equipped with automated driving systems).

[0212] A dynamic driving task (DDT) may be a real-time operational and tactical function for operating a vehicle in traffic.

[0213] An automated driving system may be a collection of hardware and software capable of performing the entire DDT on a continuous basis, whether or not it is limited to a specific operational design domain.

[0214] Safety of the intended functionality (SOTIF) may be the absence of undue risk due to insufficient functionality of the intended functionality or its implementation.

[0215] A driving policy may be a strategy and rules that define control behavior at the vehicle level.

[0216] A safety-relevant object may be any dynamic or static object that may be relevant to the safety performance of the DDT.

[0217] A scenario may be a depiction of the temporal relationships between several scenes in a sequence of scenes, including the goals and values ​​in a specific situation influenced by actions and events. A scenario may be a depiction of a continuous time series of activities that integrates a subject vehicle, all its external environments, and their interactions in the process of performing a specific driving task.

[0218] A triggering condition may be a specific condition of a scenario that acts as a catalyst for subsequent system responses that contribute to unsafe behavior, failure to prevent, detect, and mitigate reasonably foreseeable indirect misuse.

[0219] A strategic guideline may be at least one state and at least one appropriate driving action associated with that state. The term strategic guideline is used broadly to refer to any expression, explanation, description, definition, or logical relationship derived from one or more fundamental principles. A strategic guideline may be synonymous with a logical expression.

[0220] A hazardous situation may be an increased risk for a potential breach of the safety envelope and also represents an increased risk level present in the DDT.

[0221] A safety envelope may be a set of limits and conditions within which an (automated) driving system is designed to operate, subject to constraints or controls, in order to maintain operation within an acceptable level of risk. A safety envelope may be a general concept that can be used to accommodate all principles to which a driving policy can adhere, according to which an ego-vehicle operated by an (automated) driving system may have one or more boundaries around it.

[0222] (Disclosure of Technical Ideas) This specification discloses multiple technical ideas described in the following multiple clauses. Some clauses may be described in a multiple dependent form, where the subsequent clause alternatively cites the preceding clause. These multiple dependent clauses define multiple technical ideas.

[0223] <Technical Idea 1> A processing system that executes processing related to driving of a vehicle (1), comprising: a plurality of individual evaluation units (211, 212, 21n, 311, 312, 31n) that output individual evaluation results related to strategic guidelines based on sensor data, wherein at least some of the sensor data output sources are different from one another; an integrated evaluation unit (221, 321) that integrates the individual evaluation results and outputs the integrated evaluation result; and a driving planning unit (22) that plans driving behavior based on the integrated evaluation result.

[0224] <Technical Concept 2> The processing system according to Technical Concept 1, wherein each of the individual evaluation units outputs a matrix of violation metrics that is the individual evaluation result for a rule set including a plurality of rules.

[0225] <Technical Idea 3> The processing system described in Technical Idea 2, wherein each of the integrated evaluation units generates a matrix of violation metrics after integration based on the matrix of the multiple violation metrics output from each of the individual evaluation units, and outputs the matrix as the evaluation result after integration.

[0226] <Technical Concept 4> The processing system according to Technical Concept 2 or 3, which identifies a group of sensors in which a failure or false detection has occurred based on a matrix of each of the violation metrics.

[0227] <Technical Idea 5> The rule set is provided in common among a plurality of the individual evaluation units, and each of the individual evaluation units evaluates a plurality of rules that are common to each other based on the rule set. This is a processing system described in any one of Technical Ideas 2 to 4.

[0228] <Technical Concept 6> The processing system according to Technical Concept 5, wherein each of the individual evaluation units executes evaluation for the same rule using a different algorithm according to a difference in the output source of the sensor data.

[0229] <Technical Idea 7> The processing system according to Technical Idea 5, wherein each of the individual evaluation units executes evaluation for the same rule using the same algorithm and different parameters according to differences in the output source of the sensor data.

[0230] <Technical Idea 8> The rule set is provided in common among the plurality of individual evaluation units, and each of the individual evaluation units evaluates the plurality of rules that are partially different from each other, by excluding some of the plurality of rules included in the rule set from evaluation based on the rule set depending on differences in the output source of the sensor data.

[0231] <Technical Idea 9> The processing system according to any one of Technical Ideas 1 to 8, wherein data associating the individual evaluation results with the integrated evaluation results is generated and stored in a storage medium (55a).

[0232] <Technical Idea 10> The processing system according to any one of Technical Ideas 1 to 9, wherein data associating the individual evaluation results with the integrated evaluation result is generated, and the data is transmitted to an external system (96) located outside the vehicle via a communication system (43) mounted on the vehicle.

[0233] <Technical Concept 11> The processing system according to any one of Technical Concepts 1 to 10, wherein the plurality of individual evaluation units are realized by a single common processor (200, 300).

[0234] <Technical Concept 12> The processing system according to any one of Technical Concepts 1 to 10, wherein the plurality of individual evaluation units are realized by separate processors (201, 202, 20n) corresponding to the individual evaluation units.

[0235] <Technical Idea 13> The processing system according to any one of Technical Ideas 1 to 12, wherein the driving planning unit derives a tentative driving behavior; the individual evaluation unit outputs an individual evaluation result regarding a strategic guideline for the tentative driving behavior; the integrated evaluation unit integrates the individual evaluation results for the tentative driving behavior and outputs an integrated evaluation result; and the driving planning unit determines a final driving behavior by referring to the integrated evaluation result for the tentative driving behavior.

[0236] <Technical Idea 14> A processing system that executes processing related to driving of a vehicle (1), comprising: a plurality of individual evaluation units (211, 212, 21n) that output individual evaluation results related to strategic guidelines based on sensor data, wherein at least some of the output sources of the sensor data are different from each other; a plurality of individual driving planning units (251, 252, 25n) that are provided individually corresponding to each of the individual evaluation units and plan individual driving actions based on the individual evaluation results output by the paired individual evaluation units; and an integrated driving planning unit (261) that integrates each of the individual driving actions and plans a driving action after integration.

[0237] <Technical Idea 15> A processing system that executes processing related to driving of a vehicle (1), comprising: a guideline processor (200); and a planning processor (230), wherein the guideline processor realizes a plurality of guideline functions (211, 212, 21n, 311, 312, 31n) that are functions that output individual evaluation results related to strategic guidelines based on sensor data, and at least some of the sensor data output sources are different from one another, and a guideline integration function (221, 321) that integrates each of the individual evaluation results and outputs the integrated evaluation result, and the planning processor outputs a driving action plan in response to input of the integrated evaluation result.

[0238] According to this technical concept, it is possible to improve the traceability of planned driving behavior.

[0239] <Technical Idea 16> A processing system that executes processing related to driving of a vehicle (1), comprising: a plurality of guideline processors (211, 212, 21n, 220); and a planning processor (230), wherein the plurality of guideline processors have: individual processors that respectively realize a plurality of guideline functions (211, 212, 21n, 311, 312, 31n) that have a function of outputting individual evaluation results related to strategic guidelines based on sensor data, wherein at least a portion of the output sources of the sensor data differ among the plurality of guideline processors; and an integration processor that realizes a guideline integration function (221, 321) that integrates the individual evaluation results and outputs a post-integration evaluation result, wherein the planning processor outputs a driving action plan in response to input of the post-integration evaluation result.

[0240] According to this technical concept, it is possible to improve the traceability of planned driving behavior.

[0241] <Technical Idea 17> A processing system that executes processing related to driving of a vehicle (1), comprising: a plurality of individual processors (211, 212, 21n); and an integrated processor (260), wherein each of the individual processors realizes a plurality of guideline functions (211, 212, 21n) that output individual evaluation results related to strategic guidelines based on sensor data, wherein at least some of the output sources of the sensor data differ between the plurality of individual processors; and an individual planning function (251, 252, 25n) that outputs individual driving action plans based on the individual evaluation results, wherein the integrated processor integrates each of the individual driving actions and outputs the integrated driving action plan.

[0242] According to this technical concept, it is possible to improve the traceability of planned driving behavior.

[0243] <Technical Idea 18> A method for performing processing related to a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220), the method including: acquiring individual sensor data from a plurality of sensor groups (101, 102, 10n); performing individual evaluations related to strategic guidelines in parallel based on the individual sensor data; and integrating results of the individual evaluations and outputting the integrated evaluation result.

[0244] This technical concept makes it possible to improve traceability in vehicle-related processes.

[0245] <Technical Idea 19> A method for executing processing related to a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220, 241, 242, 24n), the method comprising: identifying an unknown scenario that the vehicle is encountering by referring to a scenario structure stored in a scenario database (59); and evaluating rules defined in the rule set in the unknown scenario based on a rule set stored in a rule database (58) and the identification result.

[0246] This technical concept makes it possible to provide feedback on unknown scenarios that a vehicle encounters.

[0247] <Technical Idea 20> A method for executing processing related to a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220, 241, 242, 24n), the method comprising: identifying an unknown scenario that the vehicle is encountering by referring to a scenario structure stored in a scenario database (59); and identifying, based on a rule set stored in a rule database (58) and the identification result, rules in the unknown scenario that are defined in the rule set that are difficult for the vehicle to follow.

[0248] This technical concept makes it possible to provide feedback on unknown scenarios that a vehicle encounters.

[0249] <Technical Idea 21> A recording medium for storing data relating to the driving behavior of a vehicle (1), the recording medium storing the values ​​of violation metrics associated with driving behavior that violates rule statements defined in a rule set and the sensor data used to calculate the violation metrics in association with each other.

[0250] According to this technical concept, it is possible to improve the traceability of vehicle driving behavior.

[0251] <Technical Idea 22> A method for generating data regarding driving behavior of a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220, 241, 242, 24n), the method comprising: using sensor data to calculate a value of a violation metric associated with driving behavior that violates a rule statement specified in a rule set; and generating data stored in a dedicated data format such that the value of the violation metric is associated with the sensor data used to calculate the violation metric.

[0252] According to this technical concept, it is possible to improve the traceability of vehicle driving behavior.

[0253] <Technical Idea 23> A recording medium for storing data relating to the driving behavior of a vehicle (1), the recording medium storing, in association with each other, a plurality of violation metric values ​​associated with driving behavior that violates a rule statement defined in a rule set, the violation metric values ​​being calculated using sensor data at least some of which have different output sources, and an integrated violation metric value obtained by integrating the values ​​of the plurality of violation metrics.

[0254] According to this technical concept, it is possible to improve the traceability of vehicle driving behavior.

[0255] <Technical Idea 24> A method for generating data regarding the driving behavior of a vehicle (1) by at least one processor (51b, 200, 201, 202, 20n, 220), the method comprising: calculating multiple violation metric values ​​associated with driving behavior that violates rule statements specified in a rule set, the violation metric values ​​being calculated using sensor data with at least some of the data being from different output sources; integrating the multiple violation metric values ​​and calculating the integrated violation metric value; and generating data stored in a dedicated data format so that the multiple violation metric values ​​before integration and the integrated violation metric value are associated with each other.

[0256] According to this technical concept, it is possible to improve the traceability of vehicle driving behavior.

[0257] <Technical Idea 25> A system for aggregating information from multiple vehicles, comprising at least one processor (97b) and at least one storage medium (98), wherein the at least one processor receives a message sent from the vehicle and including a violation metric value associated with a driving behavior that violates a rule statement specified in a rule set and sensor data used to calculate the violation metric, and stores the violation metric value and the sensor data in the at least one storage medium.

[0258] According to this technical concept, it is possible to improve the traceability of vehicle driving behavior.

[0259] <Technical Idea 26> A driving system for executing a dynamic driving task of a vehicle (1), comprising: a plurality of sensor groups (101, 102, 10n) configured by classifying a plurality of sensors (40) provided on the vehicle, each providing sensor data; a rule database (58) storing rule sets that implement a priority structure for a series of rules arranged based on relative importance; a scenario database (59) storing a scenario structure including a plurality of scenarios that show the vehicle, an external environment, and interactions therebetween in a process of executing the dynamic driving task; at least one processor (51b, 200, 201, 202, 20n, 220, 230); and at least one recording medium (55a), wherein the at least one processor: acquires the sensor data from each of the sensor groups; accesses the rule database to acquire the rule sets; accesses the scenario database to acquire the scenario structure; A driving system that calculates multiple violation degrees against the rules based on the sensor data, the rule set, and the scenario structure, each calculated using sensor data from a different sensor group that outputs the sensor data; integrates the multiple violation degrees to calculate a post-integration violation degree; executes the dynamic driving task based on the post-integration violation degrees; and records the multiple violation degrees before integration in the at least one storage medium.

[0260] According to this technical concept, it is possible to improve the traceability of the operation system.

[0261] <Technical Idea 27> A method of driving a vehicle (1) for executing a dynamic driving task of the vehicle (1), comprising: acquiring sensor data provided by a plurality of sensor groups formed by classifying a plurality of sensors (40) provided on the vehicle; acquiring, from a rule database (58), a rule set that implements a priority structure for a series of rules arranged based on relative importance; acquiring, from a scenario database (59), a scenario structure including a plurality of scenarios that represent the vehicle, an external environment, and their interactions in the process of executing the dynamic driving task; calculating, based on the sensor data, the rule set, and the scenario structure, a plurality of violation degrees against the rules, the violation degrees being calculated using sensor data from the sensor groups that are output from different sensors; integrating the plurality of violation degrees to calculate a post-integration violation degree; executing the dynamic driving task based on the post-integration violation degrees; and recording the plurality of violation degrees before integration in at least one storage medium (55a).

[0262] This technical concept can improve the traceability of the vehicle's execution of dynamic driving tasks.

[0263] <Technical Idea 28> A method for generating individual evaluation results for strategic guidelines, comprising: acquiring sensor data from a plurality of sensor groups installed in a vehicle; based on the sensor data acquired from each sensor group, an individual evaluation unit installed corresponding to each sensor group generates a matrix of violation metrics, which is an individual evaluation result for a rule set including a plurality of rules; based on the matrix of violation metrics calculated by each individual evaluation unit, generating a matrix of integrated violation metrics, which is an integrated evaluation result; and recording the individual matrix of violation metrics on a recording medium.

[0264] This technical idea can improve the traceability of evaluation results related to strategic guidelines.

Claims

1. A processing system for executing processing related to driving of a vehicle (1), A strategic guideline is defined as at least one state and at least one appropriate driving action for that state. A plurality of individual evaluation units (211, 212, 21n, 311, 312, 31n) that output individual evaluation results regarding the strategic guideline based on sensor data, wherein at least a portion of the output sources of the sensor data are different from each other; an integrated evaluation unit (221, 321) that integrates the individual evaluation results and outputs the integrated evaluation result; A processing system comprising: a driving planning unit (22) that plans driving behavior based on the integrated evaluation result.

2. The processing system according to claim 1 , wherein each of the individual evaluation units outputs a matrix of violation metrics that is the individual evaluation result for a rule set including a plurality of rules.

3. The processing system according to claim 2 , wherein the integrated evaluation unit generates a matrix of violation metrics after integration based on the matrices of the violation metrics output from each of the individual evaluation units, and outputs the matrix of violation metrics after integration as the evaluation result after integration.

4. The processing system according to claim 3 , further comprising: a processing unit configured to identify an output source of the sensor data in which a failure or a false detection has occurred based on a matrix of each of the violation metrics.

5. The rule set is provided in common among the plurality of individual evaluation units, The processing system according to claim 2 , wherein each of the individual evaluation units evaluates a plurality of rules that are common to each other based on the rule set.

6. The processing system according to claim 5 , wherein each of the individual evaluation units executes evaluation for the same rule using a different algorithm according to a difference in an output source of the sensor data.

7. The processing system according to claim 5 , wherein each of the individual evaluation units executes evaluation for the same rule using a same algorithm and different parameters according to differences in the output source of the sensor data.

8. The rule set is provided in common among the plurality of individual evaluation units, The processing system according to claim 2 , wherein each of the individual evaluation units evaluates the rules that differ from each other in part among the plurality of rules by excluding some of the rules included in the rule set from evaluation based on the rule set depending on the difference in the output source of the sensor data.

9. The processing system according to claim 1 , further comprising: generating data associating the individual evaluation results with the integrated evaluation result, and storing the data in a storage medium (55a).

10. The processing system of claim 1, further comprising: generating data associating the individual evaluation results with the integrated evaluation results; and transmitting the data to an external system (96) outside the vehicle via a communication system (43) mounted on the vehicle.

11. The processing system according to claim 1 , wherein the individual evaluation units are implemented by a single common processor (200, 300).

12. 2. The processing system according to claim 1, wherein the plurality of individual evaluation units are realized by separate processors (201, 202, 20n) each corresponding to an individual one of the evaluation units.

13. The driving plan unit derives a tentative driving action, The individual evaluation unit outputs an individual evaluation result regarding the strategic guideline for the provisional driving behavior; The integrated evaluation unit integrates the individual evaluation results for the provisional driving behavior and outputs the integrated evaluation result; The processing system according to claim 1 , wherein the driving planner determines a final driving action by referring to an evaluation result after integration of the tentative driving actions.

14. A processing system for executing processing related to driving of a vehicle (1), A plurality of individual evaluation units (211, 212, 21n) that output individual evaluation results regarding the strategic guideline based on sensor data, at least a part of the output sources of the sensor data being different from each other; A plurality of individual driving planners (251, 252, 25 n) are provided to correspond to the individual evaluation units individually, and plan individual driving actions based on the individual evaluation results output by the paired individual evaluation units; and an integrated driving planning unit (261) that integrates each of the individual driving actions and plans a driving action after integration.

15. A processing system for executing processing related to driving of a vehicle (1), A plurality of individual evaluation units (211, 212, 21n, 311, 312, 31n) that output individual evaluation results regarding the strategic guideline based on sensor data, wherein at least a portion of the output sources of the sensor data are different from each other; an integrated evaluation unit (221, 321) that integrates the individual evaluation results and outputs the integrated evaluation result; and a driving planning unit (22) that plans driving behavior based on the integrated evaluation result, wherein each of the individual evaluation units outputs a matrix of violation metrics that is the individual evaluation result for a rule set including a plurality of rules.

16. A processing system for executing processing related to driving of a vehicle (1), A plurality of individual evaluation units (211, 212, 21n, 311, 312, 31n) that output individual evaluation results regarding the strategic guideline based on sensor data, wherein at least a portion of the output sources of the sensor data are different from each other; an integrated evaluation unit (221, 321) that integrates the individual evaluation results and outputs the integrated evaluation result; A driving planner (22) for planning driving behavior based on the integrated evaluation result, The driving plan unit derives a tentative driving action, The individual evaluation unit outputs an individual evaluation result regarding the strategic guideline for the provisional driving behavior; The integrated evaluation unit integrates the individual evaluation results for the provisional driving behavior and outputs the integrated evaluation result; The driving planner determines a final driving action by referring to an integrated evaluation result of the tentative driving action.