Driving system, processing module, and method

Abstract

This driving system includes a processor that executes processing related to self-driving on the basis of detection results from a plurality of sensors. The plurality of sensors include a front camera and a front radar. Upon detecting a malfunction of the front camera, the processor locks detection result data received from the front camera, and stops the use of the front camera. Thereafter, if there is still a vehicle within a final recognition range of the front camera, a fail operation is executed on the basis of the detection result from the front camera that has been acquired before detection of the malfunction and the detection result from the front radar that has been acquired after detection of the malfunction.

Description

Operating system, processing module, and method CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2024-071967 filed in Japan on April 25, 2024, the contents of which are incorporated by reference in their entirety.

[0002] The present disclosure relates to a technique for controlling vehicle driving.

[0003] Patent Document 1 describes that in an autonomous driving system, if a malfunction occurs in an ECU (Electronic Control Unit) that generates a trajectory, control of the actuator continues using trajectory data that was generated before the malfunction occurred.

[0004] Patent No. 6760977

[0005] Since traffic conditions may change after a malfunction occurs, the trajectory data generated before the malfunction is not necessarily the correct trajectory.

[0006] One of the objects of the present disclosure is to provide a driving system that can ensure safety even after a malfunction occurs in the driving system.

[0007] One of the driving systems included in the present disclosure is a driving system that automatically controls the speed and steering of a vehicle, and includes a first sensor that forms a detection range in the direction of vehicle travel, a second sensor whose detection range overlaps with part or all of the detection range of the first sensor, and a processing module that creates a driving plan for the vehicle based on output signals from the first sensor and the second sensor, and the processing module is configured to detect a malfunction of the first sensor, initiate a fail operation in response to the detection of the malfunction of the first sensor, and determine the control to be performed in the fail operation based on data obtained from the second sensor.

[0008] In addition, the processing module included in the present disclosure is a processing module that executes processing for automatically controlling the speed and steering of a vehicle, and is configured to receive detection results from a first sensor that forms a detection range in the vehicle's traveling direction, receive detection results from a second sensor whose detection range overlaps with part or all of the detection range of the first sensor, create a vehicle driving plan based on the detection results of the first sensor and the detection results of the second sensor, detect a malfunction of the first sensor, initiate a fail operation in response to the detection of the malfunction of the first sensor, and determine the control to be executed in the fail operation based on data obtained from the second sensor.

[0009] A method included in the present disclosure is a method for automatically controlling the speed and steering of a vehicle, including receiving detection results from a first sensor that forms a detection range in the direction of vehicle travel, receiving detection results from a second sensor whose detection range overlaps with part or all of the detection range of the first sensor, creating a vehicle driving plan based on the detection results of the first sensor and the detection results of the second sensor, detecting a malfunction of the first sensor, initiating a fail-operation in response to the detection of the malfunction of the first sensor, and determining control to be performed in the fail-operation based on data obtained from the second sensor.

[0010] According to the above technology, a fail operation is initiated in response to a malfunction of the first sensor. The specific content of the fail operation (e.g., an action) is determined based on the detection result of a second sensor that is not malfunctioning and whose detection range overlaps with that of the first sensor. This allows the fail operation to be controlled based on the traffic conditions detected by the second sensor after the malfunction has occurred. This makes it possible to ensure safety even after a malfunction occurs in the driving system.

[0011] Note that the symbols in parentheses in the claims indicate a correspondence with the specific means described in the embodiments described below as one aspect, and do not limit the technical scope of the present disclosure.

[0012] 1 is a diagram showing a vehicle equipped with a driving system. FIG. 2 is a diagram showing the hardware configuration of the driving system. FIG. 3 is a diagram showing an example of a specific hardware configuration of the driving system. FIG. 4 is a diagram showing the functional configuration of the driving system. FIG. 5 is a diagram showing the functional configuration of a risk confirmation unit. FIG. 6 is a flowchart for explaining normal operation of the processing system. FIG. 7 is a flowchart showing the operation of the processing system related to a sensor malfunction. FIG. 8 is a flowchart showing an example of operation of the processor depending on whether or not the vehicle is inside the LPR. FIG. 9 is a flowchart showing an example of operation of the processor taking into account the type of malfunction. FIG. 10 is a flowchart showing an example of operation of the processor. FIG. 11 is a flowchart showing an example of operation of the processor depending on whether or not there is an evacuation space in the LPR. FIG. 12 is a flowchart showing an example of operation of the processor relating to the exit determination of the ODD. FIG. 13 is a flowchart showing another example of operation of the processor taking into account the type of malfunction.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The present disclosure is not limited to the following embodiments. The configurations disclosed below may be modified in various ways without departing from the spirit of the present disclosure. Various modified examples may be appropriately combined as long as no technical contradictions arise. The present disclosure also includes configurations that are not explicitly stated and are formed by combining multiple modified examples. In the following description, components having the same function may be given the same reference numerals, and specific descriptions thereof may be omitted. Furthermore, components having the same function may be given the same or similar names, and specific descriptions thereof may be omitted. When only a portion of a configuration is mentioned, descriptions given elsewhere may apply to other parts.

[0014] (Explanation of Terms) Terms related to the disclosure of this specification are explained below. This explanation is included in the embodiments of the specification.

[0015] A road user may be a traffic participant on or adjacent to an active road for the purpose of traveling from one location to another. Road users may include pedestrians, cyclists, vehicles, and other vulnerable road users. A pedestrian may be a person walking on a sidewalk adjacent to a road. A cyclist may be a person riding a bicycle. A vehicle may be a passenger car, commercial vehicle, bus, etc. A vehicle may be a manually or autonomous vehicle.

[0016] A vulnerable road user (VRU) may be a road user not in a vehicle, such as a passenger car, public transport, train, etc. A vulnerable road user may also include unprotected road users, such as cyclists, motorcyclists, pedestrians, people with disabilities, or people with reduced mobility and orientation.

[0017] The dynamic driving task (DDT) may be the real-time operational and tactical functions for operating a vehicle in traffic. The DDT may also be all real-time operational and tactical functions for operating a vehicle on a roadway. Operational functions may include lateral vehicle motion control through steering and longitudinal vehicle motion control through acceleration and deceleration. Tactical functions may include object or event detection and response. Response to a detected object / event may include planning and execution for avoidance, etc.

[0018] An operational design domain (ODD) may be the specific conditions in which a given automated driving system is designed to function, and may include, but is not limited to, the operating conditions in which a given automated driving system or its features are specifically designed to function, including, but not limited to, environmental, geographic, time-of-day restrictions, and / or the presence or absence of requirements for certain traffic / road characteristics.

[0019] An automated driving system (ADS) may be hardware and software capable of persistently performing the entire dynamic driving task, whether or not limited to a specific operational design domain. An ADS feature may be the design-specific functionality of an automated driving system within a specific ODD at a given automation level.

[0020] A fault may be an abnormal condition that causes the ADS to malfunction. A failure may be the termination of the intended operation of the ADS due to the manifestation of a fault.

[0021] A minimal risk condition (MRC) may be a vehicle state intended to reduce the risk of failure to complete a trip. Alternatively, an MRC may be a stable, stationary state that a user or automated driving system places the vehicle in after performing a DDT fallback to reduce the risk of an accident if a given trip cannot or should not continue. The term MRC may be replaced with a minimal risk condition.

[0022] A DDT fallback may be a response by a user to perform a dynamic driving task (DDT) or achieve MRC after a DDT performance-related system failure or an ODD deviation occurs, or a response by an automated driving system to achieve MRC given the same circumstances. A DDT fallback may be a response by a driver or an automated system to perform a DDT or transition to MRC after a failure occurs or a malfunction is detected, or upon detection of potentially dangerous behavior. A DDT fallback may be a takeover / fallback condition and scheme for transferring control from an automated driving system to a driver or another system in each use case involving the automated driving system or a driver.

[0023] A minimal risk maneuver (MRM) may be a vehicle movement commanded by an automated driving system during DDT fallback to achieve the MRC. An MRM may also be the operation (control) of a vehicle by an automated driving system to achieve the MRC. The term MRM may be replaced with minimum risk maneuver or risk minimization control.

[0024] Safety of the intended functionality (SOTIF) may be the absence of undue risk due to insufficient functionality of the intended functionality or its implementation.

[0025] A driving policy may be a strategy and rules that define control behavior at the vehicle level.

[0026] A scenario may be a description of the temporal relationships between several scenes in a sequence of scenes, including the goals and values ​​in a specific situation influenced by actions and events, and a description of a continuous time sequence of activities that integrates a subject vehicle, all its external environments, and their interactions in the process of performing a specific driving task.

[0027] A safety-relevant object may be any dynamic or static object (excluding the ego vehicle) that may be relevant to the safety performance of the dynamic driving task.

[0028] Reasonably foreseeable may be technically reliable and have a reliable or measurable rate of occurrence.

[0029] A triggering condition may be a specific condition in a scenario that contributes to either unsafe behavior or the inability to prevent or detect and mitigate reasonably foreseeable indirect misuse, which initiates a subsequent system response.

[0030] A safety-related model may be a representation of safety-related aspects of driving behavior based on assumptions about the reasonably foreseeable behavior of other road users. A safety-related model may include on-board or off-board safety verification or analysis devices. A safety-related model may be a mathematical model, a more conceptual set of rules, a set of scenario-based behaviors, or a combination of these.

[0031] A formal model may be a model expressed in a formal notation that is used for formal verification of system performance.

[0032] A safety envelope may be a set of limits and conditions within which a driving system is designed to operate, assuming constraints or controls to maintain operation within an acceptable risk level. A safety envelope may also be a common concept that can be used to address all principles that a driving policy can adhere to. According to the concept of a safety envelope, an ADS-operated vehicle may have one or more boundaries around the vehicle. In some scenarios, violation of one or more of these boundaries may result in different responses by the ADS-operated vehicle.

[0033] Response time may be the time it takes a road user in a given scenario to perceive a particular stimulus and begin to execute a response (braking, steering, accelerating, stopping, etc.).

[0034] Risk acceptance criteria / criterion are standards that represent the absence of unreasonable levels of risk, and may be, for example, physical parameters that define when a particular behavior is considered undesirable, a maximum number of accidents per hour, as low as reasonably practicable, etc.

[0035] A positive risk balance may be a criterion that demonstrates that a technical solution achieves an acceptable level of residual risk.

[0036] A proper response may be an action that is significant to avoid or ameliorate a dangerous situation in a reasonably foreseeable scenario in which other safety-related objects are operating within expected bounds.

[0037] <Driving System> The driving system 9 of this embodiment is a system that realizes functions related to driving the vehicle 1. The driving system 9 may be a vehicle system itself, or may be a component that constitutes part of a vehicle system. Part or all of the driving system 9 may be mounted on the vehicle 1 as shown in FIG. 1 . The vehicle 1 may be referred to as a host vehicle, a host vehicle, or the like. The vehicle 1 may be configured to be capable of wirelessly communicating directly with a roadside device 92. The vehicle 1 may be configured to be capable of communicating with other road users, such as a following vehicle 93, directly or indirectly via a communication infrastructure.

[0038] The vehicle 1 may be a road user capable of manual driving, such as a four-wheeled car or truck. The vehicle 1 may also be capable of automated driving. Autonomous driving may also be referred to as autonomous driving by a driving system 9 or autonomous traveling. Autonomous driving may be one aspect of technology for automatically controlling the speed and steering of the vehicle 1. Driving is classified into levels according to the extent to which a human driver performs all dynamic driving tasks (DDTs). There may be six automation levels, from 0 to 5, as defined in SAE J3016.

[0039] At levels 0 to 2, the driver performs some or all of the DDT. Levels 0 to 2 may be classified as so-called manual driving. Level 0 means that driving is not automated. Level 1 means that the driving system 9 assists the driver. Level 2 means that driving is partially automated. Level 2 may be divided into levels 2.0 and 2.5. Level 2.0 is a level where the system provides partial steering assistance and the driver essentially performs steering. Level 2.5 is a level where the driver must monitor the surroundings, but the driving system 9 essentially performs steering. In the present disclosure, vehicle control corresponding to level 2.5 is also referred to as automated driving with a surrounding monitoring obligation or semi-automated driving. Semi-automated driving may also be one aspect of technology for automatically controlling the speed and steering of a vehicle. The process for automatically controlling speed and steering is not limited to processes corresponding to levels 3 or higher, but may include processes corresponding to level 2.5.

[0040] At levels 3 and above, while the ADS feature is activated, the driving system 9 performs all of the DDT. Levels 3 to 5 may be classified as so-called automated driving. A system capable of driving at level 3 or above may be called an automated driving system (ADS). A vehicle equipped with an automated driving system or a vehicle capable of driving at level 3 or above may be called an automated vehicle (AV).

[0041] Level 3 indicates a state in which driving is conditionally automated. A level 3 automated driving system performs DDT but does not perform DDT fallback. That is, at level 3, DDT fallback is performed by a driver who is ready for fallback. Level 4 indicates a state in which driving is highly automated. A level 4 automated driving system performs DDT and DDT fallback. A level 4 automated driving system can hand over DDT to the driver after reaching a minimal risk condition (MRC) by performing DDT fallback, etc. Taking over DDT between the driving system 9 and a human driver is also called delegation of authority. Level 5 indicates fully automated driving.

[0042] The conditions for executing level 3 and 4 autonomous driving may include some or all of the conditions indicated by the operational design domain (ODD). The ADS function may be defined within the scope of the ODD. The driving system 9 described in this embodiment is a driving system capable of executing level 3 or higher autonomous driving. That is, the driving system 9 may be capable of executing up to level 3 autonomous driving, up to level 4 autonomous driving, or even level 5 autonomous driving. The function for implementing level 3 or higher autonomous driving is referred to as an autonomous driving function. The autonomous driving function may be positioned as one of the applications provided by the driving system 9.

[0043] The driving system 9 provides functions such as automated driving to a vehicle user of the vehicle 1 that can participate in public road traffic. The vehicle user may be a driver riding in the vehicle 1. The vehicle user may be a passenger riding in the vehicle 1. If the vehicle 1 is a POV (Personally Owned Vehicle), the vehicle user may be the owner of the vehicle 1. If the vehicle 1 is used for MaaS (Mobility as a Service), the vehicle user may be an operator such as an operations manager that manages the operation of the vehicle 1.

[0044] The architecture of the driving system 9 may be selected to enable an efficient safety of the intended functionality (SOTIF) process. The architecture of the driving system 9 may be configured based on a sense-plan-act model. The sense-plan-act model includes a sense element, a plan element, and an act element as major system elements. The sense element, plan element, and act element interact with each other. Here, sense may be replaced by perception, plan may be replaced by determine, and act may be replaced by control, respectively.

[0045] At the technical level (i.e., from a technical perspective), the driving system 9 is implemented with a plurality of sensors 40 corresponding to the sensing function, one or more processing systems 50 corresponding to the planning function, and a plurality of motion actuators 60 corresponding to the action function. At the functional level (i.e., from a functional perspective), the sensing function, the planning function, and the action function are implemented (see also Figures 4 and 5).

[0046] In detail, a detection unit 10 as an entity realizing a detection function may be constructed in the driving system 9. The detection unit 10 may be constructed mainly by a plurality of sensors 40 and a processing system 50. The processing system 50 related to the detection function may be configured to process detection information from the sensors 40 and generate an environmental model based on the detection information. A planner 20 and a risk confirmation unit 26 may be constructed in the driving system 9 mainly by using the processing system 50. The planner 20 is an entity realizing the planning function. Furthermore, the processing system 50 may be capable of outputting control signals (e.g., drive signals) for a plurality of motion actuators 60. A behavior unit 30 as an entity realizing a behavior function may be constructed in the driving system 9. The behavior unit 30 may be constructed mainly by using the processing system 50 and a plurality of motion actuators 60.

[0047] Here, the detection unit 10 may be realized in the form of a detection system serving as a subsystem provided so as to be distinguishable from the planner 20 and the behavior unit 30. The planner 20 may be realized in the form of a planning system serving as a subsystem provided so as to be distinguishable from the detection unit 10 and the behavior unit 30. The planning system may include a risk confirmation function. The risk confirmation function may be mounted on the operation system 9 independently of the detection unit 10, the planner 20, and the behavior unit 30. The behavior unit 30 may be realized in the form of a behavior system serving as a subsystem provided so as to be distinguishable from the detection unit 10 and the planner 20. The detection system, the planning system, and the behavior system may constitute components independent of each other. The subsystem referred to here may be replaced with a module, a unit, a device, a component, etc.

[0048] The detection unit 10 is responsible for detection functions, including localization (e.g., location estimation) of road users such as the vehicle 1 and other vehicles. The detection unit 10 detects the external environment, internal environment, vehicle state, and even the state of the driving system 9 of the vehicle 1. The detection unit 10 may fuse the detected information to generate an environmental model. The environmental model may also be referred to as a world model. The planner 20 applies the objective and driving policy to the environmental model generated by the detection unit 10 to derive control actions. The behavior unit 30 executes the control actions derived by the planner 20.

[0049] The driving system 9 may be optimized, in part or in whole, as appropriate, depending on the laws or customs of the region in which the vehicle 1 is used. The following description will be given assuming that the driving system 9 is used in a region where traffic drives on the left. Based on this assumption, the leftmost lane in the direction of travel will be referred to as the first lane, and the adjacent lane to the right will be referred to as the second lane. When the vehicle 1 equipped with the driving system 9 is used in a region where traffic drives on the right, the first lane below may be interpreted as the rightmost lane, and the second lane as the lane adjacent to the first lane on the left. The lane in which the vehicle 1 is traveling will also be referred to as the ego lane below.

[0050] <Physical Architecture> An example of the physical architecture of the driving system 9 will be described with reference to FIG. 2 . The driving system 9 includes a plurality of sensors 40, a plurality of motion actuators 60, a plurality of secondary actuators 65, a plurality of HMI devices 70, and a processing system 50. HMI stands for Human Machine Interface. These components can communicate with each other via one or both of wireless and wired connections. These components may also be able to communicate with each other through an in-vehicle network such as CAN (registered trademark) or Ethernet (registered trademark). Communication between devices may be achieved by any type of communication, including wired and wireless.

[0051] The multiple sensors 40 include one or more external environment sensors 41. Furthermore, the multiple sensors 40 may include one or more internal environment sensors 42. Furthermore, the multiple sensors 40 may include one or more communication systems 43. Furthermore, the multiple sensors 40 may include a map database (DB) 44. The combination of devices included in the multiple sensors 40 may be designed as appropriate. In the present disclosure, data indicating the environment outside or inside the vehicle that is sensed (or acquired) by the sensor 40 is also referred to as sensor data.

[0052] The external environment sensor 41 may include one or more object detection type sensors. The object detection type external environment sensor 41 is a sensor that detects objects present in the external environment of the vehicle 1. The object detection type external environment sensor 41 is, for example, a camera, LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging), laser radar, millimeter-wave radar, sonar, acoustic sensor, etc. The driving system 9 may be implemented with a combination of multiple types of external environment sensors 41 to monitor the front, sides, and rear directions of the vehicle 1. In the present disclosure, the detection range of the object detection type external environment sensor 41 may also be referred to as the field of view (FOV).

[0053] Furthermore, the external environment sensor 41 may include a condition detection type sensor. The condition detection type external environment sensor 41 is a device that outputs data related to the atmospheric condition (e.g., temperature), road surface condition, or brightness in the external environment of the vehicle 1. The atmospheric condition may include weather conditions (e.g., rain). The atmospheric condition may include the presence or absence of fog (concentration), wind speed, etc. The condition detection type external environment sensor 41 may include at least one of an outside air temperature sensor, a temperature sensor, and a raindrop sensor.

[0054] The internal environment sensor 42 may include a motion physical quantity detection type sensor. The motion physical quantity detection type internal environment sensor 42 is a sensor that detects a specific physical quantity related to the motion of the vehicle 1 (hereinafter referred to as a motion physical quantity). The motion physical quantity detection type internal environment sensor 42 may include at least one of a speed sensor, an acceleration sensor, a gyro sensor, etc. The internal environment sensor 42 may detect the state of an occupant of the vehicle 1. The internal environment sensor 42 may include an occupant detection type sensor. The occupant detection type internal environment sensor 42 may include at least one of an actuator sensor, an in-vehicle monitor, a biological sensor, a seat sensor, an in-vehicle equipment sensor, etc. The in-vehicle monitor here may be a sensor or system that monitors a vehicle user (e.g., a driver) inside the vehicle cabin. The actuator sensor is a sensor that detects the state of an occupant's operation of a motion actuator 60 related to motion control of the vehicle 1. The actuator sensor may include at least one of an accelerator sensor, a brake sensor, a steering sensor, etc.

[0055] The communication system 43 obtains communication data usable in the driving system 9 from an external system via wireless communication. The external system means any other system existing in the external environment of the vehicle 1. The communication system 43 may receive positioning signals from artificial satellites of a global navigation satellite system (GNSS) existing in the external environment of the vehicle 1. The positioning type communication device in the communication system 43 may be a GNSS receiver or the like.

[0056] The communication system 43 may transmit and receive communication signals to and from an external system such as a server 96. The V2X-type communication device in the communication system 43 may be a dedicated short range communications (DSRC) communication device, a cellular V2X (C-V2X) communication device, or the like. Examples of communication with the V2X system include communication with a communication system of another vehicle (V2V), communication with a roadside device 92 (V2I), communication with a pedestrian's mobile terminal (V2P), and communication with a network such as a cloud server (V2N). The roadside device 92 may be infrastructure equipment such as a communication device installed in a traffic light. The architecture of V2X communication, including V2I communication, may be an architecture specified in ISO 21217, ETSI TS 102 940-943, IEEE 1609, or the like.

[0057] The communication system 43 may receive vehicle status messages from other vehicles. The vehicle status messages may include the speed of the sender, the current location, the operation status of the turn signals, the acceleration, etc. The vehicle status messages may include at least one of the shift position, the on / off status of the brake pedal, the on / off status of the accelerator pedal, and the steering angle. The vehicle status messages may be CAM (Cooperative Awareness Message) or BSM (Basic Safety Message). Data received by the wireless communication device 15 may also be included in the sensor data.

[0058] Furthermore, the communication system 43 may transmit and receive communication signals to and from a mobile terminal 91. The mobile terminal 91 may be a smartphone, a wearable device, a tablet, or the like present in the vehicle. The mobile terminal 91 may be a smartphone or the like carried by the vehicle user. A terminal communication type communication device in the communication system 43 may be a Bluetooth (registered trademark) device, a Wi-Fi (registered trademark) device, an infrared communication device, or the like.

[0059] The map DB 44 is a database that stores map data that can be used by the driving system 9. The map DB 44 is configured using at least one type of storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The map DB 44 may include a database of high-definition (HD) maps with a high level of accuracy that are primarily used in automated driving systems. The map DB 44 may also include a database of parking lot maps that include detailed parking lot information, such as parking space information, that is used in automated parking or parking assistance. The map DB 44 may also include a database of a navigation unit that navigates the vehicle 1 along a driving route to a destination. The map DB 44 may also store a navigation map that indicates road connections. The navigation map may be used to search for a route from the current location to a destination. The map DB 44 may also include a PD map generated using probe data (PD) collected from each vehicle.

[0060] The map DB 44 suitable for the driving system 9 may acquire and store the latest map data by communicating with a map server via the communication system 43, for example. The map data is data representing the external environment of the vehicle 1, and is converted into two-dimensional or three-dimensional data. Such map data may include road data representing at least one of the position coordinates, shape, road surface condition, and standard running path of a road structure. The map data may also include marking data representing the position coordinates and / or shape of features attached to the road, such as road signs, road markings, and dividing lines. The marking data included in the map data may represent traffic signs, arrow markings, lane markings, stop lines, directional signs, landmark beacons, business signs, line pattern changes, etc. The map data may also include structure data representing at least one of the position coordinates and shapes of buildings and traffic lights facing the road. The marking data included in the map data may represent street lights, road edges, reflectors, poles, etc.

[0061] The motion actuator 60 is a device / mechanism capable of controlling vehicle motion based on an input control signal. The drive-related motion actuator 60 is a power train including at least one of an internal combustion engine and a drive motor. The drive-related motion actuator 60 may be called a propulsion device. The braking-related motion actuator 60 may be a brake actuator. The braking-related motion actuator 60 may be called a braking device. The steering-related motion actuator 60 may be a steering actuator. The steering-related motion actuator 60 may be called a steering device. Because the motion actuator 60 is an actuator that directly affects the movement of the vehicle 1, it is also referred to as a primary actuator in the present disclosure.

[0062] The secondary actuators 65 are actuators that do not directly affect the movement of the vehicle 1, but which enable safe and legal driving conditions. The secondary actuators 65 may include at least one of indicator lights, headlights, fog lights, hazard lights, a windshield wiper motor, and a rear windshield wiper motor. The indicator lights may include turn signals, brake lights, width indicators, etc.

[0063] The HMI device 70 is a device that realizes human-machine interaction, which is interaction between the user of the vehicle 1 and the driving system 9. The driving system 9 may include multiple HMI devices 70. A portion of the multiple HMI devices 70 that realizes an operation input function by an occupant may be considered to be part of the detection unit 10. A portion of the multiple HMI devices 70 that realizes an information presentation function may be considered to be part of the behavior unit 30. On the other hand, the function realized by the HMI device 70 may be positioned as a function independent of the detection function, the planning function, and the behavior function.

[0064] The HMI device 70 may include an operation input device 70a that can input user operations to transmit the will or intention of the user of the vehicle 1 to the driving system 9. The operation input device 70a may include at least one of an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a turn signal lever, a mechanical switch, a touch panel, etc. The operation input device 70a outputs an operation signal, which is a signal corresponding to an operation by the vehicle user, to the processing system 50.

[0065] The HMI device 70 may include one or more information presentation devices 70b for presenting information to a user of the vehicle 1. The information presentation device 70b may be a device that presents visual information, auditory information, tactile information, or the like. The HMI device 70 may include a visual-type information presentation device 70b, an auditory-type information presentation device 70b, a tactile-type information presentation device 70b, or a combination thereof. The visual-information presentation type HMI device 70 may be, for example, a meter display, a navigation unit, a center information display (CID), a head-up display (HUD), an illumination unit, or the like.

[0066] The auditory information presentation type HMI device 70 may be a speaker, a buzzer, etc. The tactile information presentation type HMI device 70 may be a steering wheel vibration unit, a driver's seat vibration unit, a steering wheel reaction force unit, an accelerator pedal reaction force unit, a brake pedal reaction force unit, an air conditioning unit, etc.

[0067] Furthermore, the HMI device 70 may realize an HMI function linked to a mobile terminal 91 such as a smartphone by mutually communicating with the terminal through the communication system 43. The vehicle user's mobile terminal 91 may be an additional or alternative information presentation device 70b. The driving system 9 may display information of the driving system 9 on the screen of the mobile terminal 91 through the communication system 43. Conversely, the HMI device 70 may present information acquired from the mobile terminal 91 to the vehicle user. The mobile terminal 91 may be used as an additional or alternative operation input device 70a.

[0068] Furthermore, the HMI device 70 may include an external HMI device that presents visual or auditory information to other road users in the external environment of the vehicle 1. The external HMI device is, for example, a turn signal lamp, a hazard lamp, an external display, a speaker, etc. The external display is a display whose display surface faces the outside of the vehicle 1. The external display may be provided on the rear window, a side window, or the side of the vehicle body. Some external HMI devices, such as the turn signal lamp and the hazard lamp, may be positioned in the secondary actuator 65.

[0069] The processing system 50 is a system that integrally executes processing related to the detection function, processing related to the planning function, and processing related to the action function. The processing system 50 may include at least one processing unit corresponding to processing related to the detection function, at least one processing unit corresponding to processing related to the planning function, and at least one processing unit corresponding to processing related to the action function. The processing system 50 may further execute processing related to the HMI device 70. A processing system dedicated to the HMI may be provided separately from the processing system 50. The processing system dedicated to the HMI may be an integrated cockpit system that integrally executes processing related to each HMI device 70, or an HCU (HMI control unit). The processing system 50 may be provided by an in-vehicle platform that can be generally used in AVs. The specific configuration of the processing system 50 will be described below.

[0070] <Processing System> The processing system 50 has an external communication interface 51, a main unit 52, a monitoring unit 53, a recording unit 54, and a software management unit. The external communication interface 51 is a communication interface for communicating with external devices. The external devices here are mainly other devices that constitute the operation system 9. The external devices may include a mobile terminal 91 and a server 96.

[0071] The external communication interface 51 is connected to at least one component related to processing by the processing system 50 via at least one of, for example, a local area network (LAN), a wire harness, an internal bus, and a wireless communication circuit. The at least one component connected to the external communication interface 51 may be at least one component among a variety of components, such as the sensor 40, the motion actuator 60, and the HMI device 70. The external communication interface 51 may include at least one of a circuit for wired communication and a circuit for wireless communication.

[0072] The main unit 52 is one or more dedicated computers for realizing functions such as a detection function, a planning function, and an action function. The processing system 50 may realize functions such as a detection function, a planning function, and an action function by using the main unit 52. One of the one or more dedicated computers constituting the main unit 52 may be an integration ECU that integrates the driving functions of the vehicle 1. The main unit 52 may include a determination ECU that determines DDT. The main unit 52 may include a monitoring ECU that monitors the driving of the vehicle 1. The main unit 52 may include an evaluation ECU that evaluates the driving of the vehicle 1. The main unit 52 may include a navigation ECU that navigates the driving route of the vehicle 1.

[0073] The dedicated computer constituting the main unit 52 may be a locator ECU that estimates the position of the vehicle 1. The dedicated computer may be an image processing ECU that processes image data detected by the external environment sensor 41. The dedicated computer may be an actuator ECU that controls the motion actuators 60 of the vehicle 1. The dedicated computer may be an HCU (HMI Control Unit) that comprehensively controls the HMI device 70. The one or more dedicated computers constituting the main unit 52 may include at least one external computer provided in an external center or mobile terminal 91 that can communicate via the communication system 43.

[0074] The dedicated computer constituting the main unit 52 may include a memory 52a, a processor 52b, and a communication interface 52c. The memory 52a is a storage medium that non-temporarily stores computer programs and data that can be read by the processor 52b. The memory 52a may include at least one type of storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The memory 52a may also include a rewritable volatile storage medium, such as a random access memory (RAM). The programs stored in the memory 52a may be programs for implementing at least some of the functions of the main unit 52 shown as blocks in FIG. 4. The processor 52b may include at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), a data flow processor (DFP), or a reduced instruction set computer (RISC)-CPU.

[0075] The dedicated computer constituting the main unit 52 may be a system on a chip (SoC) in which the memory 52 a, the processor 52 b, and the interface are integrated into a single chip. The dedicated computer may be configured using at least one SoC.

[0076] The communication interface 52c is hardware that allows the dedicated computer that constitutes the main unit 52 to communicate with other elements that constitute the processing system 50. The communication interface 52c may include a circuit that is compatible with a communication method with other devices / circuits. The communication interface 52c may be a so-called input / output circuit or input / output port. The communication interface 52c may support any type of wired or wireless communication.

[0077] A part or all of the external communication interface 51 may be included in the communication interface 52c. The communication interface 52c, the external communication interface 51, or both correspond to communication circuits for the processor 52b. The monitoring unit 53, the recording unit 54, and the software management unit 55, which will be described later, may also each have circuits corresponding to the communication interface 52c. A plurality of units may be configured to share the communication interface 52c. In the present disclosure, the communication interface 52c and the external communication interface 51 may also be collectively referred to as a communication IF (or communication I / F).

[0078] The monitoring unit 53 may be one aspect of on-board implementation of RSS (Responsibility Sensitive Safety) as a safety model. The monitoring unit 53 may be an on-board checker for the planning function realized by a dedicated computer. The monitoring unit 53 realizes the risk confirmation section 26, which realizes the risk confirmation function, by hardware independent of the planning section 20.

[0079] The monitoring unit 53 may be configured mainly with a dedicated computer having a memory 53 a and a processor 53 b. The dedicated computer constituting the monitoring unit 53 may be an SoC in which the memory 53 a, the processor 53 b, and the interface are integrated into a single chip.

[0080] The recording unit 54 is a device that records at least one of detection information, planning information, and action information. The recording unit 54 sequentially records event data related to the driving task of the vehicle 1. The event data is data that records events encountered by the vehicle 1. The event data may include at least one type of information related to the driving task, such as (1) information related to the operation of the motion actuators 60, (2) information related to the route or trajectory traversed or planned by the vehicle 1, (3) information related to the scenario encountered by the vehicle 1, (4) information related to the automation level or delegation of authority of the vehicle 1, and (5) information related to the execution of the DDT fallback or MRM of the vehicle 1.

[0081] The recording unit 54 may include one or more large-capacity storage media 54c. The storage media 54c may include at least one type of storage medium selected from the group consisting of semiconductor memory, magnetic media, and optical media. The storage media 54c may be mounted on the board in a form that is not easily detachable or replaceable. The storage medium 54c may be an embedded multi-media card (eMMC) using flash memory, or the like. At least one of the multiple storage media 54c may be detachable and replaceable from the recording unit 54. The storage medium 54c may be, for example, an SD card.

[0082] At least one of the recording unit 54 and the storage medium 54c may correspond to an Event Data Recorder (EDR) or a Data Storage System for Automated Driving (DSSAD). The recording unit 54 may have a function for selecting information to be recorded from the event data. In this case, the recording unit 54 may have a recording computer as a dedicated computer.

[0083] The recording computer may also have one memory and a processor. The recording unit 54 may access the storage medium 54c and perform recording in accordance with a data write command from each part of the driving system 9. The recording unit 54 may determine information transmitted over the in-vehicle network, and access the storage medium 54c and perform recording based on the judgment of the processor provided in the recording unit 54.

[0084] Such a recording unit 54 may not be provided in the processing system 50, but may be provided independently in the operation system 9. A part or all of the recording unit 54 may be provided in an external system present in the external environment, and configured to be accessible from the processing system 50 via the communication system 43. The recording unit 54 may be integrated with another unit such as the main unit 52.

[0085] The software management unit 55 is a device that realizes software management functions. The software management unit 55 manages various pieces of software used in the driving system 9, such as the main unit 52. The software here may be a computer program. The software may include parameters in the computer program used in the driving system 9. The software may include a trained model, sometimes referred to as AI, that is realized by, for example, a neural network, etc., used in the driving system 9.

[0086] The software management unit 55 may manage the software used by the software management unit 55 itself. The software management unit 55 may manage all software used in the vehicle 1. The software management may include software version management, download processing, installation processing, uninstallation processing, update processing, rollback processing, etc. Furthermore, the software management may include software testing.

[0087] The software management unit 55 may be configured using a dedicated computer having a memory 55a and a processor 55b to realize the software management function. The memory 55a may include a storage medium that non-temporarily stores computer programs and data that can be read by the processor 55b. The memory 55a may also be provided with a rewritable volatile storage medium such as RAM. "SW" in Figure 2 represents software.

[0088] Furthermore, the processing system 50 may include at least one database for executing the DDT, which may include at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, and an interface for the main unit 52 or the like to access the storage medium.

[0089] The database for executing the DDT may be a scenario database (hereinafter referred to as a scenario DB) 59. The database may be a rule database (hereinafter referred to as a rule DB) 58. At least one of the scenario DB 59 and the rule DB 58 may be configured integrally with the main unit 52. At least one of the scenario DB 59 and the rule DB 58 may not be provided in the processing system 50, but may be provided independently in the operation system 9. At least one of the scenario DB 59 and the rule DB 58 may be provided in an external system present in the external environment, and configured to be accessible from the processing system 50 via the communication system 43.

[0090] The scenario DB 59 has a scenario catalog in which multiple scenarios used in driving the vehicle 1 are stored. Each of the multiple scenarios is assigned a unique scenario ID. The multiple scenarios included in the catalog may include a scenario in which the vehicle travels near a vehicle (e.g., a bus) that is stopped on the road and has the potential to depart. The multiple scenarios may also include a scenario in which the vehicle travels next to another road user, a scenario in which the vehicle travels behind another road user, a scenario in which the path of the vehicle intersects with a VRU crossing the road, and the like. The driving system 9 can apply a situation in which the vehicle 1 is in one scenario or a combination of multiple scenarios selected from the multiple scenarios.

[0091] The scenario DB 59 may store multiple scenarios including at least one of functional scenarios, logical scenarios, and concrete scenarios. A functional scenario defines a top-level qualitative scenario structure. A logical scenario is a scenario in which a quantitative parameter range is assigned to a structured functional scenario. A concrete scenario defines a boundary of safety determination that distinguishes between a safe state and an unsafe state.

[0092] The rule DB 58 stores a rule set used for driving the vehicle 1. The rule set may include multiple rules. The rule set may further include a priority structure for the rules, which is set based on the relative importance of the multiple rules. The rule set may be an implementation of guidelines for strategic driving of the vehicle 1.

[0093] The multiple rules may include rules based on laws, regulations, or a combination thereof. The multiple rules may include rules based on preferences that are not influenced by laws, regulations, or the like. The multiple rules may include rules based on exercise behavior based on past experience. The multiple rules may include rules based on characterization of the exercise environment. The multiple rules may include rules based on ethical concerns. The multiple rules may include rules based on basic principles of a safety model (e.g., the five principles of the RSS model). The multiple rules may include traffic rules. The traffic rules may be rules specified in the Road Traffic Act or rules based on national or local customs. Rules such as traffic rules stored in the rule DB 58 may be mapped to information provided by the detection unit 10 to the planner 20 by the detection function, as well as map information obtained from the map DB 44.

[0094] As described above, the processing system 50 includes one or more memories (e.g., memories 52a, 53a) that store software and one or more processors (e.g., processors 52b, 53b) that execute the software. The one or more processors execute the software stored in the memory to realize some or all of the functions related to autonomous driving or driving assistance. The processors 52b, 53b, etc. execute processing related to vehicle driving control, including autonomous driving control, based on signals received via the communication interface. The processing related to vehicle driving control may be at least a portion of the processing executed by the detection unit 10, the planning unit 20, the action unit 30, and the risk confirmation unit 26. A configuration including either one or both of the processors 52b, 53b corresponds to a processing unit. The processing unit may include a processor and a memory.

[0095] <Implementation Example of Driving System> An example of a specific implementation of the above-described driving system 9 is shown in Fig. 3. As shown in Fig. 3, the processing system 50 may include an integrated ECU 56 and a plurality of zone ECUs 57. In the example shown in Fig. 3, the plurality of zone ECUs 57 include a first zone ECU 57a, a second zone ECU 57b, and a third zone ECU 57c.

[0096] The integrated ECU 56 may be an ECU that includes some or all of the functions of the main unit 52, the monitoring unit 53, the recording unit 54, and the software management unit 55. The integrated ECU 56 may include a plurality of SOCs 56a, 56b, and 56c and a plurality of communication IFs 51a, 51b, 51c, 51d, and 51e.

[0097] The first SOC 56a is an SOC that handles most of the functions of the integrated ECU 56a. The first SOC 56a may be referred to as the main SOC. The second SOC 56b is a redundant SOC in case of a malfunction of the first SOC 56a. The second SOC 56b may have the same functions as the first SOC 56a, or may have only the fail-safe or fail-operation functions of the first SOC 56a. The third SOC 56c is an SOC that handles functions that the first SOC 56a cannot cover. The third SOC 56c may be interpreted as an SOC that corresponds to an expanded function. The third SOC 56c absorbs differences in functionality between vehicle models or grades, or accommodates future function additions. The aforementioned processor 52b may be a processor included in the first SOC 56a. The processor 53b may be a processor included in the third SOC 56c.

[0098] The communication IFs 51a to 51e are interfaces through which the integrated ECU 56 communicates with other devices. Each of the communication IFs 51a to 51e may be hardware having at least one of an input port and an output port. The communication IFs 51a to 51e may be considered a form of the external communication interface 51. The communication methods supported by each communication IF may be any method, such as SPI (Serial Peripheral Interface), I2C (Inter-Integrated Circuit), UART (Universal Asynchronous Receiver Transmitter), Controller Area Network (CAN is a registered trademark), Ethernet (registered trademark), or FlexRay (registered trademark). Some of the communication IFs may support wireless communication such as Bluetooth (registered trademark), Wi-Fi (registered trademark), or Zigbee (registered trademark).

[0099] The communication IF 51a is connected to, for example, the interior monitor 42a, receives signals from the interior monitor 42a, and provides the signals to the first SOC 56a, etc. The communication IF 51b is connected to multiple sub-cameras 41x. The sub-cameras 41x may be fisheye cameras distributed around the vehicle body to capture images of the vehicle's surroundings. The sub-cameras 41x are also referred to as periphery monitoring cameras, etc. The multiple sub-cameras 41x may be arranged at the front end of the vehicle 1 (near the emblem), the right side mirror, the left side mirror, the rear end of the vehicle 1, etc. The sub-cameras 41x may be used, for example, to generate a bird's-eye view image of the vehicle 1 and its surroundings viewed from above. The sub-cameras 41x may be considered a type of external environment sensor 41.

[0100] The communication IF 51c is connected to the HMI device 70. The communication IF 51d is connected to multiple zone ECUs 57 via communication cables. Communication between the communication IF 51d and the zone ECUs 57 may be, but is not limited to, Ethernet in terms of communication speed, etc. The communication IF 51e may be a circuit for accessing a CAN network built in the vehicle 1. The communication IF 51e is configured to be able to communicate with, for example, a vehicle speed sensor and a shift position sensor. Devices not shown in FIG. 3, such as the map DB 44 and the motion actuator 60, may also be connected directly or indirectly to the integrated ECU 56 via the communication IF 51e or another communication IF.

[0101] Each of the multiple zone ECUs 57 is an ECU that controls devices located in a specific assigned zone of the vehicle 1. The devices may include actuators, sensors, and subsystems. For example, the first zone ECU 57a may be a front zone ECU responsible for the area near the front end of the vehicle 1 (also referred to as the front zone). The second zone ECU 57b may be a side zone ECU responsible for the right and left zones (or the center zone) of the vehicle 1. The second zone ECU 57b may be divided into two ECUs, one for the right and one for the left. The third zone ECU 57c may be a tail zone ECU responsible for the area near the rear end of the vehicle 1 (the tail zone). The zones for the vehicle 1 may be divided into a right front zone, a left front zone, a right rear zone, and a left rear zone, corresponding to the four corners of the vehicle 1, rather than being divided into front, rear, left, and right zones.

[0102] The first zone ECU 57a is responsible for monitoring the area ahead of the vehicle 1 (including the diagonally forward area). Object detection type sensors 411, 412, and 413 may be connected to the first zone ECU 57a. The second zone ECU 57b is responsible for monitoring the right and left sides of the vehicle 1. Object detection type sensors 414 and 415 may be connected to the second zone ECU 57b. The third zone ECU 57c is responsible for monitoring the area behind the vehicle 1 (including the diagonally rearward area). Object detection type sensors 416 and 417 may be connected to the third zone ECU 57c.

[0103] The sensors 411 to 417 may all be external environment sensors 41. The sensors 411 to 413 may all be external environment sensors 41 (also called forward sensors) that form a detection range in front of the vehicle 1. The sensor 411 may be a camera that captures images of the front (a so-called forward camera). The sensor 412 may be a millimeter-wave radar (a so-called forward radar) whose optical axis is directed forward. The sensor 412 may be a LiDAR. The forward camera, the forward radar, and the LiDAR may be sensors whose detection ranges overlap with each other. In other words, when the forward camera is the first sensor, the LiDAR or the forward radar can be the second sensor. The first sensor may be any external environment sensor 41. The second sensor may be another external environment sensor 41 whose detection range overlaps with that of the first sensor.

[0104] The first zone ECU 57a may be connected to a front-side sensor, which is an external environment sensor 41 that forms a detection range in the front side (i.e., diagonally forward). The front side (i.e., diagonally forward) includes the right front and the left front. The right front may be interpreted as the direction from 1 o'clock to 2 o'clock in the clock position. The left front may be interpreted as the direction from 10 o'clock to 11 o'clock. The 12 o'clock direction corresponds to the front direction of the vehicle 1. The front-side sensor may be a camera or radar with an optical axis directed diagonally forward. The front-side sensor may include a sensor for the right front and a sensor for the left front. Hereinafter, the terms "front-side camera" and "front-side radar" refer to a camera and a millimeter-wave radar with an optical axis directed diagonally forward. The external environment sensor 41 may include at least one of a front-side camera and a front-side radar.

[0105] The sensor 414 connected to the second zone ECU 57b may be a sonar. Multiple sonars may be arranged in the vehicle 1. The sensor 415 may be a rear-side camera, which is a camera with an optical axis directed diagonally rearward (i.e., rearward and lateral). The diagonally rearward (i.e., rearward and lateral) includes the right rear and left rear. The right rear may be interpreted as the direction from 4 o'clock to 5 o'clock in clock positions. The left rear may be interpreted as the direction from 7 o'clock to 8 o'clock.

[0106] The sensor 416 connected to the third zone ECU 57c may be a camera with an optical axis facing rearward (a so-called rear camera). In this disclosure, a sensor that forms a detection range behind the vehicle 1 is also referred to as a rear sensor. The rear sensor may include a millimeter-wave radar. The sensor 417 may be a rear-side radar, which is a millimeter-wave radar with an optical axis facing diagonally rearward. The sensors 415 and 417 may each be a different type of sensor that forms a detection range diagonally rearward. The connection destinations of the sensors 415 and 417 can be changed as appropriate, and the sensor 415 may be connected to the third zone ECU 57c. Conversely, the sensor 417 may be connected to the second zone ECU 57b. The connection destinations of the sensors 411 to 417 may be designed taking into account the installation location, etc. The external environment sensor 41 may include at least one of a rear-side camera and a rear-side radar.

[0107] The first zone ECU 57a controls the operation of the sensors 411-413 based on commands from the integrated ECU 56. The first zone ECU 57a receives output signals from the sensors 411-413 and transmits the detection results of the sensors 411-413 and signals indicating their operating status (normal / abnormal) to the integrated ECU 56. The second zone ECU 57b also controls the operation of the sensors 414-415 based on commands from the integrated ECU 56. The second zone ECU 57b transmits the detection results of the sensors 414-415 and signals indicating their operating status (normal / abnormal) to the integrated ECU 56. The third zone ECU 57c controls the operation of the sensors 416-417 based on commands from the integrated ECU 56. The third zone ECU 57c transmits the detection results of the sensors 416-417 and signals indicating their operating status (normal / abnormal) to the integrated ECU 56.

[0108] 2 are omitted in Fig. 3. Fig. 3 is merely an example, and the specific implementation of the operation system 9 may be changed as appropriate.

[0109] 4 and 5 show an example of a logical architecture in the driving system 9. Here, the description will focus on the processing by a computer program executed during automated driving at level 3 or higher. However, the following description may also be applied, in part or in whole, to the operation of the driving system 9 when the automation level is 2 or lower.

[0110] The detection unit 10 may include an environment recognition unit 11, a self-location recognition unit 12, and an internal recognition unit 13 as functional modules corresponding to sub-functions obtained by further classifying the detection function. The environment recognition unit 11, the self-location recognition unit 12, and the internal recognition unit 13 may also be realized by the processor 52b executing a computer program.

[0111] The environment recognition unit 11 individually processes information (sometimes referred to as sensor data) acquired from each sensor 40, and recognizes the external environment including other road users, etc. For example, the processors 52b and 53b serving as the environment recognition unit 11 acquire environmental information based on information signals received from the external environment sensor 41 using a communication interface.

[0112] The environment recognition unit 11 may individually process sensor data related to the external environment detected by each external environment sensor 41. The sensor data may be sensor data provided by millimeter-wave radar, sonar, LiDAR, etc. The environment recognition unit 11 may generate relative position data including the direction, size, and distance of an object relative to the vehicle 1 from the raw data received from the external environment sensors 41.

[0113] The sensor data may be image data provided by a camera, LiDAR, or the like. The image data may be a video signal. The environment recognition unit 11 processes the image data and extracts objects reflected within the angle of view of the image. The object extraction may include estimating the direction, size, and distance of the object relative to the vehicle 1. The object extraction may also include classifying the object using semantic segmentation.

[0114] Furthermore, the environment recognition unit 11 processes information acquired through the V2X function of the communication system 43. The environment recognition unit 11 processes information acquired from the map DB 44.

[0115] The environment recognition unit 11 may be further divided into a plurality of sensor recognition units each optimized for one sensor group. When a sensor recognition unit is associated with recognizing information from one sensor group, the sensor recognition unit may fuse information from the one sensor group.

[0116] The self-location recognition unit 12 performs localization of the vehicle 1. The self-location recognition unit 12 acquires global position data of the vehicle 1 from the communication system 43 (e.g., a GNSS receiver). In addition, the self-location recognition unit 12 may acquire position information of objects extracted by the environment recognition unit 11. The self-location recognition unit 12 also acquires map information from the map DB 44. The self-location recognition unit 12 may integrate two or more types of information from among the global position data, object position information, map information, and other information to estimate the position of the vehicle 1 on the map. In the present disclosure, information indicating the position of the vehicle 1 on the map estimated by the self-location recognition unit 12 is also referred to as estimated information of the position on the map.

[0117] The internal recognition unit 13 processes sensor data detected by each internal environment sensor 42 to recognize the vehicle state. The vehicle state may include the state of the physical quantities of motion of the vehicle 1 detected by a speed sensor, an acceleration sensor, a gyro sensor, etc. The vehicle state may also include at least one of the user state, the user's operation state of the motion actuator 60, and the switch state of the HMI device 70.

[0118] The planner 20 may include a predictor 21, an operation planner 22, and a mode manager 23 as functional modules corresponding to sub-functions obtained by further classifying the planning function. The predictor 21, the operation planner 22, and the mode manager 23 may also be realized by the processors 52b and 53b executing computer programs.

[0119] The prediction unit 21 acquires information on the external environment recognized by the environment recognition unit 11 and the self-position recognition unit 12, the vehicle state recognized by the internal recognition unit 13, etc. The prediction unit 21 may interpret the environment based on the acquired information and estimate the current situation of the vehicle 1. The situation here may be an operational situation or may include the operational situation.

[0120] The prediction unit 21 may interpret the environment and predict the behavior of objects such as other road users. The objects may be safety-relevant objects. The behavior prediction may include at least one of predicting the object's speed, acceleration, and trajectory. The behavior prediction may be performed based on reasonably foreseeable assumptions. The behavior prediction may involve calculating a range of potential behaviors. The potential behavior range of the other road users means, for example, a range that the other road users can reach within a predetermined time. The potential behavior range of the other road users may be calculated from the other road users' current speeds (detected values ​​or expected values), directions, and reasonably foreseeable maximum accelerations. A design value according to the type of road users may be applied to the reasonably foreseeable maximum acceleration.

[0121] The information generated by the prediction unit 21 is also referred to as prediction information hereinafter. Furthermore, the prediction unit 21 may estimate the user's intention based on the predicted behavior, the predicted potential hazard, and the acquired vehicle state. Information indicating the estimation result of the user's intention is also referred to as user intention information in the present disclosure.

[0122] The driving planner 22 plans autonomous driving of the vehicle 1 based on at least one type of information, such as estimated information on a map position, forecast information, user intention information, and functional constraint information (described later). The driving planner 22 provides a route planning function, a behavior planning function, and a trajectory planning function. The route planning function is a function of planning at least one of a route to a destination and a medium-distance lane plan based on estimated information on a map position and destination information. The route planning function may further include a function of determining at least one of a lane change request and a deceleration request based on the medium-distance lane plan. Here, the route planning function may be a mission / route planning function in strategic functions and may be a function of outputting a mission plan and a route plan. Here, the strategic functions may be functions of deciding whether to operate, setting a route to a destination, and adjusting or selecting a rough operation schedule.

[0123] The behavior planning function is a function that plans the behavior of the vehicle 1 based on at least one of a route to a destination, a mid-distance lane plan, a lane change request, a deceleration request, prediction information, user intention information, and function constraint information. The behavior planning function may include a function that generates a condition related to a state transition of the vehicle 1. The condition related to a state transition of the vehicle 1 may be a triggering condition. The condition related to a state transition may include a fallback condition that is a condition for executing a DDT fallback. A fallback condition may be satisfied when a system failure related to dynamic driving task performance occurs, an exit from ODD (deviation), or a potentially dangerous behavior is detected or predicted. The failure may include the occurrence of a malfunction or a malfunction.

[0124] The behavior planning function may include a function for determining a state transition of an application that realizes DDT based on the condition, and further a function for determining a state transition of a driving action. The driving planner 22 may determine to execute a DDT fallback when a fallback condition is satisfied. Furthermore, if the DDT fallback does not involve authority delegation, the driving planner 22 may further execute a minimal risk maneuver (MRM) together with the motion control unit 31 to transition the vehicle 1 to a minimal risk condition (MRC). In many cases, the MRC may be a state in which the vehicle is stopped outside the lane (e.g., on the shoulder) or within the lane, but is not limited thereto. The MRC may be a state in which the vehicle is following a preceding vehicle, or a state in which the vehicle is continuing to travel at a constant speed with its hazard lights turned on. The MRC may be determined according to traffic conditions. The specific form of the MRM may be determined according to the MRC. The MRM plan may be created by the trajectory planning function instead of the behavior planning function. Information indicating the state transition of an application, which is determined by the operation plan unit 22, is also referred to as state transition information of the application.

[0125] The behavior planning function of the driving planner 22 may include a function for determining longitudinal constraints on the path of the vehicle 1 and lateral constraints on the path of the vehicle 1 based on this state transition information. The behavior planning function may be a tactical behavior plan in the DDT function, and may output tactical behavior. The longitudinal and lateral directions are determined based on the vehicle 1. The longitudinal direction may be the longitudinal direction of the vehicle 1. The lateral direction may be the width direction of the vehicle 1. Basically, the longitudinal direction coincides with the direction in which the road on which the vehicle 1 is traveling extends. Therefore, the longitudinal direction may be interpreted as the direction in which the road extends. The lateral direction may also be interpreted as the width direction of the road. In this way, the longitudinal and lateral directions may be defined based on the road on which the vehicle 1 is traveling.

[0126] The trajectory planning function is a function that plans a driving trajectory of the vehicle 1 based on prediction information, longitudinal constraints on the path, and lateral constraints on the path. The trajectory planning function may include a function that generates a path plan. The path plan may include a speed plan, or the speed plan may be generated as a plan independent of the path plan. The trajectory planning function may include a function that generates multiple path plans and selects an optimal path plan from the multiple path plans, or a function that switches between path plans. The trajectory planning function may further include a function that generates backup data for the generated path plan. The trajectory planning function may be a trajectory planning function in the DDT function and may output a trajectory plan. In the operation planning unit 22, the terms "path" and "trajectory" may be interchangeable. The operation planning unit 22 is configured to output trajectory planning information, which is information indicating the trajectory plan (in other words, the path plan), to the motion control unit 31. Generating a trajectory plan is also referred to as "planning" below.

[0127] The mode management unit 23 monitors the driving system 9 and sets restrictions on driving-related functions. The mode management unit 23 may manage the operating mode of the driving system 9, for example, the state of the automation level. The management of the automation level may include management of switching between manual driving and automated driving, i.e., management of the transfer of authority between the user and the driving system 9, in other words, management of the takeover of driving. The mode management unit 23 may determine and implement a change in the automation level based on an operation signal input from the operation input device 70a. When the automation level is switched to a state lower than level 2, the mode management unit 23 may control the enablement state of a driving assistance application corresponding to the automation level.

[0128] The mode management unit 23 may be capable of monitoring the state of a subsystem related to the operation system 9 and detecting a malfunction of the subsystem. The malfunction of the subsystem may include an error, an unstable operation state, a fault, a breakdown, etc. The malfunction may be referred to as a fault. The malfunction (fault) of the subsystem also includes a malfunction of the external environment sensor 41.

[0129] The mode management unit 23 may determine a mode that conforms to the user's intention based on the user intention information. The mode management unit 23 may set constraints on functions related to driving based on at least one of the system malfunction determination result, the mode determination result, the vehicle state, the sensor abnormality (or sensor failure) signal output from the sensor 40, the application state transition information, and the trajectory plan. The constraints on functions may include mode restrictions (e.g., prohibition of level 3 or higher). In the present disclosure, information indicating constraints on functions related to driving is also referred to as function constraint information. The function constraint information generated by the operation of the mode management unit 23 can be referenced by the driving planner 22.

[0130] Furthermore, the mode management unit 23 may have a comprehensive function of determining, in addition to constraints on driving functions, longitudinal constraints on the path of the vehicle 1 and lateral constraints on the path of the vehicle 1. In this case, the operation planning unit 22 plans behavior and trajectories in accordance with the constraints determined by the mode management unit 23.

[0131] When the risk confirmation function is implemented as part of the planner 20, the risk confirmation function may be implemented as part of the functions realized by the predictor 21, the operation planner 22, and the mode manager 23. On the other hand, the risk confirmation function may be implemented as a function independent of the planner 20 (see also FIG. 5 ).

[0132] The behavior unit 30 may include a motion control unit 31 and an HMI output unit 71 as functional modules corresponding to sub-functions that further classify the behavior functions. The motion control unit 31 and the HMI output unit 71 may each be realized by the processor 52b executing a computer program. The motion control unit 31 controls the motion of the vehicle 1 based on a trajectory plan provided from the driving plan unit 22. Specifically, the motion control unit 31 generates accelerator request information, shift request information, brake request information, and steering request information according to the trajectory plan, and outputs them to the motion actuator 60. The accelerator request information, shift request information, brake request information, and steering request information may function as control signals (in other words, control commands) for the motion actuator 60. The accelerator request information, shift request information, brake request information, and steering request information may be rephrased as an accelerator request signal, a shift request signal, a brake request signal, and a steering request signal.

[0133] Here, the motion control unit 31 can directly obtain the vehicle state recognized by the detection unit 10 (particularly the internal recognition unit 13), such as at least one of the current speed, acceleration, and yaw rate of the vehicle 1, from the detection unit 10 and reflect this in the motion control of the vehicle 1.

[0134] The HMI output unit 71 outputs information about the HMI based on at least one of prediction information, user intention information, application state transition information, trajectory planning information, and function constraint information. The HMI output unit 71 may manage vehicle interactions. The HMI output unit 71 may generate an information presentation request based on the management status of the vehicle interactions and control the information presentation function of the HMI device 70. Furthermore, the HMI output unit 71 may generate control requests for wipers, a sensor washing device, headlights, and an air conditioning device based on the management status of the vehicle interactions and control these devices.

[0135] Next, the risk confirmation function will be described in detail. An example in which the risk confirmation unit 26 is implemented independently of the planner 20 as shown in FIG. 5 will be described below. Such a risk confirmation unit 26 may be realized by the processor 53b executing a computer program. In other embodiments, the risk confirmation function may be implemented as part of the predictor 21, the operation planner 22, or the mode manager 23.

[0136] The risk confirmation function is realized by implementing a safety model. The safety model is a model for demonstrating that there is no unacceptable risk within a specific ODD. The safety model may correspond to at least one of a safety driving model, a safety-related model, and a formal model. The safety model may be, for example, an RSS model. In other embodiments, the safety model may be another model such as a Safety Force Field (SFF) model, a more generalized model, or a composite model that combines multiple models.

[0137] In the RSS model, for example, a longitudinal safety distance and a lateral safety distance from other road users are used as indicators for confirming a collision risk. The safety distance can be considered an example of a geometric approach such as a safety envelope. The safety envelope may refer to the longitudinal safety distance and the lateral safety distance from other road users themselves, or may refer to conditions or concepts for calculating these safety distances.

[0138] The risk confirmation unit 26 implemented in the driving system 9 is arranged in parallel with the planning unit 20 and executes calculation processing. Specifically, the risk confirmation unit 26 acquires an environmental model, sensor data, etc. from the detection unit 10, evaluates risk based on this information, and outputs a response according to the risk to the behavior unit 30. This series of functions or processing may be referred to as risk confirmation or risk monitoring. Risk confirmation or monitoring may be interpreted as safety confirmation or monitoring. Risk monitoring may also be referred to as driving policy monitoring. The risk confirmation unit 26 may include a situation extraction unit 27, a situation confirmation unit 28, and a response unit 29 as functional modules that further classify its functions. The situation extraction unit 27, the situation confirmation unit 28, and the response unit 29 may be realized by the processor 53b executing computer programs stored in the memory 52a.

[0139] The situation extraction unit 27 extracts a situation based on information acquired from the detection unit 10. Data indicating the situation (hereinafter referred to as situation data) may include a list of objects (hereinafter referred to as peripheral objects) present around the vehicle 1. The peripheral objects may include other road users. The peripheral objects may include features such as lane markings, signs, and guardrails. The situation data may include data indicating a potential conflict between the vehicle 1 and the peripheral objects. In this case, the situation data may include the existence probability and attribute uncertainty for the vehicle 1 and the peripheral objects. The attributes may be position, orientation, and speed. The situation extraction unit 27 may extract multiple situations. The situation may be a traffic situation. The situation may be selected from a set of possible situations.

[0140] The situation confirmation unit 28 confirms whether the situation extracted by the situation extraction unit 27 is a safe situation or a dangerous situation. The situation confirmation unit 28 performs confirmation using a safety envelope, confirmation using another methodology, or both. The confirmation here may be called risk confirmation or safety confirmation. In risk confirmation, the safety envelope may be set based on an acceptable collision risk.

[0141] The risk confirmation may include confirmation of the estimated collision risk between the vehicle 1 and a surrounding object. The collision risk may include a collision risk over time or a peak collision risk. The collision risk may be a collision probability. In other words, uncertainty can be taken into account when confirming the risk.

[0142] When the situation confirmation unit 28 executes risk confirmation, the situation confirmation unit 28 may compare the estimated collision risk value with an acceptable collision risk threshold. The acceptable collision risk threshold may be set in advance based on risk acceptance criteria / criterion, which will be described in detail later. When the estimated collision risk value is below the acceptable collision risk threshold, the situation confirmation unit 28 may determine that the situation to be confirmed is a safe situation. When the estimated collision risk value exceeds the acceptable collision risk threshold, the situation confirmation unit 28 may determine that the situation to be confirmed is a dangerous situation.

[0143] This risk threshold may be, for example, a vertical safety distance or a horizontal safety distance. That is, in a geometric approach, the situation confirmation unit 28 may set a safety envelope with a boundary corresponding to the risk threshold in order to confirm the risk of collision between the vehicle 1 and a surrounding object. Then, if a surrounding vehicle crosses the boundary of the safety envelope and enters the range of the safety envelope, it may be determined that there is a violation of the safety envelope. A violation of the safety envelope may be, for example, a failure to maintain a vertical or horizontal safety distance. If there is a violation of the safety envelope, the situation confirmation unit 28 may determine that the situation to be confirmed is a dangerous situation. If there is no violation of the safety envelope, the situation confirmation unit 28 may determine that the situation to be confirmed is a safe situation.

[0144] The situation confirmation unit 28 may set hypotheses about surrounding objects and confirm risks based on the hypotheses. In this case, multiple hypotheses may be used. The hypotheses may include assumptions about reasonably foreseeable behavior of surrounding objects. The hypotheses may also include predictions derived based on the assumptions. The assumptions may include at least one of kinematic-based assumptions and rule-based assumptions. The assumptions about behavior may include assumed values ​​of one or more physical parameters related to motion, such as acceleration. For example, the assumed values ​​may include the maximum deceleration of a preceding vehicle, the reaction time of the host vehicle, the maximum acceleration of the host vehicle, the minimum deceleration of the host vehicle, the minimum deceleration of an oncoming vehicle, or the lateral acceleration of a pedestrian.

[0145] The assumed values ​​may be derived using a function of time that changes during the identified scenario. Alternatively, the assumed values ​​may not change during the identified scenario. The assumed values ​​may differ depending on the category of road user. For example, the assumed values ​​may change depending on whether the road user is a vulnerable road user (VRU) or not. A VRU may also be referred to as a vulnerable road user. The assumed values ​​may be adjusted to account for at least one of various road surface conditions and weather-related environmental conditions that can reasonably be expected within the ODD. The assumed values ​​may be adjusted to account for at least one of differences in road traffic laws between countries and differences in driving habits between regions.

[0146] Assumptions may affect the acceptable risk level. The acceptable risk level or risk threshold may be preset based on risk acceptance criteria / criterion. The quantitative standard of the risk acceptance criteria may be that the probability of harm occurring is below a threshold. The risk acceptance criteria may be set based on a positive risk balance, which is the primary measure of an ethically acceptable risk level. The risk acceptance criteria may be set by combining a statistical approach, such as traffic accident statistics, with a scenario-based approach.

[0147] The risk tolerance criteria may be determined based on a comparison of the capability or operation of the driving system 9 under reasonably foreseeable scenarios within the ODD with the behavior of a competent and careful driver or an experienced and attentive driver. The risk tolerance criteria may be set based on the capability of the driving system 9 being equal to or greater than the driving capability of a competent and careful driver or an experienced and attentive driver.

[0148] The acceptable risk level or risk threshold may be specified in advance by, for example, at least one of a government agency, a standardization organization, and an approval organization for the driving system 9. The acceptable risk level or risk threshold may be set in advance by, for example, a developer who develops the driving system 9. Furthermore, the driving system 9 or the risk confirmation unit 26 may be configured to change the risk threshold depending on the ODD or the use case.

[0149] Furthermore, the situation confirmation unit 28 may determine an acceptable risk level by referring to a rule set stored in the rule DB 58. The situation confirmation unit 28 may improve the estimation accuracy by incorporating the rules of the rule set into the algorithm for calculating the risk value.

[0150] The response unit 29 derives a proper response based on the confirmation result of the situation confirmation unit 28. The response unit 29 may output the proper response to the action unit 30 only when the situation is determined to be dangerous. The proper response may be a restriction on the control command of the motion actuator 60. The proper response may be a response for returning the vehicle 1 to a safe state. The proper response may be an action related to reducing the safety envelope of the vehicle 1, such as braking, or an action for moving the vehicle 1 away from the safety envelope of other road users, such as steering. The proper response may also include both steering and braking. The proper response may be displaying an image, activating lighting equipment, outputting an alarm, or transmitting a wireless signal.

[0151] Here, even if a plurality of unrelated dangerous situations are identified, the actions to be taken by the vehicle 1 need to be consolidated into a single action. Therefore, the response unit 29 may resolve potential conflicts between appropriate responses to a plurality of unrelated dangerous situations and transmit the appropriate response to the behavior unit 30.

[0152] Additionally, the risk confirmation unit 26 may be configured to generate and output event data. The event data may include at least one of situation data, a risk confirmation result for the situation, and a derived appropriate response. The risk confirmation result may include at least one of a set safety envelope range and a risk threshold. The risk confirmation result may include an assumption that is a premise for risk confirmation. The assumption here may include information indicating whether a scenario transition is included in the assumption. The risk confirmation unit 26 may store the event data in the recording unit 54. The risk confirmation unit 26 may transmit the event data to an external system (e.g., the server 96) using the communication system 43 and store it in an external database.

[0153] Furthermore, the risk confirmation unit 26 may intervene in the management of the automation level by the mode management unit 23. For example, if the risk confirmation unit 26 detects an unacceptable risk while the driving system 9 is executing DDT, the risk confirmation unit 26 may forcibly change the automation level managed by the mode management unit 23 to a lower level. The risk confirmation unit 26 may execute a DDT fallback if a dangerous situation continues or occurs after outputting an appropriate response, in other words, if the risk is not sufficiently reduced.

[0154] <Detection of Subsystem Faults> The processing system 50 has a function of detecting faults in the subsystems that constitute the driving system 9. The subsystems here include devices connected to the processing system 50, such as the external environment sensor 41, and the ECUs that constitute the processing system 50. While the following mainly describes faults in the object detection type external environment sensor 41, faults in other components (in other words, subsystems), such as the internal environment sensor 42, the communication system 43, the map DB 44, and the ECU, may also be handled in the same manner.

[0155] Hereinafter, for the sake of simplicity, the object detection type external environment sensor 41 will be simply referred to as a sensor. The term "sensor" below may be interpreted primarily as the object detection type external environment sensor 41. However, as mentioned above, the following description may also apply to subsystems other than the object detection type external environment sensor 41. The term "sensor" below may be replaced with another term such as a subsystem. A sensor malfunction may be interpreted as a system failure.

[0156] The function of detecting a sensor malfunction (in other words, a diagnostic function) may be implemented in the sensor itself, the processing system 50, or both. If a sensor has a self-diagnostic function, the sensor outputs a status signal indicating its own operating status (normal or not) to the processing system 50. The status signal may also be called a diagnostic signal. A status signal is also a type of output signal. A status signal indicating the occurrence of a malfunction may be called an error signal. For example, a forward-facing camera may output an error signal when it detects an abnormality in the image sensor or processing circuit. Other sensors, such as millimeter-wave radar and LiDAR, may also have a self-diagnostic function and be configured to output an error signal to the processing system 50 when it detects an abnormality in its internal circuit (for example, a temperature abnormality).

[0157] If the sensor has a self-diagnostic function, the processing system 50 may detect a sensor malfunction based on a status signal received from the sensor. Detecting a malfunction may include recognizing (perceiving) the malfunction and determining the malfunction. The processing system 50 may also detect a sensor malfunction based on an inability to communicate with the sensor. Sensor malfunctions may also include a state in which the sensor fails to output a recognition result, a state in which the output signal is stuck, and a disconnected communication line.

[0158] Furthermore, the processing system 50 may detect a malfunction of a sensor to be diagnosed (hereinafter, the target sensor) by comparing the detection results of the sensor with map data. The processing system 50 may determine that a malfunction has occurred in the target sensor or that there is a possibility of such a malfunction when a feature (e.g., a guardrail or a sign) included in the map data is not detected by the target sensor. The processing system 50 may also determine that a malfunction has occurred in the target sensor or that there is a possibility of such a malfunction when a feature not included in the map data is detected by the target sensor. Sensor malfunctions may include recognition anomalies. Recognition anomalies may include a pattern of falsely detecting an object that does not actually exist and a pattern of failing to recognize an object that actually exists.

[0159] The processing system 50 may detect a malfunction of the target sensor by comparing the detection results of the target sensor with the detection results of other sensors whose detection ranges partially or completely overlap. The processing system 50 may detect a malfunction of a sensor by comparing the detection results of two or more sensors whose detection ranges overlap. The processing system 50 may determine, by majority vote, that the forward camera has malfunctioned if an object detected by the forward camera is not detected by the forward radar and LiDAR. The processing system 50 may determine, by majority vote, that the forward radar has malfunctioned if an object detected by the forward camera and LiDAR is not detected by the forward radar.

[0160] The processing system 50 may detect a sensor malfunction based on the instability of the detection result provided by the sensor. The state in which the detection result is unstable means that there is a large difference between the detection result at the previous time and the detection result at the next time. Alternatively, the processing system 50 may detect a sensor malfunction based on the fixation of the detection result provided by the sensor. The external environment may change dynamically while the vehicle 1 is traveling. Even when the detection result of the target sensor does not change for a certain period of time, even when the vehicle 1 is traveling at a speed equal to or greater than a predetermined value, the processing system 50 may determine that a malfunction (e.g., freezing) has occurred in the target sensor. The recognition abnormality may include a pattern in which the recognition result is unstable and a pattern in which the recognition result is fixed.

[0161] The processing system 50 may detect a malfunction of the first sensor based on an abnormality in the trajectory (i.e., the planning result) generated based on the detection results of the first sensor. The validity of the trajectory may be determined by comparing it with the trajectory of the preceding vehicle or a trajectory generated using a map. The abnormality of the trajectory generated based on the detection results of the first sensor may also be detected by comparing it with a trajectory generated using the detection results of another sensor without using the detection results of the first sensor. Here, the first sensor may be any external environment sensor 41, as described above. Furthermore, the processing system 50 may detect a malfunction of the first sensor based on detecting an abnormality (e.g., a break) in the power line connected to the first sensor. A malfunction of the first sensor may include the inoperation of the first sensor, in other words, a shutdown of the first sensor. The first sensor may also shut down due to a partial or complete loss of power. The processing system 50 may detect a malfunction of the sensor based on a power supply problem, such as a loss of main power.

[0162] Additionally, the sensor itself or the processing system 50 may detect sensor malfunctions based on an increase in the temperature of the device (circuit board) or the occurrence of deposits. A decrease in the sensor's FOV due to deposits on the sensor surface may also be considered a malfunction. The processing system 50 may also detect malfunctions of the target sensor using a watchdog timer or a connectivity check. Furthermore, the processing system 50 may be configured to diagnose the sensor using artificial intelligence (AI). The processing system 50 or the sensor may have a diagnostic model generated by machine learning.

[0163] The sensor malfunction may be classified into a temporary malfunction and a permanent malfunction. A temporary malfunction is a malfunction that can be resolved by a general vehicle user. A general vehicle user may be understood as a person who can perform maintenance described in the vehicle 1 instruction manual, but cannot perform advanced repair work that is not described in the instruction manual.

[0164] An example of a temporary fault may be the adhesion of external disturbances, such as fallen leaves, insects, raindrops, or mud, to the surface of an object-detecting sensor. That is, a temporary fault may be considered to be a dirty sensor surface. Errors that can be recovered by restarting (resetting) the sensor, such as accidental memory errors or freezes, may also be considered to be temporary faults. A temporary fault may include a fault that will resolve over time (e.g., the adhesion of raindrops).

[0165] A permanent malfunction is a malfunction that requires repair at a dealership or repair shop. A permanent malfunction is a malfunction that cannot be resolved (recovered) by a typical vehicle user. A permanent malfunction may include a malfunction that cannot be fixed by restarting the sensor. A permanent malfunction may be, for example, a failure of the optical or mechanical part of the sensor, an abnormality (failure) of the SoC or IC, a memory malfunction, a broken wire, a detached internal connector, or circuit damage. Initial defects due to inspection omissions or the like, or defects due to design or manufacturing errors or the like may also be included in permanent malfunctions. A temporary malfunction may be referred to as an abnormality, and a permanent malfunction may be referred to as a failure. In one aspect, a permanent malfunction may be included in a specific type of malfunction.

[0166] The processing system 50 may determine whether a malfunction occurring in the sensor is temporary or permanent based on the output signal of the sensor, time-series data of the detection results, temperature information, etc. An algorithm for determining whether a detected malfunction is temporary or permanent may be designed as appropriate. If it is not clear that the malfunction occurring in the sensor is temporary, the processing system 50 may determine that the malfunction occurring in the sensor is permanent.

[0167] Furthermore, the processing system 50 may execute a predetermined simple recovery process when a sensor malfunction occurs. The recovery process may be restarting the sensor. The recovery process may be cleaning the sensor surface. If the vehicle 1 is equipped with a cleaning device for the sensor in which a malfunction has been detected, cleaning the sensor surface as a recovery process may be operating the cleaning device. Cleaning the sensor surface as a recovery process may be requesting the vehicle user to clean the sensor surface. The request for cleaning may be made using the HMI device 70. If the malfunction is not resolved even after executing the simple recovery process, the processing system 50 may determine that the malfunction occurring in the sensor is a permanent malfunction.

[0168] When the processing system 50 detects a sensor malfunction, it may determine whether the malfunction is a safety-related malfunction. A safety-related malfunction may be rephrased as a DDT performance-relevant system failure by the driving system 9. Basically, a forward sensor malfunction may be a safety-related malfunction. A sonar malfunction may not be considered a safety-related malfunction in some cases. Also, a sensor malfunction with three or more alternatively available sensors (i.e., redundant sensors) may not immediately be considered a safety-related malfunction. Whether a sensor malfunction is a safety-related malfunction may be determined based on the function / application / importance of the malfunctioning sensor, the type of malfunction, and the traffic situation (scenario).

[0169] At least one of the operation planning unit 22, the mode management unit 23, and the risk confirmation unit 26 may have a sensor diagnostic function. A plurality of functional modules may have a function for detecting sensor malfunctions. The arrangement of functions within the operation system 9 can be changed as appropriate.

[0170] Note that detecting a malfunction of a subsystem, including a sensor, corresponds to detecting one factor that contributes to the initiation of DDT fallback. At least one of the operation planning unit 22, the mode management unit 23, and the risk confirmation unit 26 may have a function to detect the establishment of a fallback condition. Supplementally, one of the other factors that contributes to the initiation of DDT fallback may be the exit of the ODD. The function to detect the exit of the ODD may be provided by at least one of the prediction unit 21, the operation planning unit 22, the mode management unit 23, and the risk confirmation unit 26. Multiple functional modules may have a function to detect the exit of the ODD. The detection here may include prediction.

[0171] <System Operation> Here, the operation of the processing system 50 under normal circumstances will be described, followed by the operation of the processing system 50 when a malfunction is detected in the external environment sensor 41. The normal circumstances here may be understood as a situation in which no malfunction is detected in any of the external environment sensors 41, i.e., a situation in which the driving system 9 is capable of performing its designed (i.e., original) performance. The subsequent processing may be understood as processing executed when the autonomous driving function is enabled, i.e., in a mode with an automation level of 3 or higher. The subsequent processing may also be understood as processing executed when the semi-autonomous driving function is enabled, i.e., in a mode equivalent to automation level 2.5. The term "autonomous driving" below may be interpreted as "semi-autonomous driving."

[0172] 6 shows an example of processing (in other words, basic operations) of the processing system 50 under normal circumstances. The series of processing steps S101 to S104 shown in FIG. 6 may be realized, for example, by the processor 52b executing a computer program stored in the memory 52a. Some or all of the processing steps may be realized by the processor 53b executing a computer program stored in the memory 53a.

[0173] The functional arrangement may be changed as appropriate, and the processor 52b may execute processing related to risk confirmation. Therefore, hereinafter, the processors 52b and 53b will also be collectively referred to as the processor 52b. The memories 52a and 53a will also be collectively referred to as the memory 52a. Hereinafter, the term "processor 52b" may be replaced with the term "processor 53b," or the term "main unit 52," the monitoring unit 53, or the processing system 50. The processor 52b, the processor 53b, the main unit 52, the monitoring unit 53, or the processing system 50 correspond to processing modules. Depending on the context, the processor 52b may be replaced with the detection unit 10, the planning unit 20, the risk confirmation unit 26, or the action unit 30. The term "memory 52a" may be replaced with the term "memory 53a."

[0174] S101 is a step in which the processor 52b receives sensor data from a plurality of sensors (i.e., the external environment sensor 41) and stores the data in the memory 52a. S101 may be executed periodically or in response to the reception of sensor data. After S101 is executed, the processor 52b executes S102.

[0175] S102 is a step in which the processor 52b updates the environment model based on sensor data received from the sensors by the driving system 9. S101 corresponds to a step in which information indicating the external environment, internal environment, vehicle state, and state of the driving system 9 of the vehicle 1 is updated. After S102 is executed, the processor 52b executes S103. Note that the processor 52b may execute S103 periodically.

[0176] In step S103, the processor 52b creates a control plan based on the environmental model. The creation of the control plan may include identifying a scenario and predicting the behavior of other road users. The identification of the scenario may involve selecting a scenario corresponding to the current situation from a catalog of scenarios stored in the scenario DB 59 based on the latest environmental model.

[0177] Assuming the behavior of other road users may involve setting boundaries of a reasonably foreseeable range of movement of other road users in a specified scenario. The reasonably foreseeable range of movement of other road users may be calculated based on kinematic properties of the other road users detected by the external environment sensor 41. The kinematic properties may be speed, acceleration, orientation, etc. Assuming the behavior of other road users may involve calculating a potential range of movement of the other road users. The behavior assumption may be performed for each road user detected by the external environment sensor 41. The behavior assumption may also be performed for a virtual road user that is assumed to be outside the FOV of the external environment sensor 41.

[0178] The creation of the control plan includes setting a driving trajectory. The processor 52b may set a driving trajectory so as to avoid the potential movement areas of other detected or expected road users. In addition to setting a driving trajectory, the creation of the control plan may also include creating a plan for specific control of the motion actuators 60 and control of secondary actuators to achieve the driving trajectory. The control plan may include acceleration / deceleration schedule information for speed adjustment on the set trajectory, steering amount schedule information, and operation timing information for secondary actuators such as turn signals. Once the creation of the control plan is complete, the processor 52b executes S104.

[0179] S104 is a step in which the processor 52b outputs a command signal to the motion actuator 60 in accordance with the plan created in S103. S104 may include the processor 52b outputting a command signal to the secondary actuator 65 in accordance with the plan created in S103.

[0180] 7 is a flowchart for explaining an example of the operation of the processing system 50 when a malfunction of a sensor (i.e., the external environment sensor 41) is detected. The flowchart shown in FIG. 7 may be executed in parallel with the processing of the basic operation shown in FIG. 6. The processing related to the detection of a malfunction of the sensor may include S201 to S207 as shown in FIG. 7.

[0181] S201 is a step in which the processor 52b communicates with the sensor and receives a signal (data) from the sensor for identifying the operating state. The signal for identifying the operating state may be a status signal or a signal indicating a detection result. S201 is a step in which the operating state of the sensor (whether it is normal or not) is determined based on the received signal. S201 may include communication to check whether a signal can be received from the sensor. S201 may be executed periodically or in response to a specific event. For example, the processor 52b may execute S201 in response to a mode change, driving a certain distance, cutting in, changing lanes, a change in the speed limit, or the like. Instead of diagnosing all sensors simultaneously, the processor 52b may be configured to diagnose multiple sensors sequentially.

[0182] S202 is a step of determining whether a sensor has a malfunction based on the communication result (received signal or success or failure of communication) in S201. The determination of whether a malfunction has occurred may be performed using one or more of the methods described above. Furthermore, the determination of whether a malfunction has occurred may also be performed using a method other than the methods described above. When the processor 52b detects a malfunction in one (or more) of the external environment sensors 41, it executes S203. On the other hand, when a malfunction has not been detected in any of the sensors, it terminates this flow. Note that when the processor 52b detects a malfunction in one (or more) of the sensors, it may identify the sensor(s) that has the malfunction. In the present disclosure, a sensor determined to have a malfunction is also referred to as a malfunctioning sensor. In contrast, a sensor that is not malfunctioning is also referred to as a normal sensor. The time when a sensor malfunction is detected is also referred to as the time of malfunction detection.

[0183] S203 is a step in which the processor 52b initiates a fail-safe operation. The fail-safe operation may be understood as control to stop use of the malfunctioning sensor and continue autonomous driving to a safe location / until a specific condition is met. Continuing autonomous driving to a safe location may mean stopping the vehicle 1 at a safe location and then releasing (terminating) the autonomous driving function. The state in which a specific condition is met may mean a state in which the malfunction is resolved or a state in which the vehicle user takes over DDT. The fail-safe operation may be replaced with a fallback operation in one phase. The fail-safe operation may involve performing multiple different controls in stages. An initial fail-safe operation may be to continue autonomous driving after decelerating. That is, the fail-safe operation may include, as an appropriate response, reducing the driving speed from a base value to a predetermined value or setting the driving speed to a value that is a predetermined amount lower than the base value. The base value is a target value for the original driving speed that should be applied when no malfunction is detected. The base value may be determined according to the road speed limit, the speed of the preceding vehicle, or a speed set by the vehicle user.

[0184] Furthermore, the fail operation may include making the target value of the distance to the preceding vehicle (so-called inter-vehicle distance) longer by a predetermined amount than the original value. The original value of the inter-vehicle distance is a parameter determined according to the traveling speed of the preceding vehicle or vehicle 1. The original value of the inter-vehicle distance may be determined according to a level set by the vehicle user. The driving system 9 may be configured to be able to switch the original value of the inter-vehicle distance between three levels, such as long, medium, and short.

[0185] If the vehicle 1 is traveling in a lane other than the first lane, the fail operation may include moving the vehicle 1 to the first lane. The fail operation may include stopping the overtaking function. The overtaking function is a function that automatically overtakes a preceding vehicle by changing lanes. The fail operation may include requesting a fallback ready. The request for fallback ready may be to request the vehicle user to prepare to take over driving operations (DDT), for example, particularly in an automation level 3 vehicle. The request for fallback ready may be to request the vehicle user to monitor the surroundings of the vehicle 1. The request for fallback ready may be implemented by displaying a predetermined image on the display. The request for fallback ready may be accompanied by at least one of outputting a notification sound and outputting a voice message. The fail operation may ultimately be a control that stops the vehicle 1.

[0186] The control executed as the fail operation may be determined depending on the situation. The fail operation does not have to be a fixed control. The fail operation may be a response of the driving system 9 toward an MRC determined depending on the situation. That is, in one aspect, the fail operation may be a DDT fallback. The fail operation may determine an MRC based on the detection results of a normal sensor or map data, and perform an operation according to the determined MRC. If the malfunction detected in S201 is a safety-related malfunction, the processor 52b may initiate a DDT fallback as the fail operation. Furthermore, the fail operation may be a proper response or a fault reaction. The fail operation may be a strategic operation (failure mitigation strategy (manoeuvres)) to mitigate the impact of the malfunction. The fail operation may include, for example, the following steps S204 to S207.

[0187] S204 is a step of locking detection results (in other words, sensor data) already acquired from the malfunctioning sensor before the malfunction was detected. Locking the detection results may be a process of prohibiting the acquisition and storage of new detection results and leaving only acquired data in the memory 52a. As a result, the processor 52b operates to create a control plan without using new detection results output by the malfunctioning sensor after the malfunction is detected. S204 may include stopping the malfunctioning sensor. The processor 52b may be configured to store in the memory 52a only the detection results of sensors for which no malfunction has been detected. The detection results stored in the memory 52a may be deleted after a certain period of time has elapsed. The data stored in the memory 52a may be deleted in order from the oldest to the newest. After S205 is executed, the processor 52b executes S205. S204 may include stopping the malfunctioning sensor.

[0188] S205 is a step of updating the environment model using newly received detection results from external environment sensors 41 other than the malfunctioning sensor (i.e., normal sensors). In generating (updating) the environment model, detection results from malfunctioning sensors acquired before the malfunctioning sensor, which are stored in the memory 52a, may be used in addition to detection results from normal sensors newly acquired after the malfunctioning sensor is detected. When S205 is completed, the processor 52b executes S206.

[0189] In step S206, the processor 52b creates a control plan using the updated environmental model. The environmental model is updated using the detection results of the normal sensors newly acquired after the malfunction was detected. Therefore, step S206 corresponds to the process of determining the action to be taken in the fail operation based on the data acquired from the normal sensors.

[0190] Similar to S104, S207 is a step in which the processor 52b outputs a command signal according to the control plan determined in S206 to the motion actuator 60. S207 may include the processor 52b outputting a command signal according to the plan created in S207 to the secondary actuator 65. S205 to S207 may be periodically repeated until the vehicle 1 stops.

[0191] The processor 52b may change the information used to update the environment model in S205 depending on whether the vehicle 1 is still within the final recognition area. The final recognition range is the road range that was included in the FOV of the malfunctioning sensor before the malfunction was detected. The final recognition range may be the range in which the malfunctioning sensor was able to detect features such as road edges or lane marks. If the malfunctioning sensor is a forward sensor (e.g., a forward camera) and the FOV of the malfunctioning sensor included the road range up to 120 m ahead immediately before the malfunction was detected, the road range up to 120 m ahead from the malfunction detection point becomes the final recognition range. The malfunction detection point is the position of the vehicle 1 at the time the malfunction was detected.

[0192] When a sensor malfunction is detected, the processor 52b may determine the last perceived range of the malfunctioning sensor in S211 of FIG. 8. The last perceived range of the malfunctioning sensor may be determined by referring to the detection results before the malfunction was detected. For simplicity of description, in FIG. 8 and other figures, the last perceived range is denoted as LPR. LPR may be understood as, for example, an abbreviation for last perceived range, but is not limited to this. LPR may be treated as a term representing the range that the sensor was close to before the sensor malfunctioned.

[0193] When the vehicle 1 is inside the LPR of the malfunctioning sensor (YES in S212), the processor 52b may generate an environment model and create a control plan based on the past detection results of the malfunctioning sensor stored in the memory 52a and the latest detection results of other normal sensors (S213). Also, when the vehicle 1 is outside the LPR of the malfunctioning sensor (NO in S212), the processor 52b does not use the past detection results of the malfunctioning sensor stored in the memory 52a to create the control plan. The processor 52b may generate an environment model and create a control plan based on the latest detection results of other normal sensors (S214). Of course, not only the latest detection results of the normal sensors but also past detection results of the normal sensors may be used to create the control plan.

[0194] The processor 52b may be configured to continue the autonomous driving within the LPR when a malfunction occurs in any of the forward sensors, but to terminate the autonomous driving when the vehicle 1 leaves the LPR. If there is an evacuation space within the LPR, the processor 52b may stop the vehicle 1 in the evacuation space.

[0195] According to the above operation, after detecting a malfunction, the processor creates a post-malfunction control plan using the past detection results of the malfunctioning sensor and the latest detection results of the normal sensor. Since control is planned within the LPR based on more information, safety can be improved. Furthermore, the detection results of the normal sensor acquired after the malfunction is detected are reflected in the control plan. Therefore, even if the detection results of the malfunctioning sensor stored in the memory 52a no longer match the real world due to changes in traffic conditions after the malfunction occurs, the behavior of the vehicle 1 can be adapted to the real world. In other words, flexibility and safety in response to changes in traffic conditions after the malfunction occurs can be improved.

[0196] In addition, if the distance to the evacuation space is less than a predetermined value, processor 52b may autonomously drive vehicle 1 to the evacuation space using only the detection results of normal sensors, without using past detection results of the defective sensor.

[0197] 9, the processor 52b may change the specific action of the fail-operation depending on the type of malfunction that has occurred in the external environment sensor 41. That is, when the processor 52b detects a malfunction in a certain external environment sensor 41, it identifies the type of the malfunction in S221. The type of malfunction may be, for example, temporary or permanent. The type of malfunction may be identified by analyzing the output signal of the malfunctioning sensor or by comparing the output signal of the malfunctioning sensor with the output signal of a normal sensor.

[0198] If the processor 52b determines that the detected malfunction is temporary (YES in S222), it may decide to continue the automatic operation as a fail operation in S223. On the other hand, if the processor 52b determines that the detected malfunction is permanent (for example, a complete breakdown) (NO in S222), it executes S228.

[0199] A fail-operation in response to the occurrence of a temporary malfunction may be to continue the automated driving after reducing the driving speed by a predetermined amount. If a malfunction is detected while the vehicle is traveling in an overtaking lane such as the second lane, the processor 52b may create and execute a plan to change lanes to the first lane. The fail-operation in response to the occurrence of a temporary malfunction is not limited to the examples described in this section, and various controls may be adopted.

[0200] After S223, the processor 52b executes a recovery process in S224. The recovery process may involve outputting an operation command signal to a cleaning device corresponding to the malfunctioning sensor. The cleaning device may be a device that sprays a cleaning liquid (such as water) onto the sensor surface of the malfunctioning sensor, a device that blows air onto the sensor surface, or a wiping device (wiper).

[0201] The recovery process may be to display a recovery request image on the display. The recovery request image may be an image including text or an icon that requests the vehicle user to clean the sensor surface of the target sensor after temporarily stopping the vehicle 1. The processor 52b may display the recovery request image together with a notification sound. The recovery request image may also display the remaining time or remaining distance. The remaining time or remaining distance may be the remaining time or remaining distance until a takeover request or MRM, which will be described later, is executed. S224 is an optional element and may be omitted.

[0202] After performing the recovery process, the processor 52b determines whether a predetermined recovery waiting time has elapsed since the recovery process was performed (S225). The recovery waiting time may be 20 seconds, 30 seconds, 60 seconds, or the like. The recovery waiting time may be the estimated time until the vehicle exits the LPR. The recovery waiting time may be determined according to the vehicle's traveling speed.

[0203] When the recovery wait time has elapsed since the execution of the recovery process, the processor 52b determines in S226 whether the malfunction continues (remains). If the malfunction has been resolved when the recovery wait time has elapsed since the execution of the recovery process, the processor 52b ends the fail operation in S227 and resumes normal control (in other words, normal operation). Returning to normal control may mean terminating control specific to the fail operation, such as lifting the speed limit, lifting the limit on the overtaking function, or terminating the fallback ready request.

[0204] On the other hand, if the malfunction still persists (remains) after the recovery waiting time has elapsed since the execution of the recovery process, the processor 52b executes a takeover request (TOR) in S228. TOR is a request by the driving system 9 to take over driving operations from the vehicle user, based on the driving system's judgment. TOR may be referred to as a DDT takeover request, intervention request, fallback request, or handover request. TOR may include displaying an image on a display requesting the takeover of driving operations. TOR may also include outputting an audio message or warning sound from a speaker requesting the takeover. TOR is a system response to the malfunction and may be considered part of a fail-over operation.

[0205] After TOR, the processor 52b waits for a response from the vehicle user for a predetermined time. This waiting time (also referred to as the response waiting time) may be, for example, 10 seconds. The vehicle user's response may be gripping the steering wheel or depressing the pedals. If the vehicle user's response is obtained before the response waiting time has elapsed, the processor 52b executes a notification of authority transfer in S230. The notification of authority transfer is a process of notifying the vehicle user that the authority (or responsibility) for driving operation has been transferred from the system to the vehicle user. The notification of authority transfer may be executed using both visual information (e.g., an image display) and auditory information (a voice message) to enhance the vehicle user's awareness. The notification of authority transfer may include information on the DDT to be performed by the vehicle user and information on the DDT supported by the driving system 9.

[0206] On the other hand, if no response is received from the vehicle user even after the response waiting time has elapsed since the start of TOR, the processor 52b decides to start MRM in S231. For example, in S231, an MRC is determined and a control action for transitioning to the MRC is planned. The MRC may be stopping within the lane, stopping in a turn-off space, stopping on the shoulder, slowing down within the lane, or slowing down on the shoulder. The turn-off space may be a parking / stopping spot along the road that is wide enough to accommodate the vehicle. The turn-off space may be a safe parking area or an emergency parking strip. The shoulder may be understood as the portion outside the lane that is not a turn-off space. Slowing down, stopping, or parking on the shoulder may be slowing down, stopping, or parking with the vehicle body along the edge of the road. Slowing down, stopping, or parking on the shoulder may include a state in which a portion of the vehicle body protrudes into the lane.

[0207] The MRM may be at least one of an emergency stop within the lane, an autonomous stop after autonomous driving within the lane, and exiting the lane. The MRM may involve turning on hazard lights, providing an external alert using V2X, or displaying an emergency image on an external display. That is, the MRM may include driving a secondary actuator in addition to operating the motion actuator 60 to reach the MRC. The processor 52b outputs a command signal to the actuator according to the MRM plan created in S231 to guide the vehicle 1 to the MRC.

[0208] In S231, the processor 52b may further plan and execute an MRM notification that matches the planned control action. The MRM notification is a process of notifying a vehicle user of the flow of actions to be executed as an MRM. The MRM notification may include generating data indicating specific contents of the MRM and displaying the data on a display or the like. The MRM notification may also include notifying a person outside the vehicle 1 of the behavior of the vehicle 1 as an MRM using an external HMI device such as an external display.

[0209] In a driving system 9 compatible with automation level 4 (also referred to as level 4-ADS), TOR in steps S228 to S229 may be omitted and MRM may be started immediately. Of course, a level 4-ADS may also be configured to attempt TOR upon detection of a predetermined malfunction. A driving system 9 compatible with automation levels up to 3 (also referred to as level 3-ADS) may be configured to execute TOR depending on the type and duration of the malfunction.

[0210] According to the above operation, if the malfunction of the external environment sensor 41 is temporary, TOR or MRM is not immediately performed. This can improve convenience for the vehicle user. On the other hand, if the malfunction of the external environment sensor 41 is permanent, TOR / MRM is performed promptly. This can improve safety.

[0211] Note that step S222 may be a step of determining whether the type of malfunction is a safety-related malfunction. The processor 52b may be configured to promptly execute TOR or DDT fallback if the malfunction is a safety-related malfunction, and to execute step S223 if the malfunction is not a safety-related malfunction. Such an operation also achieves both safety and convenience.

[0212] <Functional Restrictions> Fail operation may also involve some functional restrictions. Functional restrictions mean that some of the functions originally provided by the driving system 9 are stopped (made unusable). Basic functions of autonomous driving include (1) lane keeping, (2) lane changing, (3) following the preceding vehicle, and (4) collision avoidance / mitigation control. Basic functions may be rephrased as basic control.

[0213] Lane keeping may involve steering to stay in the center of the lane, and may also be called lane centering. Lane changing may involve moving laterally to a slower lane (e.g., the first lane) or out of the lane. Vehicle following may involve automatically controlling speed so that the distance to the vehicle ahead remains constant. The vehicle following function may also be called ACC (Adaptive Cruise Control). Collision avoidance / mitigation control may be Advanced / Autonomous Emergency Braking (AEB) or Advanced / Autonomous Emergency Steering (AES).

[0214] When the processor 52b detects a sensor malfunction, it identifies the malfunctioning sensor from among the plurality of external environment sensors 41 based on the sensor output signal, etc. (S241 in FIG. 10 ). Then, the processor 52b may determine the range of function restriction depending on the detection direction (in other words, the monitoring area) of the malfunctioning sensor (S242).

[0215] For example, if a malfunction (e.g., breakdown) occurs in the forward camera, the lane mark recognition capability may be reduced. A reduction in lane mark recognition capability may affect the safety or reliability of functions (1) and (2). For this reason, if a malfunction occurs in the forward camera, the processor 52b may enable the basic functions (1) to (4) within the LPR, while terminating (disabling) functions (1) and (2) outside the LPR and enabling only functions (3) and (4). Functions (3) and (4) may be executed based on the detection results of the forward radar or LiDAR. Of course, if a malfunction occurs in the forward camera, the processor 52b may also disable functions (1) to (2) even within the LPR. Furthermore, if a malfunction occurs in the forward camera, the processor 52b may maintain functions (1) to (2) by utilizing other cameras, such as front side cameras or sub-cameras.

[0216] Furthermore, if a malfunction (e.g., breakdown) occurs in the forward radar, the ability to recognize the distance to the preceding vehicle may be reduced. When a malfunction occurs in the forward radar, the processor 52b may enable the basic functions (1) to (4) within the LPR, while stopping (disabling) the function (4) outside the LPR and enabling only the functions (1) to (3). The functions (1) to (3) may be executed based on the detection results of other forward sensors, such as LiDAR. Note that the processor 52b may be configured to maintain the function (4) by using a diagonally forward radar when a malfunction occurs in the forward radar.

[0217] If a malfunction (e.g., breakdown) occurs in the rear-side camera, the ability to detect other vehicles approaching from diagonally behind may be reduced. In other words, the ability to recognize open spaces for lane changes may be reduced. If a malfunction occurs in the rear-side camera, the processor 52b may enable the basic functions (1) to (4) for a certain period of time, but after the certain period of time has elapsed, it may stop (disable) the function (2) and enable only functions (1), (3), and (4). Of course, if a malfunction occurs in the rear-side camera, the processor 52b may immediately stop the function (2). This is because traffic conditions can change dynamically.

[0218] If a malfunction occurs in the rear-side radar, the ability to recognize open spaces for lane changes may also be reduced. Even if a malfunction occurs in the rear-side radar, the processor 52b may enable the basic functions (1) to (4) for a certain period of time, but may stop (disable) the function (2) after the certain period of time has elapsed, and enable only (1), (3) to (4). Of course, if a malfunction occurs in the rear-side radar, the processor 52b may immediately stop the function (2).

[0219] After identifying the faulty sensor (S251 in FIG. 11), the processor 52b may determine the specific details of the fail operation according to the role, function, or importance of the faulty sensor (S252). The importance may be a parameter that is dynamically determined according to the current situation.

[0220] For example, if a malfunction occurs in the forward sensor when there is no preceding vehicle, the vehicle may gradually decelerate to a predetermined safe speed, continue driving at the safe speed for a predetermined first time, and then execute DDT fallback. The first time may be the same length as the recovery waiting time or may be a different length. The safe speed may be set based on the speed limit set for the road and may be a speed lower than the speed limit. For example, the safe speed may be 10 km / h lower than the speed limit, or half the speed limit. The DDT fallback may include TOR. The DDT fallback may also include MRM.

[0221] On the other hand, if a malfunction occurs in the front sensor when there is a preceding vehicle, TOR or MRM may be initiated promptly. If a malfunction occurs in the front sensor when there is another vehicle that may cut in, TOR or MRM may also be initiated promptly. In a situation where there is a preceding vehicle or a situation where there is a possibility of cutting in, the importance of the front sensor is high, and the impact of a malfunction on safety is greater than when there is no preceding vehicle, etc. In a situation (in other words, a scenario) in which the importance of the front sensor is high, the fail operation may be control to stop the vehicle 1 within a relatively short distance. In a situation in which the importance of the front sensor is low, the fail operation may be control to stop the vehicle 1 after traveling a relatively long distance.

[0222] Furthermore, when the vehicle 1 is moving forward, the importance of the rear sensor may be less than the importance of the front sensor. Therefore, if a malfunction occurs in the rear sensor while the vehicle 1 is moving forward, the processor 52b may execute control that is relatively close to normal operation as a fail-operation. Control that is close to normal operation may mean that the amount of deceleration is small. Control that is close to normal operation may mean that the range of functional limitations is small.

[0223] <Relationship between Traveling Direction and Faulty Sensor> The processor 52b may determine whether to execute a fail operation or the specific content of the fail operation based on the relationship between the faulty sensor and the traveling direction. The processor 52b may be configured to execute a fail operation when a fault occurs in a sensor related to the traveling direction, but not to execute a fail operation when a fault occurs in a sensor not related to the traveling direction.

[0224] The correlation between the sensor and the traveling direction may be evaluated based on the correlation between the sensor's detection direction and the traveling direction. The processor 52b may set the fail operation to a control closer to normal operation (e.g., slight deceleration) the smaller the correlation between the sensor's detection direction and the traveling direction when a malfunctioning sensor occurs. When moving forward, the rear sensor may be a sensor with a small correlation with the traveling direction, and when reversing, the sensor may be a sensor with a large correlation with the traveling direction. When moving forward, the front sensor may be a sensor with a large correlation with the traveling direction, and when reversing, the sensor may be a sensor with a small correlation with the traveling direction.

[0225] The processor 52b may set a longer distance for which autonomous driving can continue after detecting a malfunction, as the correlation between the malfunctioning sensor and the traveling direction becomes smaller. The processor 52b may determine specific details of a fail operation such that the vehicle stops earlier, as the correlation between the malfunctioning sensor and the traveling direction becomes larger.

[0226] <Positional Relationship Between Evacuation Space and LPR> The processor 52b may change the action taken by the vehicle 1 or the vehicle user depending on whether or not there is an evacuation space within the LPR. When the processor 52b detects a malfunction of a forward sensor, the processor 52b acquires the final recognition range (i.e., the LPR) of the forward sensor in S261 of FIG. 12. Then, the processor 52b may determine whether or not there is an evacuation space within the LPR based on the detection results of the external environment sensors 41, including the malfunctioning sensor, or map data (S262).

[0227] If there is an evacuation space within the LPR (YES in S262), the processor 52b may determine to continue the automated driving to the evacuation space as a fail operation (S263). A driving plan (e.g., a trajectory) to the evacuation space may be generated from the detection result of the malfunctioning sensor acquired before the malfunction is detected and the detection result of another forward sensor acquired after the malfunction is detected. The driving speed to the evacuation space may be the original value or a value reduced from the original value.

[0228] On the other hand, if there is no evacuation space in the LPR (NO in S262), the processor 52b may execute TOR while temporarily driving the vehicle 1 as a fail operation (S264). If no response is obtained from the vehicle user in response to TOR (NO in S265), the processor 52b may execute MRM (S267). Furthermore, if a response is obtained from the vehicle user in response to TOR, the processor 52b may execute a notification of authority transfer (S266). The driving plan during the execution of TOR may be generated from the detection results of the malfunctioning sensor acquired before the malfunction is detected and the detection results of other forward sensors acquired after the malfunction is detected. The driving speed during the execution of TOR may be reduced from the original value.

[0229] With the above configuration, the vehicle 1 can be moved appropriately according to the relative positions of the LPR and the evacuation space. It also reduces the risk of the vehicle 1 stopping in a lane or on the shoulder of the road even when an evacuation space is nearby. This helps maintain smooth traffic flow. If the driving system 9 supports automation level 4, TOR in steps S264 to S265 may be omitted, and MRM may be started immediately.

[0230] While Figure 12 illustrates a pattern in which TOR is executed when there is no evacuation space within the LPR, the specific content of the fail operation is not limited to this. The various operational examples described above may be applied. Furthermore, if there is an evacuation space within a predetermined distance (e.g., 200 m or 400 m) from the farthest point of the LPR, automated driving may continue to the evacuation space using other forward sensors. However, if some of the forward sensors are invalid due to a malfunction, robustness may be reduced compared to normal situations. Therefore, near the end of the LPR and after exiting the LPR, the processor 52b may request a fallback ready.

[0231] In addition, if there is an evacuation space within a specified distance (e.g., 100 m) from the point where the malfunction was detected, automatic driving may be continued to the evacuation space using only the detection results of the normal sensor, without using the past detection results of the malfunctioning sensor.

[0232] <Detection of ODD Exit> When the processor 52b detects a malfunction in one of the external environment sensors 41, the processor 52b may use the remaining normal sensors to determine whether or not the vehicle 1 is inside the ODD. Furthermore, the processor 52b may use the normal sensors to determine (predict) the possibility of the ODD exiting (S301 in FIG. 13). Then, when the exiting of the ODD is predicted (YES in S302), the processor 52b may perform an emergency stop (S303).

[0233] For example, in a system configuration in which the ODD includes a detection of no pedestrians around the vehicle 1, if a pedestrian is detected by a normal sensor, the processor 52b may start decelerating the vehicle to stop within the lane or on the shoulder. When decelerating to stop the vehicle, the processor 52b may turn on the hazard lights as the vehicle decelerates.

[0234] In a system configuration in which the ODD includes a function to detect the absence of a fallen object, when information indicating the presence of a fallen object ahead of the vehicle 1 is acquired using a normal sensor or communication system, the processor 52b may start decelerating to stop the vehicle 1 within the lane or on the shoulder. The stopping position may be in front of the fallen object. Note that if the malfunctioning sensor is, for example, a sonar sensor, and the impact on safety is small, the processor 52b may stop the vehicle 1 after passing the fallen object. A case in which the impact on safety is small may be interpreted as a case in which the impact on the lane change function and forward monitoring function for collision avoidance is small, and the risk caused by the malfunction is at an acceptable risk level.

[0235] <Another Operation Example (1)> In FIG. 9, a mode in which subsequent operation is changed depending on whether the detected malfunction is temporary or not has been described, but the operation of the processing system 50 is not limited to this. As shown in FIG. 14, the processor 52b may determine subsequent operation depending on whether the detected malfunction corresponds to a malfunction related to DDT performance or not. The malfunction here may be referred to as a system malfunction. The flowchart shown in FIG. 14 includes steps S401 to S409. These processes may be executed in addition to or instead of the processes described above. Note that if the process of FIG. 14 conflicts with the process of FIG. 9, the process of FIG. 14 may take precedence.

[0236] In S401, similar to S221, the processor 52b determines whether the detected malfunction is a malfunction related to DDT performance based on the received signal from the malfunctioning sensor (and other sensors). For example, the memory 52a may store definition data (e.g., a list) of malfunctions related to DDT performance. The memory 52a may also store conditions for determining whether a malfunction is related to DDT performance. Such data indicating whether a malfunction is related to DDT performance is referred to as a malfunction definition file.

[0237] The processor 52b may determine whether the detected malfunction is a malfunction related to DDT performance by referring to the malfunction definition file. For example, if the vehicle 1 is equipped with three front sensors and one of the three front sensors fails, the processor 52b may determine that a malfunction related to DDT performance has occurred.

[0238] Furthermore, if a sensor for which there is no alternatively available sensor fails, the processor 52b may determine that a malfunction related to DDT performance has occurred. For example, if there is only one right front sensor and it fails, the processor 52b may determine that a malfunction related to DDT performance has occurred. This is because if the only right front sensor fails, the VRU's protection capability when turning right may be reduced. An alternatively available sensor may be understood as a sensor whose detection range (in other words, monitoring area) overlaps. Conversely, if there are a certain number or more alternatively available sensors, the failure of one of those sensors may not be considered a malfunction related to DDT performance. The malfunction related to DDT performance may be a specific type of malfunction. S401 may be executed based on detecting that a malfunction has occurred in one of the external environment sensors 41.

[0239] When the processor 52b determines that the detected malfunction is not related to DDT performance (NO in S402), the processor 52b notifies the vehicle user and restricts functions in S403, and then continues the automated driving. The notification to the vehicle user may be to present information about the malfunctioning sensor. The notification to the vehicle user may include information about the functions to be restricted. The function restriction executed in S403 may be, for example, stopping the automatic overtaking function or lowering the upper limit of the driving speed. The functions to be restricted may be determined according to the characteristics of the malfunctioning sensor.

[0240] On the other hand, when the processor 52b determines that the detected malfunction is related to DDT performance (YES in S402), the processor 52b starts DDT fallback. As DDT fallback, the processor 52b creates and executes a provisional control plan using the sensor data acquired from the malfunctioning sensor before the malfunction occurred and the sensor data acquired from the normal sensor after the malfunction occurred, which are stored in the memory 52a.

[0241] The interim control plan may be a plan for vehicle behavior until the response waiting time at TOR has elapsed. The interim control plan may be a control plan until a decision is made to start MRM. The interim control plan may basically be to maintain driving within the ego lane. The interim control plan may be to maintain a constant distance from the preceding vehicle, a constant speed, or a gradual deceleration. If the vehicle is driving in the passing lane at the time the malfunction is detected, the interim control plan may include moving to the first lane.

[0242] When planning the provisional control, the detection results of the defective sensors stored in the memory 52a may be used, but this is not necessarily the case. The detection results of the defective sensors stored in the memory 52a do not necessarily have to be used.

[0243] With the start of the provisional control, the processor 52b may execute TOR in S406. If a response to TOR is obtained from the vehicle user (YES in S407), the processor 52b may execute notification of authority transfer in S408. Furthermore, if a response to TOR is not obtained from the vehicle user (NO in S407), the processor 52b may start MRM in S409.

[0244] If the driving system 9 supports automation level 4, the processor 52b may omit TOR and immediately start MRM. If the driving system 9 does not support automation level 4 or higher, the processor 52b may plan and execute an emergency stop within the lane or on the shoulder in S409.

[0245] According to the above configuration, if a malfunction related to DDT performance occurs, DDT forkback can be initiated, and the driving system or the vehicle user can perform an operation to stop the vehicle at a safe location. This maintains or improves safety when a malfunction occurs in the driving system. Furthermore, while waiting for the vehicle user's response to TOR or during the period until MRM begins, provisional control is implemented using the detection results of normal sensors. The provisional control primarily focuses on maintaining lane travel and does not abruptly stop or turn the vehicle 1. This provisional control reduces the risk of confusing other road users or creating an unsafe situation.

[0246] <Another Operation Example (2)> In the above, an example of a configuration in which an MRC is determined and then an MRM corresponding to the determined MRC is executed has been exemplified as a system operation when an MRM is executed, but the operation of the processor 53b is not limited to this. The processor 53b may determine the type and action of an MRM according to the traffic situation or the details of the malfunction without determining an MRC, and may cause the vehicle 1 to reach the MRC.

[0247] The driving system 9 may be provided with multiple types of MRMs. The types of MRMs may include, for example, a first type, a second type, and a third type. The first type of MRM may decelerate and stop while traveling straight, the second type of MRM may decelerate and stop within a lane, and the third type of MRM may move toward stopping on a road shoulder. The first type may be referred to as a straight stop type. The second type may be referred to as an in-lane stop type. The third type may be referred to as a road shoulder stop type.

[0248] Which type of MRM to preferentially execute among the multiple types of MRM that the driving system 9 can execute may be registered in advance in the driving system 9 by a designer or a user. For example, the processor 53b may be configured to preferentially adopt the third type as long as the surrounding traffic conditions permit. When the vehicle user does not respond to the takeover request, the processor 53b may determine the type of MRM based on at least one of the detection result of the first sensor acquired before the detection of the malfunction and the detection result of the second sensor acquired after the detection of the malfunction.

[0249] The processor 53b may also determine the MRM type to be preferentially applied depending on the type of malfunction occurring in the driving system 9. If a malfunction occurs in the function for recognizing lane marks and free spaces in the evacuation direction, the processor 53b may be configured to preferentially select the first type. The evacuation direction refers to the direction in which a shoulder exists as a stopping location, and is generally the left side in areas where left-hand traffic is permitted. The term "free space" refers to an area on the road where no other road users exist. If the malfunction is not related to the function for recognizing free spaces and lane marks and these recognition functions are normal, the processor 53b may be configured to preferentially select the third type over the second and first types. If the malfunction is related to the recognition of free spaces in the evacuation direction but does not affect the ability to recognize lane marks, the processor 53b may be configured to preferentially select the second type over the third and first types.

[0250] The processor 53b may change the MRM type and the MRC even after the start of the MRM. The MRM type and the final MRC may be dynamically changed depending on the situation. This allows the vehicle behavior to adapt to changes in traffic conditions.

[0251] <Other Operation Example (3)> Because the first sensor may stop operating due to an abnormality in the power line, such as a loss of power, the loss of power to the first sensor may be regarded as a malfunction of the first sensor. In response to the loss of power to the first sensor, the processor 52b may initiate a fail operation.

[0252] <Supplementary Note (1)> This specification discloses a number of technical ideas described in the following paragraphs. The present disclosure also includes methods, processing modules, recording media having programs recorded thereon, and programs corresponding to the following driving systems. Note that the first sensor below may be any external environment sensor 41 that forms a detection range in a direction related to the direction of travel. The second sensor may be any other external environment sensor 41 whose detection range overlaps with that of the first sensor. When the vehicle moves forward, the first and second sensors may be front sensors. When the vehicle moves backward, the first and second sensors may be rear sensors.

[0253] [Technical Idea 1] A driving system that automatically controls the speed and steering of a vehicle, comprising: a first sensor (411) that forms a detection range in the traveling direction of the vehicle; a second sensor (412) whose detection range overlaps with part or all of the detection range of the first sensor; and a processing module (52b) that creates a driving plan for the vehicle based on output signals of the first sensor and the second sensor, wherein the processing module is configured to: detect a malfunction of the first sensor; initiate a fail operation in response to the detection of the malfunction of the first sensor; and determine the control to be executed in the fail operation based on data acquired from the second sensor.

[0254] The processing module may detect a malfunction of the first sensor from an output signal of the first sensor, or may detect a malfunction of the first sensor based on other information. The malfunction of the first sensor may include a malfunction of the first sensor itself, as well as a stop of operation of the first sensor due to a power supply abnormality. Detecting the malfunction of the first sensor may be replaced with detecting an abnormality in the power supply for the first sensor. The term "fail operation" may be replaced with "operation for mitigating the malfunction."

[0255] [Technical Idea 2] The operation system according to Technical Idea 1, wherein the processing module is configured to: when the malfunction of the first sensor is not detected, receive data indicating the detection result of the first sensor from the first sensor and store it in memory; and when the malfunction of the first sensor is detected, create a control plan for the fail operation based on the detection result of the first sensor that was acquired before the malfunction was detected and stored in the memory and the detection result of the second sensor that was acquired after the malfunction was detected.

[0256] [Technical Idea 3] The driving system according to Technical Idea 1, wherein the processing module is configured to, in the fail operation, receive data indicating the detection result of the first sensor from the first sensor and store it in memory, create a driving plan for the vehicle based on the detection result of the first sensor acquired before the detection of the malfunction and stored in memory and the detection result of the second sensor acquired after the detection of the malfunction, if the vehicle is outside the road range detected by the first sensor before the detection of the malfunction, not use the detection result of the first sensor stored in memory but use the detection result of the second sensor.

[0257] [Technical Idea 4] A driving system described in any one of Technical Ideas 1 to 3, wherein the processing module is configured to determine whether or not there is an evacuation space within the road range detected by the first sensor up until the detection of the malfunction, and to change behavior depending on whether or not there is an evacuation space within the road range detected by the first sensor up until the detection of the malfunction.

[0258] [Technical Idea 5] A driving system described in any one of Technical Ideas 1 to 4, configured to receive data indicating the detection results of the first sensor from the first sensor and store it in memory, determine whether there is an evacuation space within the road range detected by the first sensor up until the detection of the malfunction based on the detection results of the first sensor stored in the memory, and if there is an evacuation space within the road range detected by the first sensor up until the detection of the malfunction, automatically drive the vehicle to the evacuation space based on the detection results of the second sensor obtained after the detection of the malfunction.

[0259] [Technical Idea 6] A driving system described in any one of Technical Ideas 1 to 5, which determines whether there is an evacuation space within the road range detected by the first sensor before the malfunction is detected, and executes a takeover request if there is no evacuation space within the road range detected by the first sensor before the malfunction is detected.

[0260] [Technical Idea 7] The operating system described in any one of Technical Ideas 1 to 6, wherein the processing module is configured to: when it detects a malfunction of the first sensor, determine whether the malfunction is a temporary malfunction or a permanent malfunction based on the output signal of the first sensor before or after the detection of the malfunction; when it determines that the detected malfunction is the permanent malfunction, immediately execute a takeover request; and when it determines that the detected malfunction is the temporary malfunction, execute the takeover request after the malfunction has continued for a certain period of time.

[0261] [Technical Idea 8] The operating system according to Technical Idea 7, wherein the processing module is configured to: execute a predetermined recovery process to eliminate the malfunction when it is determined that the detected malfunction is the temporary malfunction; and implement the takeover request based on the fact that the malfunction remains after the recovery process.

[0262] [Technical Idea 9] The driving system described in any one of Technical Ideas 1 to 8, wherein the malfunction of the first sensor corresponds to a system failure related to dynamic driving task (DDT) performance, and the fail operation is a DDT fallback.

[0263] [Technical Idea 10] The driving system described in any one of Technical Ideas 1 to 9, wherein the processing module is configured to: execute a takeover request based on the fact that the detected malfunction is a malfunction of a specific type; if the vehicle user does not respond to the takeover request, determine a minimal risk condition (MRC) based on at least one of the detection results of the first sensor obtained before the malfunction was detected and the detection results of the second sensor obtained after the malfunction was detected; and execute a minimal risk maneuver (MRM) according to the determined minimal risk condition. [Technical Idea 11] The driving system described in any one of Technical Ideas 1 to 9, wherein the processing module is configured to: execute a takeover request based on the fact that the detected malfunction is a malfunction of a specific type; if the vehicle user does not respond to the takeover request, determine a type of minimal risk maneuver (MRM) based on at least one of the detection results of the first sensor obtained before the malfunction was detected and the detection results of the second sensor obtained after the malfunction was detected; and initiate the determined minimal risk maneuver to bring the vehicle to a minimal risk condition (MRC).

[0264] [Technical Idea 12] A driving system described in any one of Technical Ideas 1 to 11, wherein the processing module determines whether or not to exit the operational design domain (ODD) of the vehicle based on the detection result of the second sensor during the fail operation, and if it determines to exit the operational design domain, creates a plan to stop the vehicle inside the operational design domain.

[0265] [Technical Idea 13] An operating system according to any one of Technical Ideas 1 to 12, comprising a plurality of sensors including the first sensor and the second sensor, wherein the processing module is configured to: detect that a malfunction has occurred in any of the plurality of sensors based on the output signals of each of the plurality of sensors; and determine an action to be taken as the fail operation depending on the sensor in which the malfunction has occurred.

[0266] [Technical Idea 14] An operating system according to any one of Technical Ideas 1 to 13, comprising a plurality of sensors including the first sensor and the second sensor, wherein the processing module detects that a malfunction has occurred in any of the plurality of sensors based on the output signals of each of the plurality of sensors, and determines the range of functions to be restricted in the fail-over operation depending on the sensor in which the malfunction has occurred.

[0267] <Supplementary Note (2)> The various flowcharts shown in this disclosure are all examples, and the number of steps constituting the flowcharts and the execution order of the processes can be changed as appropriate. The controls shown in each flowchart may be combined / executed in parallel to the extent that there is no contradiction. Terms such as acquisition, determination, detection, generation, and calculation may be used interchangeably. The acquisition of certain data by a certain device also includes the device generating the data based on a signal input from another device / sensor. The number of computers included in the driving system 9 and the functions they are responsible for may be changed as appropriate.

[0268] The apparatus, system, and methods described herein may be implemented by a special-purpose computer having a processor programmed to perform one or more functions embodied in a computer program. The apparatus and methods described herein may also be implemented using special-purpose hardware logic circuitry. The apparatus and methods described herein may also be implemented by a combination of a processor executing a computer program and one or more hardware logic circuits. The processor (52b, 53b) as a processing module may include at least one of a CPU, an MPU, a GPU, a DFP, and a RISC-CPU as a core. Some or all of the functionality of the processing module may be implemented in hardware. The processing module may be implemented using at least one of a SoC, an IC (Integrated Circuit), and an FPGA (Field-Programmable Gate Array). The computer program includes instructions for execution by a computer. The computer program may be stored on at least one computer-readable non-transitory tangible storage medium. The computer program recording medium may be a variety of media such as a hard-disk drive (HDD), a solid-state drive (SSD), or a flash memory.

Claims

1. A driving system that automatically controls the speed and steering of a vehicle, comprising: a first sensor (411) that forms a detection range in the direction of travel of the vehicle; a second sensor (412) whose detection range overlaps with part or all of the detection range of the first sensor; and a processing module (52b) that creates a driving plan for the vehicle based on output signals from the first sensor and the second sensor, wherein the processing module is configured to: detect a malfunction of the first sensor; initiate a fail-operation in response to detecting the malfunction of the first sensor; and determine the control to be executed in the fail-operation based on data obtained from the second sensor.

2. The operating system of claim 1, wherein the processing module is configured to: when the malfunction of the first sensor is not detected, receive data indicating the detection result of the first sensor from the first sensor and store it in memory; and when the malfunction of the first sensor is detected, create a control plan for the fail operation based on the detection result of the first sensor acquired and stored in memory before the malfunction was detected and the detection result of the second sensor acquired after the malfunction was detected.

3. The driving system of claim 1, wherein the processing module is configured to, in the fail operation, receive data indicating the detection results of the first sensor from the first sensor and store it in memory; if the vehicle is within a road range detected by the first sensor before the malfunction is detected, create a driving plan for the vehicle based on the detection results of the first sensor acquired before the malfunction is detected and stored in memory and the detection results of the second sensor acquired after the malfunction is detected; and if the vehicle is outside the road range detected by the first sensor before the malfunction is detected, create a driving plan for the vehicle using the detection results of the second sensor without using the detection results of the first sensor stored in memory.

4. The driving system of claim 1, wherein the processing module is configured to determine whether or not there is an evacuation space within the road range detected by the first sensor up until the detection of the malfunction, and to change behavior depending on whether or not there is an evacuation space within the road range detected by the first sensor up until the detection of the malfunction.

5. The driving system of claim 1, configured to receive data indicating the detection results of the first sensor from the first sensor and store it in memory, determine whether there is an evacuation space within the road range detected by the first sensor up until the detection of the malfunction based on the detection results of the first sensor stored in the memory, and if there is an evacuation space within the road range detected by the first sensor up until the detection of the malfunction, automatically drive the vehicle to the evacuation space based on the detection results of the second sensor obtained after the detection of the malfunction.

6. The driving system of claim 1, which determines whether there is an evacuation space within the road range detected by the first sensor before the malfunction is detected, and executes a takeover request if there is no evacuation space within the road range detected by the first sensor before the malfunction is detected.

7. The operating system of claim 1, wherein the processing module is configured to: when detecting a malfunction of the first sensor, determine whether the malfunction is a temporary malfunction or a permanent malfunction based on the output signal of the first sensor before or after the detection of the malfunction; when determining that the detected malfunction is the permanent malfunction, promptly execute a takeover request; and when determining that the detected malfunction is the temporary malfunction, execute the takeover request after the malfunction has continued for a certain period of time.

8. The operating system of claim 7, wherein the processing module is configured to: execute a predetermined recovery process to resolve the malfunction if it determines that the detected malfunction is the temporary malfunction; and implement the takeover request based on the malfunction remaining after the recovery process.

9. The driving system of claim 1, wherein the malfunction of the first sensor corresponds to a system failure related to dynamic driving task (DDT) performance, and the fail operation is a DDT fallback.

10. The driving system of claim 1, wherein the processing module is configured to: execute a takeover request based on the detected malfunction being a malfunction of a specific type; if the vehicle user does not respond to the takeover request, determine a minimal risk condition (MRC) based on at least one of the detection results of the first sensor obtained before the malfunction was detected and the detection results of the second sensor obtained after the malfunction was detected; and execute a minimal risk maneuver (MRM) according to the determined minimal risk condition.

11. The driving system of claim 1, wherein the processing module is configured to: execute a takeover request based on the detected malfunction being a specific type of malfunction; if the vehicle user does not respond to the takeover request, determine a type of minimal risk maneuver (MRM) based on at least one of the detection results of the first sensor obtained before the malfunction was detected and the detection results of the second sensor obtained after the malfunction was detected; and cause the vehicle to reach a minimal risk condition (MRC) by initiating the determined minimal risk maneuver.

12. The driving system of claim 1, wherein the processing module determines whether or not to exit the operational design domain (ODD) of the vehicle based on the detection result of the second sensor during the fail operation, and if it determines to exit the operational design domain, creates a plan to stop the vehicle inside the operational design domain.

13. The operating system of claim 1, further comprising a plurality of sensors including the first sensor and the second sensor, wherein the processing module is configured to: detect that a malfunction has occurred in any of the plurality of sensors based on the output signals of each of the plurality of sensors; and determine an action to be taken as the fail operation depending on the sensor in which the malfunction has occurred.

14. The operating system according to claim 1, further comprising a plurality of sensors including the first sensor and the second sensor, wherein the processing module detects that a malfunction has occurred in any of the plurality of sensors based on the output signals of each of the plurality of sensors, and determines the range of functions to be restricted in the fail-operation according to the sensor in which the malfunction has occurred.

15. A processing module that executes processing for automatically controlling the speed and steering of a vehicle, the processing module being configured to execute the following operations: receive detection results from a first sensor (411) that forms a detection range in the traveling direction of the vehicle; receive detection results from a second sensor (412) whose detection range overlaps with part or all of the detection range of the first sensor; create a driving plan for the vehicle based on the detection results of the first sensor and the detection results of the second sensor; detect a malfunction of the first sensor; initiate a fail-operation in response to the detection of a malfunction of the first sensor; and determine the control to be executed in the fail-operation based on data acquired from the second sensor.

16. A method for automatically controlling the speed and steering of a vehicle, comprising: receiving detection results from a first sensor (411) that forms a detection range in the traveling direction of the vehicle; receiving detection results from a second sensor (412) whose detection range overlaps with part or all of the detection range of the first sensor; creating a driving plan for the vehicle based on the detection results of the first sensor and the detection results of the second sensor; detecting a malfunction of the first sensor; initiating a fail-operation in response to the detection of the malfunction of the first sensor; and determining control to be performed in the fail-operation based on data acquired from the second sensor.

Images