Test execution device, test execution method, and test execution program

Abstract

A software management unit (57), as a test execution device, is provided with at least one processor (57b) for executing tests in vehicles (1, 1A, 1B) capable of participating in public road traffic. The processor (57b) executes: preparing, in advance, one or a plurality of test patterns pertaining to driving takeover between autonomous driving by driving systems 2 of the vehicles (1, 1A, 1B) and driving by drivers; and testing a test pattern pertaining to driving takeover in a situation where the driving takeover from the driving systems (2) to the driver occurs.

Description

Test execution device, test execution method, and test execution program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2024-78079 filed in Japan on May 13, 2024, the contents of which are incorporated by reference in their entirety.

[0002] The disclosure herein relates to testing with vehicles that are capable of participating in public road traffic.

[0003] Patent Literature 1 discloses a technique for generating test cases for vehicles capable of autonomous driving using vehicles that can participate in public road traffic.

[0004] U.S. Pat. No. 10,255,168

[0005] Now, vehicle systems are becoming more complex every year, and providing a V&V (Verification and Validation) process to accommodate these increasingly complex systems is becoming increasingly important. In particular, for vehicles capable of autonomous driving, there are cases where a driver-driven transition is required between autonomous driving by the driving system and driving by the driver, and there is a demand for appropriate improvements to the vehicle system specifications regarding this transition.

[0006] One of the purposes of the disclosure of this specification is to provide a test execution device, a test execution method, and a test execution program that facilitate appropriate improvement of vehicle system specifications.

[0007] One aspect disclosed herein is a test execution device having at least one processor for conducting tests in a vehicle capable of participating in public road traffic, wherein the at least one processor performs the following steps: preparing one or more test patterns in advance regarding a driving handover between autonomous driving by the vehicle's driving system and driving by a driver; and testing the test patterns regarding the driving handover in a situation in which a driving handover from the driving system to the driver occurs.

[0008] Another aspect disclosed herein is a test implementation method for conducting tests on a vehicle capable of participating in public road traffic, comprising: preparing in advance one or more test patterns related to a driving handover between autonomous driving by the vehicle's driving system and driving by a driver; and testing the test patterns related to the driving handover in a situation in which a driving handover from the driving system to the driver occurs.

[0009] Another aspect disclosed herein is a test implementation program for conducting tests in a vehicle that can participate in public road traffic, configured to cause at least one processor to: prepare in advance one or more test patterns related to a driving handover between autonomous driving by the vehicle's driving system and driving by a driver; and test the test patterns related to the driving handover in a situation in which a driving handover from the driving system to the driver occurs.

[0010] According to these aspects, since test patterns for driver handover are prepared in advance, when a situation arises in which a handover from the driving system to the driver occurs, the test patterns can be executed smoothly. In other words, it is possible to not overlook driver handover events that occur in vehicles that can participate in public road traffic and to use them for testing test patterns, thereby facilitating V&V for driver handovers and making it easier to appropriately improve the vehicle system specifications.

[0011] Note that the symbols in parentheses included in the claims etc. are intended to exemplify the correspondence with the parts of the embodiments described below, and are not intended to limit the technical scope.

[0012] 1 is a diagram showing an example of the hardware configuration of an operation system, etc.; FIG. 1 is a diagram showing the functional configuration of an operation system; FIG. 2 is a diagram showing an example of the configuration of a cockpit; FIG. 3 is a diagram showing an overview of a management system; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A diagram showing a specific example of a driver changeover pattern; A flowchart illustrating a test-related process; A diagram showing a specific example of a driver changeover pattern; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A diagram showing an example of the hardware configuration of a processing system.

[0013] Hereinafter, several embodiments will be described with reference to the drawings. Note that corresponding components in each embodiment are given the same reference numerals, and redundant description may be omitted. When only a portion of the configuration is described in each embodiment, the configuration of another embodiment described previously can be applied to the remaining portion of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of several embodiments can also be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.

[0014] (Explanation of Terms) Terms related to the disclosure of this specification are explained below. This explanation is included in the embodiments of the specification.

[0015] A road user may be a traffic participant on or adjacent to an active road for the purpose of traveling from one location to another.

[0016] A dynamic driving task (DDT) may be a real-time operational and tactical function for operating a vehicle in traffic, and a DDT may be all real-time operational and tactical functions for operating a vehicle on a roadway.

[0017] An automated driving system (ADS) may be a collection of hardware and software capable of executing the entire DDT on a continuous basis, whether or not it is limited to a specific operational design domain.

[0018] An ADS feature may be the design-specific functionality of the ADS within a particular ODD at a given level of autonomous driving.

[0019] DDT fallback may be a response by a driver or automated system to either perform DDT or transition to a minimal risk state after a failure occurs or upon detection of a malfunction or potentially dangerous behavior. DDT fallback may also be a method of controlling the transition from autonomy to driver or other system control using takeover / fallback states and associated use cases. DDT fallback may also be a user response to perform DDT or achieve MRC after a system failure related to DDT performance occurs or upon deviation from ODD, or a response by an ADS to achieve MRC given the same circumstances.

[0020] A Minimal Risk Condition (MRC) may be a state of the vehicle to reduce the risk if a given trip cannot be completed, or may be a stable, stopped state that a user or ADS places on the vehicle after a DDT fallback is performed to reduce the risk of an accident if a given trip cannot or should not be continued.

[0021] An operational design domain (ODD) may be the specific conditions in which a given (automated) driving system is designed to function, or the operating conditions in which a given ADS or its functions are specifically designed to function, including, but not limited to, environmental, geographic, time-of-day restrictions, and / or the presence or absence of requirements for certain traffic and road characteristics.

[0022] Safety of the intended functionality (SOTIF) may be the absence of undue risk due to insufficient functionality of the intended functionality or its implementation.

[0023] A driving policy may be a strategy and rules that define control behavior at the vehicle level.

[0024] A scenario may be a description of the temporal relationships between several scenes in a sequence of scenes, including the goals and values ​​in a specific situation influenced by actions and events, and a description of a continuous time sequence of activities that integrates a subject vehicle, all its external environments, and their interactions in the process of performing a specific driving task.

[0025] A Minimal Risk Maneuver (MRM) may be a vehicle movement commanded by the automated driving system during DDT fallback to achieve an MRC.

[0026] A safety-relevant object may be any dynamic or static object that may be relevant to the safety performance of the DDT.

[0027] Reasonably foreseeable may be technically reliable and have a reliable or measurable rate of occurrence.

[0028] A triggering condition may be a specific condition of a scenario that acts as a catalyst for subsequent system responses that contribute to unsafe behavior, failure to prevent, detect, and mitigate reasonably foreseeable indirect misuse.

[0029] Risk acceptance criteria / criterion are standards that represent the absence of unreasonable levels of risk, and may be, for example, physical parameters that define when a particular behavior is considered undesirable, a maximum number of accidents per hour, as low as reasonably practicable, etc.

[0030] A proper response may be an action that is significant to avoid or ameliorate a dangerous situation in a reasonably foreseeable scenario in which other safety-related objects are operating within expected bounds.

[0031] A safety-related model may be a representation of safety-related aspects of driving behavior based on assumptions about the reasonably foreseeable behavior of other road users. A safety-related model may be an on-board or off-board safety verification or analysis device, a mathematical model, a more conceptual set of rules, a set of scenario-based behaviors, or a combination of these.

[0032] A formal model may be a model expressed in a formal notation that is used to verify system performance.

[0033] A safety envelope may be a set of limits and conditions within which an (automated) driving system is designed to operate, subject to constraints or controls, in order to maintain operation within an acceptable level of risk. A safety envelope may be a general concept that can be used to accommodate all principles to which a driving policy can adhere, according to which an ego-vehicle operated by an (automated) driving system may have one or more boundaries around it.

[0034] Verification may be an activity to determine that operation of a vehicle equipped with an (autonomous) driving system achieves the safety of a defined (autonomous) driving system application in the intended environment.

[0035] Validation may be an activity to determine that a test object meets specified requirements.

[0036] A positive risk balance may be a criterion that demonstrates that a technical solution achieves an acceptable level of residual risk.

[0037] Object and event detection and response (OEDR) may be a subtask of DDT that involves monitoring the driving environment and executing appropriate responses to such objects and events.

[0038] Response time may be the time it takes a road user in a given scenario to perceive a particular stimulus and begin to execute a response (braking, steering, accelerating, stopping, etc.).

[0039] A vulnerable road user (VRU) may be a road user not in a vehicle such as a passenger car, public transport, train, etc. A vulnerable road user may also be an unprotected road user such as a cyclist, motorcyclist, pedestrian, person with a disability, or person with reduced mobility and orientation.

[0040] (First embodiment) <Driving system> The driving system 2 of the first embodiment shown in FIG. 1 realizes functions related to driving a vehicle 1. The driving system 2 may be a vehicle system itself, or may be a component that constitutes part of a vehicle system. Part or all of the driving system 2 is mounted on the vehicle 1. This vehicle 1 may be referred to as a subject vehicle, a host vehicle, or the like. The vehicle 1 may be configured to be able to communicate with other vehicles, etc., directly or indirectly via a communication infrastructure. The other vehicles may be referred to as target vehicles.

[0041] The vehicle 1 may be a road user capable of manual driving, such as a four-wheeled automobile or truck. The vehicle 1 may also be capable of automated driving. Autonomous driving may also be referred to as autonomous driving by the driving system 2. Driving is classified into levels according to the extent to which a human driver performs all dynamic driving tasks (DDTs). Automation levels are specified, for example, in SAE J3016. At levels 0 to 2, the driver performs some or all of the DDTs. Levels 0 to 2 may be classified as so-called manual driving. Level 0 indicates that driving is not automated. Level 1 indicates that the driving system 2 assists the driver. Level 2 indicates that driving is partially automated.

[0042] At levels 3 and above, while the ADS feature is activated, the driving system 2 performs all of the DDT. Levels 3 to 5 may be classified as so-called automated driving. A system capable of driving at level 3 or above may be called an automated driving system (ADS). A vehicle equipped with an automated driving system or a vehicle capable of driving at level 3 or above may be called an automated vehicle (AV).

[0043] Level 3 indicates conditional automation of driving. A level 3 automated driving system performs DDT but does not perform DDT fallback. That is, DDT fallback is performed by a driver who is ready for fallback. Level 4 indicates highly automated driving. A level 4 automated driving system performs DDT and DDT fallback. A level 4 automated driving system can hand over DDT to the driver after reaching a minimal risk condition (MRC) by performing DDT fallback, etc. Handing over DDT between the driving system 2 and a human driver is also called driving handover or delegation of authority. Level 5 indicates fully automated driving.

[0044] The conditions for executing level 3 and level 4 autonomous driving may include some or all of the conditions indicated by the operational design domain (ODD). For example, the ADS function may be defined within the scope of the ODD. The driving system 2 described in this embodiment is a driving system capable of executing level 3 or higher autonomous driving. That is, the driving system 2 may be capable of executing autonomous driving up to level 3, up to level 4, or even level 5 autonomous driving.

[0045] The driving system 2 provides functions such as automated driving to a vehicle user of a vehicle 1 that can participate in public road traffic. For example, the vehicle user may be a driver riding in the vehicle 1. The vehicle user may be a passenger riding in the vehicle 1. For example, if the vehicle 1 is a POV (Personally Owned Vehicle), the vehicle user may be the owner of the vehicle 1. For example, if the vehicle 1 is used for MaaS (Mobility as a Service), the vehicle user may be an operator such as an operations manager that manages the operation of the vehicle 1.

[0046] The architecture of the driving system 2 is selected to enable an efficient safety of the intended functionality (SOTIF) process. For example, the architecture of the driving system 2 may be configured based on a sense-plan-act model. The sense-plan-act model includes a sense element, a plan element, and an act element as major system elements. The sense element, plan element, and act element interact with each other. Here, sense may be replaced with perception, plan with determine, and act with control, respectively.

[0047] At the technical level (i.e., from a technical perspective), the driving system 2 implements at least a plurality of sensors 40 corresponding to sensing functions, at least one processing system 50 corresponding to planning functions, and a plurality of motion actuators 60 corresponding to acting functions. At the functional level (i.e., from a functional perspective), the sensing, planning, and acting functions are implemented (see also FIG. 2 ).

[0048] In detail, a detection unit 10 serving as a processing unit for realizing a detection function may be constructed in the driving system 2, mainly consisting of a plurality of sensors 40, a processing system 50 that processes detection information from the plurality of sensors 40, and the processing system 50 that generates an environmental model based on information from the plurality of sensors 40. A planning unit 20 and a risk confirmation unit 26 serving as processing units for realizing a planning function may be constructed in the driving system 2, mainly consisting of a plurality of motion actuators 60 and at least one processing system 50 that outputs operation signals for the plurality of motion actuators 60.

[0049] Here, the detection unit 10 may be realized in the form of a detection system serving as a subsystem provided so as to be distinguishable from the planner 20 and the action unit 30. The planner 20 may be realized in the form of a planning system serving as a subsystem provided so as to be distinguishable from the detection unit 10 and the action unit 30. The planning system may include a risk confirmation function. The risk confirmation function may be mounted in the operation system 2 independently of the detection unit 10, the planner 20, and the action unit 30. The action unit 30 may be realized in the form of an action system serving as a subsystem provided so as to be distinguishable from the detection unit 10 and the planner 20. The detection system, the planning system, and the action system may constitute components independent of each other. The subsystem referred to here may be replaced with a module, a unit, a device, etc.

[0050] The detection unit 10 is responsible for detection functions, including localization (e.g., location estimation) of road users such as the vehicle 1 and other vehicles. The detection unit 10 detects the external environment, internal environment, vehicle state, and the state of the driving system 2 of the vehicle 1. The detection unit 10 fuses the detected information to generate an environmental model. The environmental model may also be referred to as a world model. The planner 20 applies the objective and driving policy to the environmental model generated by the detection unit 10 to derive control actions. The behavior unit 30 executes the control actions derived by the planner 20.

[0051] <Physical Architecture> An example of the physical architecture of the driving system 2 will be described using Figure 1. The driving system 2 includes a plurality of sensors 40, a plurality of motion actuators 60, a plurality of HMI devices 70, and at least one processing system 50. These components can communicate with each other via one or both of wireless and wired connections. These components may also be able to communicate with each other through an in-vehicle network such as CAN (registered trademark).

[0052] The plurality of sensors 40 includes one or more external environment sensors 41. Furthermore, the plurality of sensors 40 may include at least one of one or more internal environment sensors 42, one or more communication systems 43, and a map database (DB) 44.

[0053] The external environment sensor 41 may detect targets present in the external environment of the vehicle 1. Target detection type external environment sensors 41 include, for example, a camera, LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging), laser radar, millimeter wave radar, ultrasonic sonar, acoustic sensor, etc. Typically, a combination of multiple types of external environment sensors 41 may be implemented to monitor the front, sides, and rear directions of the vehicle 1.

[0054] Furthermore, the external environment sensor 41 may detect atmospheric conditions and weather conditions in the environment outside the vehicle 1. The condition detection type external environment sensor 41 is, for example, an outside air temperature sensor, a temperature sensor, a raindrop sensor, or the like.

[0055] The internal environment sensor 42 may detect a specific physical quantity related to vehicle motion (hereinafter referred to as a motion physical quantity) in the internal environment of the vehicle 1. The motion physical quantity detection type internal environment sensor 42 is, for example, a speed sensor, an acceleration sensor, a gyro sensor, etc. The internal environment sensor 42 may detect the state of an occupant in the internal environment of the vehicle 1. The occupant detection type internal environment sensor 42 is, for example, an actuator sensor, a sensor and its system for monitoring a vehicle user (e.g., a driver) in the vehicle cabin (hereinafter referred to as an interior monitor), a biological sensor, a seating sensor, an in-vehicle equipment sensor, etc. Here, the actuator sensor in particular is, for example, an accelerator sensor, a brake sensor, a steering sensor, etc., which detect the state of an occupant's operation of a motion actuator 60 related to the motion control of the vehicle 1.

[0056] The communication system 43 obtains communication data usable in the driving system 2 via wireless communication. The communication system 43 may receive positioning signals from artificial satellites of a global navigation satellite system (GNSS) that exist in the external environment of the vehicle 1. The positioning type communication device in the communication system 43 is, for example, a GNSS receiver.

[0057] The communication system 43 may transmit and receive communication signals to and from an external system (e.g., a server 96) present in the external environment of the vehicle 1. Examples of V2X-type communication devices in the communication system 43 include dedicated short range communications (DSRC) communication devices, cellular V2X (C-V2X) communication devices, etc. Examples of communication with a V2X system present in the external environment of the vehicle 1 include communication with a communication system of another vehicle (V2V), communication with infrastructure equipment such as a communication device installed in a traffic light or a roadside device (V2I), communication with a mobile terminal of a pedestrian (V2P), and communication with a network such as a cloud server (V2N). The architecture of V2X communication, including V2I communication, may be an architecture defined in ISO 21217, ETSI TS 102 940-943, IEEE 1609, etc.

[0058] Furthermore, the communication system 43 may transmit and receive communication signals to and from a mobile terminal 91, such as a smartphone, present inside the vehicle 1. Examples of terminal communication type communication devices in the communication system 43 include Bluetooth (registered trademark) devices, Wi-Fi (registered trademark) devices, infrared communication devices, etc. Furthermore, if the vehicle user's mobile terminal 91 is associated with the vehicle 1 in advance, the communication system 43 may transmit and receive communication signals to and from the mobile terminal present in the external environment.

[0059] The map DB 44 is a database that stores map data available to the driving system 2. The map DB 44 includes at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The map DB 44 may include a database of a navigation unit that navigates the vehicle 1 along a route to a destination. The map DB 44 may include a database of probe data (PD) maps generated using probe data (PD) collected from each vehicle. The map DB 44 may include a database of high-precision maps with a high level of accuracy that are primarily used in automated driving systems. The map DB 44 may also include a database of parking lot maps that include detailed parking lot information, such as parking space information, that is used in automated parking or parking assistance applications.

[0060] The map DB 44 suitable for the driving system 2 acquires and stores the latest map data, for example, by communicating with a map server via a V2X communication system 43. The map data is data representing the external environment of the vehicle 1, and is converted into two-dimensional or three-dimensional data. The map data may include road data representing at least one of the position coordinates, shape, road surface condition, and standard running route of a road structure. The map data may also include marking data representing at least one of the position coordinates and shape of road signs, road markings, and lane markings attached to a road. The marking data included in the map data may represent landmarks such as traffic signs, arrow markings, lane markings, stop lines, directional signs, landmark beacons, business signs, and changes in road line patterns. The map data may also include structure data representing at least one of the position coordinates and shape of buildings and traffic lights facing the road. The marking data included in the map data may represent landmarks such as street lights, road edges, reflectors, and poles.

[0061] The motion actuator 60 can control vehicle motion based on an input control signal. The drive-type motion actuator 60 is, for example, a power train including at least one of an internal combustion engine, a drive motor, etc. The braking-type motion actuator 60 is, for example, a brake actuator. The steering-type motion actuator 60 is, for example, a steering.

[0062] As shown in FIG. 3 , a plurality of HMI (Human Machine Interface) devices 70 may be mounted on the vehicle 1. The HMI devices 70 realize human-machine interaction, which is an interaction between a user of the vehicle 1 and the driving system 2. Of the plurality of HMI devices 70, a portion that realizes an operation input function by an occupant may be part of the detection unit 10. Of the plurality of HMI devices 70, a portion that realizes an information presentation function may be part of the behavior unit 30. On the other hand, the function realized by the HMI device 70 may be positioned as a function independent of the detection function, the planning function, and the behavior function.

[0063] The HMI device 70 may be an operation input device 70a that can input user operations to transmit the will or intention of the user of the vehicle 1 to the driving system 2. Examples of the operation input type HMI device 70 include an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a turn signal lever, a mechanical switch, and a touch panel of a navigation unit. Of these, the accelerator pedal controls the powertrain as a motion actuator 60. The brake pedal controls a brake actuator as a motion actuator 60. The steering wheel controls a steering actuator as a motion actuator 60.

[0064] The HMI device 70 may further include a feedback device 70a1 as the operation input device 70a, which receives feedback from the vehicle user. For example, the feedback device 70a1 may include a computer and a microphone. When a feedback function is selected in a CID 70b2 (described later), the feedback device 70a1 records the vehicle user's voice using the microphone for a predetermined period of time (e.g., 45 seconds). This allows the vehicle user to provide feedback such as praise or dissatisfaction about the vehicle 1 or the driving system 2. The feedback device 70a1 may then transmit the recorded vehicle user voice data to an external system in the external environment via the communication system 43. The external system may be a server 96 (described later). Collecting the vehicle user's feedback in the external system can be useful for improving the vehicle 1 or the driving system 2. The feedback device 70a1 may store the vehicle user's voice data and the transmission record to the external system in a storage medium 55c in response to a write command to the recording device 55.

[0065] The HMI device 70 may be an information presentation device 70b that presents information such as visual information, auditory information, and cutaneous information to the user of the vehicle 1. The HMI device 70 of the visual information presentation type, i.e., the information presentation device 70b, is, for example, a meter display 70b1, a navigation unit, a CID (center information display) 70b2, a HUD (head-up display) 70b3, an illumination unit, etc.

[0066] The meter display 70b1 is a display device that is arranged, for example, in a driver-facing portion of the instrument panel that faces the driver's seat. The meter display 70b1 displays to the driver information necessary for driving, focusing on the vehicle status including the speed of the vehicle 1. The meter display 70b1 may be a graphic meter that displays all information using images, or may be a combination meter that combines an image display with an analog display using a device.

[0067] The CID 70b2 is a display device disposed, for example, in the center of the instrument panel. The CID 70b2 has the largest display screen of all the in-vehicle display devices mounted on the instrument panel. The CID 70b2 is capable of displaying images not only to the driver but also to passengers. The CID 70b2 may include a touch panel that can be operated by the vehicle user. In this case, the CID 70b2 also corresponds to the operation input device 70a.

[0068] The HUD 70b3 is a display device arranged on the instrument panel on the opposite side of the driver's seat from the meter display 70b1, i.e., on the rear side of the area facing the driver's seat. The HUD 70b3 projects an image onto the front windshield of the vehicle 1, thereby displaying a virtual image VI that the driver can visually recognize as floating outside the vehicle.

[0069] Furthermore, instead of the CID 70b2 and the meter display 70b1, a pillar-to-pillar display arranged to cross the left and right A-pillars may be employed. Even in this case, the pillar-to-pillar display may be divided into several screens, each of which may be controlled in the same manner as the CID 70b2 and the meter display 70b1.

[0070] In addition, an information presentation device 70b (hereinafter referred to as a passenger display) of a visual information presentation type that mainly targets passengers other than the driver may be provided. For example, among pillar-to-pillar displays, a screen disposed opposite the passenger seat and a display installed in the rear seat correspond to passenger displays.

[0071] The auditory information presentation type HMI device 70 is, for example, a speaker, a buzzer, etc. The tactile information presentation type HMI device 70 is, for example, a steering wheel vibration unit, a driver's seat vibration unit, a steering wheel reaction force unit, an accelerator pedal reaction force unit, a brake pedal reaction force unit, an air conditioning unit, etc.

[0072] Furthermore, the HMI device 70 may realize an HMI function linked to a mobile terminal 91 such as a smartphone by mutually communicating with the terminal through the communication system 43. For example, as an alternative means for presenting information by the HMI device 70, information from the driving system 2 may be displayed on the screen of the vehicle user's smartphone through the communication system 43. On the other hand, the HMI device 70 may present information acquired from the smartphone to the vehicle user. Also, for example, an operation input to a smartphone may be an alternative means for inputting an operation to the HMI device 70.

[0073] At least one processing system 50 is provided. For example, the processing system 50 may be an integrated processing system that integrally executes processing related to the detection function, processing related to the planning function, and processing related to the action function. In this case, the integrated processing system 50 may further execute processing related to the HMI device 70, or a processing system dedicated to the HMI may be provided separately. For example, the processing system dedicated to the HMI may be an integrated cockpit system that integrally executes processing related to each HMI device 70. The processing system 50 may be provided by an in-vehicle platform that can be used generally for AVs.

[0074] For example, the processing system 50 may be configured to have at least one processing unit corresponding to processing related to the detection function, at least one processing unit corresponding to processing related to the planning function, and at least one processing unit corresponding to processing related to the behavioral function.

[0075] The processing system 50 has a communication interface to the outside and is connected to at least one type of element related to processing by the processing system 50, such as each sensor 40, motion actuator 60, and HMI device 70, via at least one type of interface, such as a LAN (Local Area Network), a wire harness, an internal bus, or a wireless communication circuit.

[0076] The processing system 50 includes a main unit 51 mainly composed of at least one dedicated computer. The processing system 50 may realize functions such as a detection function, a planning function, and an action function by the main unit 51 which is a combination of multiple dedicated computers. The main unit 51 may be referred to as an operation control device.

[0077] For example, the dedicated computer constituting the main unit 51 may be an integrated ECU that integrates the driving functions of the vehicle 1. The dedicated computer constituting the main unit 51 may be a determination ECU that determines DDT. The dedicated computer constituting the main unit 51 may be a monitoring ECU that monitors the driving of the vehicle 1. The dedicated computer constituting the main unit 51 may be an evaluation ECU that evaluates the driving of the vehicle 1. The dedicated computer constituting the main unit 51 may be a navigation ECU that navigates the driving route of the vehicle 1.

[0078] Furthermore, the dedicated computer constituting the main unit 51 may be a locator ECU that estimates the position of the vehicle 1. The dedicated computer constituting the main unit 51 may be an image processing ECU that processes image data detected by the external environment sensor 41. The dedicated computer constituting the main unit 51 may be an actuator ECU that controls the motion actuator 60 of the vehicle 1. The dedicated computer constituting the main unit 51 may be an HCU (HMI Control Unit) that comprehensively controls the HMI device 70. The dedicated computer constituting the main unit 51 may include at least one external computer provided in an external center or mobile terminal 91 that can communicate via the communication system 43, for example.

[0079] The dedicated computer constituting the main unit 51 has at least one memory 51a and one processor 51b. The memory 51a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data that can be read by the processor 51b. Furthermore, the memory 51a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 51b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0080] The dedicated computer constituting the main unit 51 may be a SoC (System on a Chip) that integrates the memory 51a, processor 51b, and interface into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0081] Furthermore, the processing system 50 may include at least one database for executing the DDT, which may include at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, and an interface for accessing the storage medium.

[0082] The database may be a scenario database (hereinafter referred to as a scenario DB) 59. The database may be a rule database (hereinafter referred to as a rule DB) 58. At least one of the scenario DB 59 and the rule DB 58 may be configured integrally with the main unit 51. At least one of the scenario DB 59 and the rule DB 58 may not be provided in the processing system 50, but may be provided independently in the operation system 2. At least one of the scenario DB 59 and the rule DB 58 may be provided in an external system present in the external environment, and configured to be accessible from the processing system 50 via the communication system 43.

[0083] The scenario DB 59 has a scenario catalog in which multiple scenarios used for driving the vehicle 1 are stored. The driving system 2 can, for example, apply a situation in which the vehicle 1 is placed to one scenario selected from the multiple scenarios or a combination of multiple scenarios. The scenario DB 59 may store multiple scenarios including at least one of a functional scenario, a logical scenario, and a concrete scenario. A functional scenario defines a top-level qualitative scenario structure. A logical scenario is a scenario in which quantitative parameter ranges are assigned to a structured functional scenario. A concrete scenario defines a safety judgment boundary that distinguishes between a safe state and an unsafe state.

[0084] The rule DB 58 stores a rule set used for driving the vehicle 1. The rule set may include multiple rules. The rule set may further include a priority structure for the rules, which is set based on the relative importance of the multiple rules. The rule set may be an implementation of guidelines for strategic driving of the vehicle 1.

[0085] The plurality of rules may include rules based on laws, regulations, or a combination thereof. The plurality of rules may include rules based on preferences that are not influenced by laws, regulations, or the like. The plurality of rules may include rules based on exercise behavior based on past experience. The plurality of rules may include rules based on characterization of the exercise environment. The plurality of rules may include rules based on ethical concerns. The plurality of rules may include rules based on basic principles of a safety model (e.g., the five principles of the RSS model). The plurality of rules may include traffic rules. The traffic rules may be rules specified in the Road Traffic Act or may be rules based on national or local customs.

[0086] The rules such as traffic rules stored in the rule DB 58 may be positioned as information provided from the detection unit 10 to the planning unit 20 by the detection function, similar to the map information acquired from the map DB 44 .

[0087] The processing system 50 may also include at least one recording device 55 that records at least one of the detection information, planning information, and behavioral information of the driving system 2. The recording device 55 sequentially records event data related to the driving task of the vehicle 1. The event data is data that records events encountered by the vehicle 1. The event data may include at least one of various types of information related to the driving task, such as information about the operation of the motion actuators 60, information about the route or trajectory traversed or planned by the vehicle 1, information about a scenario encountered by the vehicle 1, information about the automation level or delegation of authority of the vehicle 1, and information about the execution of a DDT fallback or a minimal risk maneuver (MRM) by the vehicle 1.

[0088] The recording device 55 may include at least one large-capacity storage medium 55c. The storage medium 55c may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The storage medium 55c may be mounted on a board in a form that is not easily removable or replaceable, such as an embedded multimedia card (eMMC) using flash memory. At least one of the storage media 55c may be removable and replaceable from the recording device 55, such as an SD card.

[0089] At least one of the recording device 55 and the storage medium 55c may correspond to an EDR (Event Data Recorder) or a DSSAD (Data Storage System for Automated Driving). The recording device 55 may have a function for selecting information to be recorded from the event data. In this case, the recording device 55 may have a dedicated computer.

[0090] The dedicated computer provided in the recording device 55 has at least one memory 55a and one processor 55b. The memory 55a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data that can be read by the processor 55b. Furthermore, the memory 55a may be a rewritable volatile storage medium, such as a random access memory (RAM). The processor 55b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0091] The dedicated computer may be a SoC (System on a Chip) in which the memory 55a, processor 55b, and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0092] The recording device 55 may access the storage medium 55c and perform recording in accordance with a data write command from each part of the driving system 2. The recording device 55 may determine information transmitted over the in-vehicle network, and, based on the judgment of the processor 55b provided in the recording device 55, access the storage medium 55c and perform recording.

[0093] Furthermore, the recording device 55 may not be provided in the processing system 50, but may be provided independently in the operation system 2. The recording device 55 may be provided in an external system present in the external environment, and configured to be accessible from the processing system 50 via the communication system 43.

[0094] Furthermore, the processing system 50 may include at least one risk confirmation unit 53. The risk confirmation unit 53 may be one aspect of on-board implementation of RSS (Responsibility Sensitive Safety) as a safety model. The risk confirmation unit 53 may be an on-board checker for the planning function realized by a dedicated computer. The risk confirmation unit 53 realizes the risk confirmation section 26, which realizes the risk confirmation function, by hardware independent of the planning section 20.

[0095] The risk confirmation unit 53 may be primarily configured as a dedicated computer having at least one memory 53a and one processor 53b. The memory 53a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data readable by the processor 55b. Furthermore, the memory 55a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 55b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0096] The dedicated computer may be a SoC (System on a Chip) in which the memory 53a, processor 53b, and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0097] As described above, the processing system 50 includes memories 51a, 53a, and 55a storing software. The processors 51b, 53b, and 55b are configured to operate the software to realize automated driving, allowing authority to be transferred between the system itself and the user. The software here may include the computer program itself used in the driving system 2. The software here may include an algorithm in the computer program used in the driving system 2. The software here may include parameters in the computer program used in the driving system 2. The software here may include a trained model, sometimes referred to as AI, implemented by, for example, a neural network, used in the driving system 2. Furthermore, the software may include data stored in a database referenced by the processing system 50, data stored in the map DB 44, and the like. One piece of software may correspond to one application or multiple applications, may be part of one application, or may be software commonly used by multiple applications.

[0098] Furthermore, the processing system 50 may include at least one software management unit 57. The software management unit 57 implements software management functions. The software management unit 57 may be referred to as a vehicle software management device or a test execution device that implements the function of implementing tests, which will be described later.

[0099] The software management unit 57 manages various software used in the processing system 50, such as the main unit 51, the risk confirmation unit 53, the recording device 55, the rule DB 58, and the scenario DB 59. The software management unit 57 may further manage software used in the driving system 2 outside the processing system 50. For example, the software management unit 57 may manage data stored in the map DB 44, software used for drawing processing by the information presentation device 70b, software used for communication processing by the communication system 43, etc.

[0100] Software management may include software version management, download and installation processes, uninstallation processes, etc. Software management may also include software testing.

[0101] The software management unit 57 may be configured primarily as a dedicated computer having at least one memory 57a and one processor 57b to realize the software management function. The memory 57a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data readable by the processor 57b. Furthermore, the memory 57a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 57b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0102] The dedicated computer may be a SoC (System on a Chip) in which the memory 57a, processor 57b, and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0103] <Logical Architecture in Autonomous Driving> Next, an example of a logical architecture in the driving system 2 will be described using Figure 2. The description here will focus on processing by a computer program executed during autonomous driving at level 3 or higher. The detection unit 10 may include an environment recognition unit 11, a self-location recognition unit 12, and an internal recognition unit 13 as processing units for realizing sub-functions obtained by further classifying the detection function by the processor 51b executing a computer program.

[0104] The environment recognition unit 11 individually processes information (sometimes referred to as sensor data) related to the external environment acquired from each sensor 40, and realizes a function of recognizing the external environment including targets, other road users, etc. The environment recognition unit 11 individually processes the sensor data detected by each external environment sensor 41. The sensor data may be sensor data provided by, for example, millimeter-wave radar, sonar, LiDAR, etc. The environment recognition unit 11 may generate relative position data including the direction, size, and distance of an object relative to the vehicle 1 from the raw data detected by the external environment sensors 41.

[0105] The sensor data may be image data provided by, for example, a camera, LiDAR, or the like. The environment recognition unit 11 processes the image data and extracts objects reflected within the angle of view of the image. The object extraction may include estimating the direction, size, and distance of the object relative to the vehicle 1. The object extraction may also include classifying the object using, for example, semantic segmentation.

[0106] Furthermore, the environment recognition unit 11 processes information acquired through the V2X function of the communication system 43. The environment recognition unit 11 processes information acquired from the map DB 44.

[0107] The environment recognition unit 11 may be further divided into a plurality of sensor recognition units each optimized for one sensor group. When a sensor recognition unit is associated with recognizing information from one sensor group, the sensor recognition unit may fuse information from the one sensor group.

[0108] The self-location recognition unit 12 performs localization of the vehicle 1. The self-location recognition unit 12 acquires global position data of the vehicle 1 from the communication system 43 (e.g., a GNSS receiver). In addition, the self-location recognition unit 12 may acquire position information of targets extracted by the environment recognition unit 11. The self-location recognition unit 12 also acquires map information from the map DB 44. The self-location recognition unit 12 integrates this information to estimate the position of the vehicle 1 on the map.

[0109] The internal recognition unit 13 processes sensor data detected by each internal environment sensor 42 and realizes a function of recognizing the vehicle state. The vehicle state may include the state of the physical quantities of motion of the vehicle 1 detected by a speed sensor, an acceleration sensor, a gyro sensor, etc. The vehicle state may also include at least one of the user state, the user's operation state of the motion actuator 60, and the switch state of the HMI device 70.

[0110] The planning unit 20 may include a prediction unit 21, an operation planning unit 22, and a mode management unit 23 as processing units for realizing sub-functions that are further classified into planning functions by having processors 51b, 53b execute computer programs.

[0111] The prediction unit 21 acquires information on the external environment recognized by the environment recognition unit 11 and the self-position recognition unit 12, the vehicle state recognized by the internal recognition unit 13, etc. The prediction unit 21 may interpret the environment based on the acquired information and estimate the current situation of the vehicle 1. The situation here may be an operational situation or may include the operational situation.

[0112] The prediction unit 21 may interpret the environment and predict the behavior of an object, such as another road user. The object may be a safety-relevant object. The behavior prediction may include at least one of predicting the object's speed, predicting the object's acceleration, and predicting the object's trajectory. The behavior prediction may be performed based on reasonably foreseeable assumptions. Furthermore, the prediction unit 21 may infer the user's intention based on the predicted behavior, predicted potential hazards, and the acquired vehicle state.

[0113] The driving planning unit 22 plans autonomous driving of the vehicle 1 based on at least one of the estimated information of the vehicle 1's position on a map by the self-position recognition unit 12, the prediction information and user intention estimation information by the prediction unit 21, and the function constraint information by the mode management unit 23.

[0114] The driving planner 22 realizes a route planning function, a behavior planning function, and a trajectory planning function. The route planning function is a function of planning at least one of a route to a destination and a mid-range lane plan based on estimated information about the position of the vehicle 1 on a map. The route planning function may further include a function of determining at least one of a lane change request and a deceleration request based on the mid-range lane plan. Here, the route planning function may be a mission / route planning function in a strategic function, and may be a function of outputting a mission plan and a route plan.

[0115] The behavior planning function is a function that plans the behavior of the vehicle 1 based on at least one of the route to the destination planned by the route planning function, a mid-distance lane plan, a lane change request and a deceleration request, prediction information and user intention estimation information by the prediction unit 21, and function constraint information by the mode management unit 23. The behavior planning function may include a function that generates conditions related to state transitions of the vehicle 1. The conditions related to state transitions of the vehicle 1 may correspond to triggering conditions. The conditions related to state transitions may include fallback conditions for executing DDT fallbacks.

[0116] The behavior planning function may include a function for determining a state transition of an application that realizes the DDT and further a state transition of a driving action based on this condition. As a result, the driving planner 22 plans the execution of the DDT fallback, and if this does not involve authority delegation, the driving planner 22 may further execute the MRM together with the motion control unit 31 to transition the vehicle 1 to a minimum risk state. The MRM plan may be realized by the behavior planning function or the trajectory planning function.

[0117] The behavior planning function may also include a function for determining, based on the information on these state transitions, longitudinal constraints on the path of the vehicle 1 and lateral constraints on the path of the vehicle 1. The behavior planning function may be a tactical behavior plan in the DDT function, and may output tactical behavior.

[0118] The trajectory planning function is a function that plans a driving trajectory of the vehicle 1 based on the judgment information by the prediction unit 21, longitudinal constraints on the path of the vehicle 1, and lateral constraints on the path of the vehicle 1. The trajectory planning function may include a function that generates a path plan. The path plan may include a speed plan, or the speed plan may be generated as a plan independent of the path plan. The trajectory planning function may include a function that generates multiple path plans and selects an optimal path plan from the multiple path plans, or a function that switches between path plans. The trajectory planning function may further include a function that generates backup data of the generated path plan. The trajectory planning function may be a trajectory planning function in the DDT function, and may output a trajectory plan.

[0119] The mode management unit 23 monitors the driving system 2 and sets constraints on driving-related functions. The mode management unit 23 may manage the autonomous driving mode, for example, the state of the automation level. The management of the automation level may include management of switching between manual driving and autonomous driving, i.e., management of the transfer of authority between the user and the driving system 2, in other words, management of the takeover of driving. The mode management unit 23 may monitor the state of the subsystem related to the driving system 2 and determine a system malfunction (e.g., an error, an unstable operation state, a system failure, or a malfunction). The mode management unit 23 may determine a mode based on the user's intention based on the user's intention estimation information generated by the internal recognition unit 13. The mode management unit 23 may set constraints on driving-related functions based on at least one of the system malfunction determination result, the mode determination result, the vehicle state determined by the internal recognition unit 13, the sensor abnormality (or sensor failure) signal output from the sensor 40, the application state transition information and the trajectory plan determined by the driving planner 22, etc.

[0120] Furthermore, the mode management unit 23 may have a comprehensive function of determining, in addition to constraints on driving functions, longitudinal constraints on the path of the vehicle 1 and lateral constraints on the path of the vehicle 1. In this case, the operation planning unit 22 plans behavior and trajectories in accordance with the constraints determined by the mode management unit 23.

[0121] When the automation level is switched to level 2 or lower, the mode management unit 23 may control the enablement state of the driving assistance application according to the automation level.

[0122] When the risk confirmation function is implemented as part of the planner 20, the risk confirmation function may be implemented as part of the functions realized by the predictor 21, the operation planner 22, and the mode manager 23. On the other hand, the risk confirmation function may be implemented as a function independent of the planner 20.

[0123] The risk confirmation function is a function that acquires an environmental model, sensor data, etc. from the detection unit 10, evaluates a risk based on this information, and outputs a response based on the risk to the behavior unit 30 or the operation planning unit 22. This series of functions or processes may be referred to as risk confirmation or risk monitoring.

[0124] More specifically, the risk confirmation function outputs a situation based on information acquired from the detection unit 10. The risk confirmation function confirms whether the situation is safe or dangerous. This confirmation may include confirmation of an estimated result of a collision risk between the vehicle 1 and a surrounding object. In this confirmation, an index such as a collision probability may be used to take uncertainty into account. The risk confirmation function may compare an acceptable collision risk threshold with the estimated collision risk value to determine whether the situation is dangerous. The acceptable collision risk threshold may be set in advance based on risk acceptance criteria / criterion, which will be described in detail later.

[0125] The risk confirmation function derives a proper response based on the result of this confirmation. The proper response may be provided to the behavior unit 30 or the driving plan unit 22 only if the situation is determined to be a dangerous situation. The proper response may be a restriction on the control command of the motion actuator 60. The proper response may be a response to return the vehicle 1 to a safe state.

[0126] The risk confirmation function is realized by implementing a safety model. The safety model may be referred to as a safety-related model. The safety model may be a formal model. For example, an RSS model may be adopted as the safety model, but other models such as an SFF model, a more generalized model, or a composite model combining multiple models may also be adopted. SFF stands for Safety Force Field. The safety model monitors the risk of the vehicle 1 based on a driving policy. Risk monitoring can also be said to be monitoring of the driving policy.

[0127] In the RSS model, for example, the longitudinal and lateral safety distances from other road users are used as indicators for determining the collision risk. The safety distances are an example of a geometric approach such as a safety envelope.

[0128] The behavior unit 30 may include a motion control unit 31 and an HMI output unit 71 as processing units for realizing sub-functions obtained by further classifying the behavior functions by the processor 51b executing a computer program. The motion control unit 31 controls the motion of the vehicle 1 based on the trajectory plan (e.g., a path plan and a speed plan) acquired from the driving plan unit 22. Specifically, the motion control unit 31 generates accelerator request information, shift request information, brake request information, and steering request information according to the trajectory plan, and outputs them to the motion actuator 60.

[0129] Here, the motion control unit 31 can directly obtain the vehicle state recognized by the detection unit 10 (particularly the internal recognition unit 13), such as at least one of the current speed, acceleration, and yaw rate of the vehicle 1, from the detection unit 10 and reflect this in the motion control of the vehicle 1.

[0130] The HMI output unit 71 outputs information related to the HMI based on at least one of prediction information and user intention estimation information from the prediction unit 21, application state transition information and trajectory planning from the operation planning unit 22, and function constraint information from the mode management unit 23. The HMI output unit 71 may manage vehicle interactions. The HMI output unit 71 may generate an information presentation request based on the management state of the vehicle interactions and control the information presentation function of the HMI device 70. Furthermore, the HMI output unit 71 may generate control requests for wipers, a sensor washing device, headlights, and an air conditioning device based on the management state of the vehicle interactions and control these devices.

[0131] <Testing in Public Road Traffic> A verification and validation (V&V) process is required for the driving system 2 described above. The V&V here may be V&V of the intended functions of the software used in the driving system 2 or V&V of SOTIF. Scenarios that the vehicle 1 may encounter can be classified into known dangerous scenarios, known non-hazardous scenarios, unknown dangerous scenarios, and unknown non-hazardous scenarios. The V&V process may be a process for reducing the risks of known dangerous scenarios and unknown dangerous scenarios among these scenarios.

[0132] The V&V of the operation system 2 includes verification to satisfy the safety requirements at each technology level and verification to safely integrate each element. The verification to satisfy the safety requirements at each technology level may include evaluation of at least one, and preferably all, of the following functions and capabilities. The verification may also include evaluation of other functions and capabilities.

[0133] For example, the detection unit 10 may evaluate the functionality of the sensors 40 or external data sources (eg, map data sources), the functionality of the sensor algorithms that model the environment, and the reliability of the infrastructure and communication systems 43 .

[0134] For example, an evaluation target related to the planning unit 20 is the capability of a decision algorithm. The capability of the decision algorithm is the ability to safely handle potential functional deficiencies, and the ability to make appropriate decisions according to an environmental model, a driving policy, a current destination, etc. Also, for example, an evaluation target related to the planning unit 20 may include one of the following: absence of unreasonable risks due to dangerous behavior of the intended functions, the capability of the system to safely handle use cases of the ODD, robust performance of the execution of the driving policy across the ODD, suitability of the DDT fallback, and suitability of the minimum risk state.

[0135] Furthermore, for example, the evaluation target may include not only the nominal performance of the system or function but also its robust performance, such as the system's robust performance against adverse environmental conditions affected by various disturbances, the appropriateness of system operation against known trigger conditions, the sensitivity of the intended function, and the monitoring capability against various scenarios.

[0136] V&V may be performed with the goal that the automated driving performed by the driving system 2 achieves a positive risk balance. A positive risk balance can be said to be a primary measure of an ethically acceptable risk level.

[0137] More specifically, V&V may be performed with the goal of achieving a risk tolerance criterion that may be set based on a positive risk balance. A quantitative criterion for the risk tolerance criterion may be, for example, that the probability of occurrence of harm is below a threshold. The risk tolerance criterion may be set by combining a statistical approach, such as traffic statistics, with a scenario-based approach.

[0138] When verifying or testing software related to at least one of the behavior planning and trajectory planning by the driving planning unit 22, it is necessary to confirm that the driving system 2 behaves at least safer than a competent and careful driver or an experienced and careful driver. When testing software in a virtual environment, a verification method based on software-in-the-loop (SiL) may be performed using, for example, a reference data set including scenarios stored in a scenario DB.

[0139] When verifying or testing software related to mode management by the mode management unit 23, particularly verification or testing related to ODD determination and management of operating and non-operating states, may be performed in SiL, hardware-in-the-loop (HiL), or both.

[0140] Furthermore, when verifying or testing software related to the HMI using the HMI output unit 71 or the like, the verification or testing may be performed using HiL and driver-in-the-loop (DiL). The DiL test may be targeted at vehicle users, such as drivers who have no prior experience or knowledge of the driving system 2 and are unfamiliar with driving, such as automated driving.

[0141] In this way, verification methods such as SiL, HiL, and DiL may be selected depending on the target and purpose in verifying or testing software in the driving system 2. The loops used in SiL, HiL, and DiL may be open loops or closed loops.

[0142] In order to manage unacceptable risks and improve the driving system 2, it is preferable to have a robust management system MS for the driving system 2 of vehicles 1 participating in public road traffic after they are released to the market. For example, the management system MS shown in FIG. 4 may perform software changes to the vehicle population over-the-air (OTA). The management system MS includes a vehicle population including multiple vehicles 1A, 1B, ... and a server 96. The vehicles 1A and 1B managed by the management system MS may have the same configuration as the vehicle 1 equipped with the driving system 2 described above. However, as long as there is at least compatibility in the software specifications, some of the hardware specifications, such as vehicle type and model, may differ from each other.

[0143] Testing in the software change process may use data collected while each vehicle 1A, 1B belonging to the vehicle population is traveling on public roads. Also, safety metrics may be used in testing in the change process. The validation results using safety metrics do not have to be used solely to determine whether or not the test software is applicable. For example, the results may be used to set the ODD itself and set parameter limits by the ODD. The setting here may include specification changes due to software updates.

[0144] In the following, we may refer to software used for temporary testing as test software and software used officially (permanently) as official software. In this case, permanent application may mean application until the next update of the official software.

[0145] Furthermore, testing by this management system MS is not limited to software updates, and may also be conducted to improve the environment in which the driving system 2 is used. That is, the results of the tests may be fed back into urban planning. For example, the road shapes at intersections, merging points, etc. may be changed to shapes that provide greater safety as a result of the tests. Also, for example, the control of traffic lights at intersections, etc. may be changed to control that reduces the frequency of traffic congestion as a result of the tests. In this way, the management system functions as a test system.

[0146] The safety indicator may be a safety metric of the automated driving system. The safety metric may refer to a quantifiable measure based on collision rates. Examples of safety metrics include collision severity and frequency, citable offense severity and frequency, longitudinal and lateral distance, longitudinal and lateral acceleration, longitudinal and lateral jerk, and OEDR reaction time. The collision severity may be rated on a six-point scale using, for example, the Abbreviated Injury Scale (AIS). The longitudinal and lateral distances are indicators of maintaining a safety envelope or safety distance.

[0147] Furthermore, the evaluation of the safety index or the like may be a human evaluation. The human here may be a vehicle user of the vehicle under test. The human here may be a vehicle user who is unfamiliar with automated driving. In a DiL test, the evaluation metric may be an evaluation input by a vehicle user, i.e., a driver, through the operation input device 70a (e.g., the feedback device 70a1) of the vehicle under test, or may be an evaluation input through a mobile terminal 91 owned by the driver. The human here may be another vulnerable road user (VRU), such as a pedestrian, who encounters the vehicle under test. In this case, the evaluation may be an evaluation transmitted by another VRU who senses danger from the vehicle under test to the driving system 2 or server 96 of the vehicles 1A, 1B using their own mobile terminal 91 or the like.

[0148] <Example of Server Configuration in Management System> As shown in FIGS. 1 and 4 , the server 96 is installed in an external environment relative to the vehicles 1A and 1B. The server 96 includes a communication interface 96e and is communicatively connected to each of the vehicles 1A and 1B, for example, by V2X communication via a communication infrastructure. The server 96 may be connected to an operation terminal operated by a human operator, for example, and may constitute a remote management center that manages a vehicle population together with the operation terminal. For example, the communication interface 96e includes an input / output circuit that allows the server 96 to transmit and receive various data to and from the outside. The configuration includes at least one of a terminal and a wireless communication device for connecting the input / output circuit to a communication infrastructure for V2I communication with each of the vehicles 1A and 1B.

[0149] As shown in FIG. 1, the server 96 may be primarily configured as a dedicated computer having at least one memory 96a and one processor 96b. The memory 96a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data that can be read by the processor 96b. Furthermore, the memory 96a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 96b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0150] The dedicated computer may be a SoC (System on a Chip) in which the memory 96a, processor 96b, and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0151] The server 96 may further have a management database (hereinafter referred to as management DB) 96c. The management DB 96c may store information for identifying the vehicles 1A, 1B that are to be managed by the management system MS. The management DB 96c may store information related to the specifications of each vehicle 1A, 1B. The management DB 96c may store various information collected from each vehicle 1A, 1B. The various information may include information for performing evaluation in the test described below. The various information may also include information related to the software application status of each vehicle 1A, 1B participating in the test.

[0152] The server 96 implements a software improvement function based on a V&V process that is suitable for the driving system 2. As shown in Fig. 4, the server 96 may include a test management unit 97a, a software distribution unit 97b, a data collection unit 97c, and a test software evaluation unit 97d as processing units for realizing functions by a processor 96b executing a computer program.

[0153] The test management unit 97a manages tests using the vehicle population. Tests may be conducted using one type of executable test software, which is an improved version of the software currently being used by the vehicle population (hereinafter referred to as the current software). When two types of executable test software are prepared and these two types of test software are compared, this test is called an AB test. The multiple test software may be three or more types of software that have similar functions and can be compared with each other.

[0154] Next, we will explain two typical examples of AB testing: a first example in which one of multiple test software programs is assigned to one test vehicle, and a second example in which multiple test software programs are assigned to one test vehicle.

[0155] The test software is provided, for example, by a human administrator of the server 96 (hereinafter referred to as the test administrator). In this case, the test is conducted with the aim of selecting the most suitable software from among the release candidate software. On the other hand, if the test is conducted to optimize parameters used in a computer program, the test software may be software automatically generated by the test management unit 97a. For example, the test software may be automatically generated in a form that changes parameters such as the judgment threshold, upper limit, lower limit, waiting time, display time, and display size of an existing program.

[0156] The test management unit 97a manages the scale and duration of the test. The scale and duration may be set to values ​​entered into the server 96 by the test administrator, or may be automatically set by the test management unit 97a. The test management unit 97a determines test target vehicles suitable for conducting the test from among the vehicles 1A, 1B belonging to the vehicle population based on the scale and duration. If the number of vehicles incorporated into the management system MS is smaller than the optimal scale for conducting the test, all of the vehicles 1A, 1B belonging to the vehicle population may be designated as test target vehicles.

[0157] In a first example, the test management unit 97a assigns one of multiple test software programs to each of the vehicles 1A and 1B belonging to the vehicle population. In an A / B test comparing test software A and test software B, for example, half of the test vehicles test test software A, and the other half test test software B. The test management unit 97a may randomly assign test software programs to each of the vehicles 1A and 1B using pseudo-random numbers. The test management unit 97a may also refer to vehicle information stored in the management DB 96c and assign test software programs so as to reduce bias in the conditions between the groups testing each test software program. On the other hand, in a second example, the test management unit 97a assigns all of the multiple test software programs to each of the vehicles 1A and 1B belonging to the vehicle population.

[0158] The test management unit 97a may impose restrictions on the content of the test. For example, the test management unit 97a may limit the area in which the test is conducted to, for example, a specific country or region. The test management unit 97a may limit the time period in which the test is conducted to, for example, only daytime hours.

[0159] Furthermore, the test management unit 97a sets at least one evaluation metric for evaluating the test software. The evaluation metric may be set to a metric determined by an input operation by the test administrator to the server 96 based on the functions and characteristics of the test software or the test intent. Alternatively, the evaluation metric may be set by the test management unit 97a based on the functions and characteristics of the test software. The evaluation metric may include an index related to safety.

[0160] The test management unit 97a may also manage at least one of a method for obtaining consent from the vehicle user regarding the application of the test software to each test target vehicle and the content of the consent. The test management unit 97a may leave the management of at least one of a method for obtaining consent and the content of the consent to the driving system 2 of each test target vehicle.

[0161] The software distribution unit 97b distributes the test software to the driving systems 2 of the vehicle under test based on the test software allocation set by the test management unit 97a. Information regarding the test plan may be distributed together with the distribution of the test software. The information regarding the test plan includes information regarding the test period and evaluation, and may also include information such as the scale of the test. As a result, the distributed test software is temporarily applied to each driving system 2, and the test begins.

[0162] In the second example, the software distribution unit 97b may distribute multiple test software programs at once. Alternatively, in the second example, the software distribution unit 97b may first distribute only test software A and then distribute test software B after the testing of test software A is completed.

[0163] The data collection unit 97c collects, as probe data, information about the operation or operation results of the test software from each driving system 2 to which the test software is temporarily applied. The collected information may include at least one of the safety indicators themselves and information for deriving the safety indicators. The data collection unit 97c accumulates the data sequentially collected from each vehicle 1A, 1B in the management DB 96c.

[0164] The test software evaluation unit 97d evaluates and preferably compares multiple test software programs. The test software evaluation unit 97d compares the evaluation metrics set by the test management unit 97a between the test software programs. If multiple evaluation metrics exist, the test software evaluation unit 97d compares each evaluation metric between the test software programs.

[0165] Specifically, the test software evaluation unit 97d statistically processes the data from each vehicle 1A, 1B stored in the management DB 96c. For example, if the evaluation metric can be expressed as a rate such as an occurrence rate, the test software evaluation unit 97d calculates the occurrence rate per vehicle and / or per unit time from the occurrence frequency in the data from each vehicle 1A, 1B.

[0166] Furthermore, in an evaluation using a safety metric as an evaluation metric, it is preferable to take into account the severity potential. When evaluating the severity and frequency of collisions for test software, even if the test software is temporarily applied to multiple vehicles 1A and 1B, if the actual number of collisions is small, it is difficult to perform a statistical evaluation. For this reason, the test software evaluation unit 97d may estimate the rate of serious collisions with high severity from the occurrence rates of appropriate precursor events, such as near collisions and low-severity collisions.

[0167] Furthermore, the distribution and sensitivity of crash types may differ between an automated driving system and a human driver. For this reason, in the statistical evaluation of the safety metrics for the test software, the vehicle 1 to which the test software is applied may be classified into an automated driving vehicle of level 3 or higher and a vehicle driven by a human user, and then data may be collected and evaluated.

[0168] Furthermore, the test software evaluation unit 97d may have a function of selecting one optimal test software from multiple test software. The optimal test software may be the test software with the highest safety. The test management unit 97a may decide that the test software selected by the test software evaluation unit 97d is the official software to be officially adopted.

[0169] If the multiple test software programs are improved versions of the current software currently officially applied to each vehicle 1A, 1B, the test software evaluation unit 97d may perform a relative evaluation of the selected test software against the current software. The test software evaluation unit 97d may determine that the selected test software does not have higher performance than the current software. In this case, the test management unit 97a may exclude any of the multiple test software programs from the official software to be officially adopted.

[0170] On the other hand, the test software evaluation unit 97d does not need to have the function of selecting the optimal test software. In this case, the test management unit 97a presents the comparison results of the evaluation metrics to the test manager via the server HMI 96d and accepts the test manager's input operation of the selection results of the official software to be officially adopted. Based on this input operation, the test management unit 97a determines the official software.

[0171] Based on the determination of the official software, the software distribution unit 97b may distribute the official software to each vehicle 1A, 1B belonging to the vehicle population. The distribution destination vehicles may include vehicles belonging to the vehicle population that were not selected as test vehicles. Furthermore, the software distribution unit 97b may request other management systems to adopt the selected software or may recommend the selected software to other management systems.

[0172] <Configuration Example of Software Management Unit> Each vehicle 1A, 1B belonging to the vehicle population is equipped with an individual driving system 2. The driving system 2 is equipped with a software management function in addition to a recognition function, a planning function, and an action function. The driving system 2 may each include a software application unit MF1, a motion measurement unit MF2, and an HMI linkage unit MF3 as processing units for realizing functions by the processor 57b of the software management unit 57 executing a computer program.

[0173] The software application unit MF1 manages the application of software implemented in the corresponding driving system 2. The software application here may include downloading and installing the software, i.e., preparing the software for use. The management of software application may include defense against external attacks that exploit security vulnerabilities, management of privacy in the software, and management of software updates. The update management may include software version management.

[0174] The software application unit MF1 manages the application of software that may include test software. When the vehicle 1A, 1B is selected as a test target, the software application unit MF1 temporarily applies the test software downloaded from the server 96 to the driving system 2 and starts verifying the test software.

[0175] In the second example, when multiple test software A and B are distributed together, the software application unit MF1 may select test software A from the multiple test software A and B to be temporarily applied to the driving system 2, and after verification of test software A is completed, the next test software B may be temporarily applied to the driving system 2.

[0176] The software whose application is managed by the software application unit MF1 may include official software. The software application unit MF1 may permanently apply the official software selected as a result of the test to the driving system 2. Alternatively, the software application unit MF1 may update the current software to the official software selected based on the result of the test.

[0177] The operation measurement unit MF2 measures the operation of the temporarily applied test software and the impact of the operation (hereinafter referred to as the operation result). The measurement target is at least one of the evaluation index specified by the server 96 and the data required to calculate the evaluation index. The measurement of the operation result may be performed continuously depending on the characteristics of the evaluation metric, or may be performed only on specified occasions based on a preset trigger.

[0178] Furthermore, the operation measurement unit MF2 transmits the measured test results to the server 96. The test results, including the progress, may be transmitted sequentially, or may be transmitted all at once at the end of the test period. The test results include at least one of test software application information to the operation system 2 by the software application unit MF1 and measurement information measured by the operation measurement unit MF2. The test software application information may include the date and time when the test software was applied to the operation system 2, the method of applying the test software to the operation system 2, etc. The software application method is information indicating, for example, that the test software was installed on hardware that constitutes a redundant system other than the main hardware, or that the test software was installed using a shadow mode mechanism. The measurement information is information related to the measurement target described above.

[0179] The operation measurement unit MF2 may also record the test results in the storage medium 55c. A transmission log to the server 96 may also be recorded in the storage medium 55c together with the test results.

[0180] The HMI linkage unit MF3 cooperates with the HMI device 70 to provide the following functions: explaining the test to the driver and obtaining the above-mentioned consent regarding the test; issuing software management notifications; and receiving feedback from the vehicle user. To provide software management notifications, the HMI linkage unit MF3 controls the HMI device 70 based on a request from the software application unit MF1 to minimize confusion among vehicle users, such as passengers, during software changes, such as when applying test software. The software change may be a software change that includes a change in notification behavior, or a software change that does not include a change in notification behavior. The software change that includes a change in notification behavior will be described in detail below. The software change may be a change to a single computer program, or a change to multiple related computer programs.

[0181] <Driving Switchover Test> When the driving system 2 is at automation level 3 or 4, a driving switchover may occur between the driving system 2 and the driver. In particular, when a driving switchover from the driving system 2 to the driver occurs, the system specifications should be improved so that the driver can appropriately respond to a driving switchover request from the system side. Then, the management system MS can conduct tests using vehicles 1A and 1B participating in public road traffic in order to improve the system specifications related to the driving switchover.

[0182] The test software for the driver changeover is installed in at least one of the memories 51 a, 53 a, and 57 a of the driving system 2, thereby implementing a test pattern for the driver changeover in the driving system 2.

[0183] The test pattern for driver changeover includes at least one of a test pattern related to the HMI (hereinafter referred to as the HMI pattern) for prompting the driver to change over, and a test pattern (planning pattern) related to the trajectory planning and behavior planning of vehicles 1A and 1B from just before the driver changeover until the driver changeover.

[0184] The test pattern for driver change should be a pattern that can verify at least one system specification, such as the timing of the driver change, the countdown pattern until that timing, the conditions for the location where the driver change is performed, the speed conditions for the driver change, the acceleration conditions for the driver change, the notification pattern requesting or warning of a driver change, the conditions for determining whether the driver change is complete, and the response if the driver does not agree to the driver change.

[0185] For example, the test patterns A and B for driver alternation prepared in advance as an A / B test may be the patterns shown below. However, it is assumed that ODD includes the conditions shown below.

[0186] ODD: The vehicle is traveling on a road for motor vehicles only (including expressways, etc.). The lane in which the vehicle is traveling is congested or close to being congested, and the vehicle ahead is traveling near the center of the lane. The speed of the vehicle is 70 km / h or less.

[0187] Test pattern A: When the traffic jam clears and the preceding vehicle reaches a speed of 70 km / h or more, the vehicle continues autonomous driving with a maximum speed of 60 km / h, and takes over driving when the preceding vehicle can no longer be detected by the external environment sensor.

[0188] Test pattern B: When the traffic jam clears and the preceding vehicle reaches a speed of 70 km / h or more, the vehicle continues autonomous driving by following the preceding vehicle, and performs a driver switch when the vehicle reaches a speed of 70 km / h.

[0189] The current pattern is assumed to be the pattern shown below.

[0190] Current pattern: When the traffic jam clears and the preceding vehicle increases to 70 km / h or more, the vehicle continues autonomous driving with a maximum speed of 40 km / h, and takes over driving when the preceding vehicle can no longer be detected by the external environment sensor.

[0191] The management system MS executes test patterns A and B on multiple test vehicles and compares the test patterns A and B. To this end, the test management unit 97a in the server 97 sets an evaluation metric for evaluating whether the driver changeover was properly performed. This evaluation metric may evaluate the safety or collision risk during the driver changeover.

[0192] For example, the evaluation metric may include the time from when the HMI initiates a request to the driver to take over driving until the driver accepts the takeover. The evaluation metric may include a value indicating the lateral sway of the vehicle 1 immediately after the driver handover. The evaluation metric may include a value indicating the stability of the speed immediately after the driver handover, or a value indicating the stability of the acceleration and deceleration immediately after the driver handover. The evaluation metric may include a value indicating the collision risk immediately after the driver handover. The evaluation metric may be or include a safety metric.

[0193] The evaluation metric for evaluating the driving change may include the driver's evaluation. On the other hand, the evaluation metric for evaluating the driving change may exclude the driver's evaluation and may be calculated using only objective data such as the vehicle state. By excluding the driver's preferences from the evaluation, it becomes possible to purely evaluate the safety or collision risk during the driving change.

[0194] In the server 97, a data collection unit 97c executes test pattern A on each test vehicle and collects evaluation results obtained by evaluating the driving change using the evaluation metric based on test pattern A. Similarly, the data collection unit 97c executes test pattern B on each test vehicle and collects evaluation results obtained by evaluating the driving change using the evaluation metric based on test pattern B. A test software evaluation unit 97d aggregates the evaluation results of each test pattern.

[0195] An example of a method for conducting a test related to a driver changeover will now be described with reference to the flowchart of Figure 5. This series of processes in S101 to S111 may be realized, for example, by having the processor 96b of the server 96 execute a computer program stored in the memory 96a, and then having the processor 57b of the software management unit 57 in each vehicle under test execute a computer program (test implementation program) stored in the memory 57a. A trigger for starting this process may be provided by an input operation by the test administrator to the server HMI 96d.

[0196] In the first step S101, a test is planned in the server 96 (for example, the test management unit 97a). The test plan includes at least one of, and preferably all of, the determination of the test method, scale, period, and test target vehicle described above, and the determination of the test pattern evaluation method (including evaluation metrics). In step S102 following step S101, the server 96 transmits test software to the test target vehicle. After processing step S102, the process proceeds to step S103.

[0197] In S103, the software management unit 57 (particularly the software application unit MF1) of each test target vehicle sets the evaluation metrics for evaluating the test patterns determined in S101 in an evaluable format. For example, an evaluation program is installed in the memory 57a. The evaluation program may be included in the test software or may be distributed separately from the test software. If the evaluation program has already been installed (for example, if it has been used in a previous test), the evaluation program is activated. After processing S103, the process proceeds to S104.

[0198] In S104, the HMI linkage unit MF3 controls the HMI device 70 so that an explanation about the test is given to the driver as a vehicle user. For example, explanation content such as a video is played using the CID 70b2 and a speaker of the information presentation device 70b.

[0199] Next, the HMI linking unit MF3 controls the HMI device 70 to obtain consent for the test from the driver as the vehicle user. For example, a document containing the consent details (contract details) is displayed on the CID 70b2, along with an "Agree" button for the driver to indicate their consent. The driver can indicate their consent by reading the document and touching the "Agree" button. The HMI linking unit MF3 obtains information indicating that the "Agree" button has been touched from the CID 70b2, and records the consent acquisition result together with the acquisition date and time in the storage medium 55c via the recording device 55. The consent acquisition result may be transmitted to the server 96 and recorded in the management DB 96c.

[0200] The process of S104 is preferably executed after the driver gets into the vehicle 1 and before the start of automatic driving by the driving system 2. If the process is executed after the start of automatic driving, the driving system 2 may hand over to the driver during the explanation or consent acquisition, which may confuse the driver.

[0201] In more detail, the processing of S104 may be performed immediately after the driver turns on the start switch (power switch) of vehicle 1, or may be performed between the time the driver turns on the automatic driving enable switch of vehicle 1 and the time automatic driving actually begins.

[0202] The explanation may be provided only when the vehicle 1 is stopped, or may be provided while the vehicle 1 is moving. In the latter case, the HMI linkage unit MF3 may change the content of the explanation content depending on whether the vehicle 1 is stopped or moving. For example, a change in the content of the explanation content may include a change in the granularity (level of detail) of the explanation. For example, when the explanation is provided while the vehicle 1 is moving, it may only explain that a test will be performed when the driver is changed over. On the other hand, when the explanation is provided while the vehicle 1 is stopped, it may include an explanation that the test pattern to be tested is "when the traffic jam clears and the preceding vehicle reaches 70 km / h or more, the vehicle will continue autonomous driving at an upper limit of 60 km / h, and will perform a driver change when the preceding vehicle can no longer be confirmed by the external environment sensor," i.e., a more detailed explanation of the test pattern.

[0203] In S105 after processing S104, the software application unit MF1 determines whether a situation has arisen in which a driver handover from the driving system 2 to the driver will occur, based on the automation level management information acquired from the mode management unit 23. After this, it is necessary to determine whether to execute a test pattern, and this determination must be made before the driver handover occurs, when the possibility of the driver handover occurring is high. If the answer is Yes, proceed to S106. If the answer is No, the process of S105 is executed again after a predetermined time.

[0204] In S106, the software application unit MF1 determines whether the cause of this driver change is an ODD exit. If the answer is No (if the cause is an unexpected event other than an ODD exit), the process proceeds to S107. If the answer is Yes, the process proceeds to S108.

[0205] Here, an unexpected event other than the exit of the ODD refers to, for example, an event in which the driving system 2 falls into a situation in which it cannot continue autonomous driving due to a system failure that was not anticipated by the ODD, an event in which the driving system 2 falls into a situation in which it cannot continue autonomous driving due to the vehicle 1 encountering an unknown scenario, etc. An event in which the driving system 2 falls into a situation in which it cannot continue autonomous driving due to the vehicle 1 encountering an unknown scenario may mean that the performance limit of the driving system 2 has been exceeded in terms of robustness.

[0206] In S107, the software application unit MF1 prohibits the execution of the test pattern and determines to execute the current pattern in the upcoming operation changeover. That is, since the operation changeover will be executed under a heavy load on the operation system 2, the operation changeover will be executed using the current pattern for which sufficient experience has been accumulated. Since the test has not been executed, the process returns to S105 after processing S107.

[0207] In S108, the software application unit MF1 determines to execute the test pattern in the operation changeover that is about to be carried out. After the process of S108, the process proceeds to S109.

[0208] In S109, the action measurement unit MF2 evaluates the driving shift performed using the test pattern using the driving metrics defined in S103. The action measurement unit MF2 then generates data indicating the evaluation results in a predefined format that facilitates transmission to the server 96 and statistical processing by the server. After processing S109, the process proceeds to S110.

[0209] In S110, the action measurement unit MF2 records the evaluation result processed in S109 in the storage medium 55c via the recording device 55. The action measurement unit MF2 also transmits the evaluation result to the server 96 via the communication system 43. After processing in S110, the process proceeds to S111.

[0210] In S111, the server 96, for example, includes a data collection unit 97c that collects evaluation results, including evaluation results received from other vehicles. The test software evaluation unit 97d then statistically processes the evaluation results and evaluates the test patterns. S111 marks the end of this series of processing.

[0211] The series of steps S101 to S111 may be performed once for test pattern A, and then once for test pattern B. Furthermore, by repeatedly performing steps S105 to S110 until a predetermined number of times is reached, the same test pattern can be tested multiple times on the same vehicle.

[0212] According to the first embodiment described above, test patterns related to driving changeovers are prepared in advance, so that when a situation arises in which a changeover from the driving system 2 to a driver occurs, the test patterns can be executed smoothly. In other words, it is possible to not overlook driving changeover events that occur in vehicles 1 that can participate in public road traffic and to utilize them for testing test patterns, thereby facilitating V&V related to driving changeovers and making it easier to appropriately improve the system specifications of the vehicle 1.

[0213] Furthermore, according to the first embodiment, an evaluation metric is set to evaluate whether a driver changeover based on a test pattern was properly performed. The driver changeover is then evaluated using the evaluation metric. Furthermore, the evaluation results of the driver changeover using the evaluation metric are recorded in the storage medium 55c. In this way, the objective evaluation results obtained using the evaluation metric can be statistically processed and utilized for V&V, making it even easier to appropriately improve the system specifications of the vehicle 1.

[0214] Furthermore, according to the first embodiment, the HMI device 70 provides the driver with an explanation about the test regarding the driver changeover after the driver gets into the vehicle 1 and before autonomous driving begins. The explanation is provided at a time that avoids a timing when a driver changeover may occur, which allows the driver to deepen his or her understanding of the test.

[0215] Furthermore, according to the first embodiment, the HMI device 70 attempts to obtain consent from the driver for the driving change test after the explanation. Since consent is obtained after the driver has received the explanation about the test and has clearly memorized it, the possibility of obtaining consent can be increased.

[0216] According to the first embodiment, the test pattern is tested when the driver change is caused by exiting the ODD. On the other hand, the test pattern is prohibited when the driver change is caused by other unexpected events. In a situation where the load on the driving system 2 or the driver is considered to be high, the test pattern is prohibited, thereby preventing unexpected situations from occurring due to the test.

[0217] Second Embodiment As shown in Fig. 6, the second embodiment is a modification of the first embodiment. The second embodiment will be described, focusing on the differences from the first embodiment.

[0218] In the second embodiment, the test is a test that compares a current pattern with a test pattern. The server 96 (particularly the test management unit 97 a and the software distribution unit 97 b) prepares one test pattern and distributes test software including the test pattern to a test target vehicle in which current software including the current pattern is installed.

[0219] In the vehicle 1, the software management unit 57 executes the current pattern to evaluate the driving change using the evaluation metric, and then executes the test pattern to evaluate the driving change using the same evaluation metric. By executing the current pattern and the test pattern using the same vehicle 1 and the same driver and evaluating them using the same evaluation metric, the accuracy of the comparison can be dramatically improved.

[0220] Next, an example of a method for conducting a test related to driver changeover will be described using the flowchart in Figure 6. This series of processes from S201 to S210 may be realized, for example, by the processor 96b of the server 96 executing a computer program stored in the memory 96a, and the processor 57b of the software management unit 57 in each vehicle under test correspondingly executing a computer program stored in the memory 57a. A trigger for starting this process may be provided by an input operation by the test administrator to the server HMI 96d.

[0221] The processes of S201 to S203 are the same as those of S101 to S103 in the first embodiment. However, as described above, the planned test is a test that compares the current pattern with the test pattern. After the process of S203, the process proceeds to S204.

[0222] In S204, the software application unit MF1 identifies a situation in which a driving changeover from the driving system 2 to the driver occurs based on the automation level management information acquired from the mode management unit 23, and determines to execute the current pattern for the driving changeover. The driving changeover is executed using the current pattern. After processing S204, the process proceeds to S205.

[0223] In S205, the operation measurement unit MF2 evaluates the driving changeover executed using the current pattern using the driving metrics defined in S203. After the process of S205, the process proceeds to S206.

[0224] In S206, explanation and consent acquisition are performed in the same manner as in S104. That is, the processing of S206 is performed after a current pattern other than the test pattern is executed in a situation where a driving changeover occurs from the driving system 2 to the driver. In this way, the test pattern is explained to the driver after he has experienced the current pattern, which deepens the driver's understanding of the test.

[0225] In S207 after the processing of S206, the software application unit MF1 identifies a situation in which a driving handover from the driving system 2 to the driver occurs based on the automation level management information acquired from the mode management unit 23, and determines to execute a test pattern for the driving handover. The driving handover is executed using the test pattern. After the processing of S207, the process proceeds to S208.

[0226] In S208, the action measurement unit MF2 evaluates the driving changeover executed using the test pattern using the driving metrics defined in S203. After the processing of S208, the process proceeds to S209.

[0227] In S209, the operation measurement unit MF2 associates the evaluation results of the current pattern with the evaluation results of the test pattern, and records these evaluation results in the storage medium 55c via the recording device 55. The operation measurement unit MF2 also transmits the evaluation results of the current pattern with the evaluation results of the test pattern, and transmits these evaluation results to the server 96 via the communication system 43. S210, which follows the processing of S209, is the same as S211 in the first embodiment. The series of processes ends with S210.

[0228] According to the second embodiment described above, in a situation where a handover of driving from the driving system 2 to the driver occurs, the HMI device 70 executes the current pattern and then provides the driver with an explanation of the test related to the handover. Since the driver receives an explanation of the test after experiencing the current pattern, the driver can deepen his or her understanding of the test.

[0229] 7 to 9, the second embodiment is a modification of the first embodiment. The second embodiment will be described, focusing on the differences from the first embodiment.

[0230] The test in the third embodiment is performed using an intermediate pattern that is different from the current pattern and the test pattern. The intermediate pattern may be generated by a test administrator, may be automatically generated by the server 96 that is given the current pattern and the test pattern, or may be automatically generated by the software management unit 57.

[0231] The intermediate pattern can be generated by extracting the difference between the current pattern and the test pattern and applying a part of the difference to the current pattern. One or more intermediate patterns can be generated between the current pattern and the test pattern.

[0232] Then, in the driving system 2, the software application unit MF1 prepares an intermediate pattern in addition to the test pattern. If the intermediate pattern has been generated by the test manager or the server 96, the software application unit MF1 may download the intermediate pattern in addition to the test pattern from the server 96. If the intermediate pattern is generated by the software management unit 57, the software application unit MF1 may generate it.

[0233] Then, the software application unit MF1 determines to test an intermediate pattern that has been changed less from the current pattern than the test pattern before testing the test pattern. In other words, by testing the intermediate pattern first and then the test pattern, it becomes possible to ease the load on the driver that would otherwise be required to respond to a sudden change from the current pattern to the test pattern.

[0234] An example of an intermediate pattern will now be described with reference to FIG. 7 . In the example shown in FIG. 7 , the test pattern for driver changeover is a test pattern for verifying the manner of notification requesting or warning a driver changeover. Specifically, the current pattern is a pattern in which the meter display 70b1 and the CID 70b2 respectively display an image display PC1 showing the driver gripping the steering wheel (to drive) and a text display TD1 prompting the driver to perform a driving operation. In contrast, the test pattern is a pattern in which the meter display 70b1 and the CID 70b2 respectively display an image display PC2 showing the driver being switched from an artificial intelligence to a human and a text display TD2 showing the number of seconds remaining until the driver changeover should occur.

[0235] In contrast to these current pattern and test pattern, the intermediate pattern is a pattern in which image display PC1 and character display TD1 are displayed on the meter display 70b1, while image display PC2 and character display TD2 are displayed on the CID 70b2. In this way, among the multiple information presentation devices 70b that present notification content during a driver changeover, the content of some of the information presentation devices 70b can be changed to the test pattern, and the content of the other information presentation devices 70b can be maintained as the current pattern, thereby conducting a test.

[0236] Next, an example of a method for conducting a test related to driver changeover will be described using the flowchart of Figure 8. This series of processes from S301 to S312 may be realized, for example, by the processor 96b of the server 96 executing a computer program stored in the memory 96a, and the processor 57b of the software management unit 57 in each vehicle under test correspondingly executing a computer program stored in the memory 57a. A trigger for starting this process may be provided by an input operation by the test administrator to the server HMI 96d.

[0237] The initial processing of S301 to S303 is the same as S101 to S103 in the first embodiment. After the processing of S303, the process proceeds to S304.

[0238] In S304, the software application unit MF1 extracts differences between the current pattern and the test pattern. Then, the software application unit MF1 generates an intermediate pattern by applying some of the extracted differences to the current pattern. After processing S304, the process proceeds to S305.

[0239] In S305, the software application unit MF1 identifies a situation in which a driving changeover from the driving system 2 to the driver occurs based on the automation level management information acquired from the mode management unit 23, and determines to execute an intermediate pattern for the driving changeover. The driving changeover is executed using the intermediate pattern. After processing S305, the process proceeds to S306.

[0240] In S306, the action measurement unit MF2 evaluates the driving changeover executed using the intermediate pattern using the driving metrics defined in S303. After the processing of S306, the process proceeds to S307.

[0241] In S307, the software application unit MF1 determines whether the driving changeover executed using the intermediate pattern was successful. This determination may be, for example, whether the evaluation metric is equal to or greater than a preset threshold. If the answer is No, proceed to S308. If the answer is Yes, proceed to S310.

[0242] In S308, the software application unit MF1 determines whether the number of failures in the intermediate pattern (the number of times that S307 is determined to be No) is equal to or greater than a preset threshold. If the answer is No, the process returns to S305. If the answer is Yes, the process proceeds to S309.

[0243] In S309, the software application unit MF1 decides to stop the test. The software application unit MF1 records the fact that the test has been stopped in the recording device 55 and transmits this to the server 96. The series of processes ends with S309.

[0244] On the other hand, if the answer is Yes in S307, S310 to S313 are the same as S207 to S210. The series of processes ends with S313.

[0245] Further, another example of the intermediate pattern will be described with reference to Fig. 9. In the example shown in Fig. 9, the test pattern for operation changeover is a test pattern for verifying the speed conditions at the time of operation changeover. Specifically, the current pattern is a pattern similar to test pattern A in the first embodiment. In contrast, the test pattern is a pattern similar to test pattern B in the first embodiment.

[0246] 9, the intermediate pattern includes multiple patterns (for example, two patterns). Intermediate pattern A is a pattern in which the upper speed limit of the vehicle in the current pattern is changed from 60 km / h to 63 km / h. Intermediate pattern B is a pattern in which the upper speed limit of the vehicle in the current pattern is changed from 60 km / h to 67 km / h.

[0247] In this case, the software management unit 57 first performs a driving change using intermediate pattern A, and if that is successful, performs a driving change using intermediate pattern B, and if that is successful, performs the test pattern.

[0248] According to the third embodiment described above, an intermediate pattern is generated by extracting the difference between a test pattern and a current pattern different from the test pattern, and the intermediate pattern is prepared by applying a part of the difference to the current pattern. Then, before testing the test pattern, the intermediate pattern is tested in a situation where a change of driving role occurs from the driving system 2 to a driver. By testing an intermediate pattern with fewer changes from the current pattern before testing the test pattern, it is possible to suppress driver confusion caused by the pattern change when the driver is changed over.

[0249] Furthermore, according to the third embodiment, the test pattern includes changes to multiple notification contents associated with a driver changeover, which are notified to the driver via the HMI device 70 of the vehicle 1. The intermediate pattern is a pattern in which some of the changes to the notification contents are applied to the current pattern. By conducting a step-by-step test using the intermediate pattern, it is possible to prevent the notification contents from changing significantly all at once. Therefore, it is possible to prevent driver confusion caused by changes in notification contents when the driver changes.

[0250] According to the third embodiment, an evaluation metric is set to evaluate whether the driver changeover using the intermediate pattern was properly performed. The driver changeover is evaluated using the evaluation metric. Depending on the evaluation result of the driver changeover using the evaluation metric, it is determined whether to test the test pattern next.

[0251] According to the third embodiment, if the evaluation results indicate that the driver changeover was not performed smoothly, the intermediate pattern is tested again. By repeatedly testing the intermediate pattern, it is possible to prevent the driver from becoming confused when testing the test pattern.

[0252] According to the third embodiment, if the intermediate pattern test is performed multiple times and the evaluation results indicate that the driver changeover has not been performed smoothly a predetermined number of times or more, the driver changeover test is stopped. Based on the results of the intermediate pattern, it is estimated that the driver changeover will not be performed smoothly even with the test pattern, and by canceling the test with the test pattern, it is possible to prevent unexpected situations from occurring due to the test.

[0253] Fourth Embodiment As shown in Fig. 10, the fourth embodiment is a modification of the first embodiment. The fourth embodiment will be described, focusing on the differences from the first embodiment.

[0254] In the fourth embodiment, the test is performed so that different test patterns are executed depending on the cause of the driver change. Specifically, if the driver change is caused by the ODD leaving, a test pattern that imposes a relatively high load on the driver (hereinafter referred to as a high-load pattern) is selected. If the driver change is caused by an unexpected event other than the ODD leaving, a test pattern that imposes a relatively low load on the driver (hereinafter referred to as a low-load pattern) is selected. The low-load pattern may be a pattern equivalent to the intermediate pattern described in the third embodiment.

[0255] Next, an example of a method for conducting a test related to driver changeover will be described using the flowchart in Figure 10. This series of processes from S401 to S411 may be realized, for example, by the processor 96b of the server 96 executing a computer program stored in the memory 96a, and the processor 57b of the software management unit 57 in each vehicle under test correspondingly executing a computer program stored in the memory 57a. A trigger for starting this process may be provided by an input operation by the test administrator to the server HMI 96d.

[0256] In S401, a test is planned. For example, a high-load pattern may be provided by a test administrator, and a low-load pattern may be generated by the server 96 based on the current pattern and the high-load pattern, like the intermediate pattern in the third embodiment.

[0257] The processing of S402 to S406 is the same as S102 to S106 in the first embodiment. If the answer is Yes in S406 (if the reason for the driver change is the exit of the ODD), proceed to S407. If the answer is No in S406 (if the reason for the driver change is other than the exit of the ODD), proceed to S408.

[0258] In S407, the software application unit MF1 determines to execute the high load pattern in the operation changeover that is about to be carried out. After the processing of S407, the process proceeds to S409.

[0259] In S407, the software application unit MF1 determines to execute the low load pattern in the operation changeover that is about to be carried out. After the process of S408, the process proceeds to S409.

[0260] The processes of S409 to S411 are the same as those of S109 to S111 in the first embodiment. The series of processes ends with S411.

[0261] According to the fourth embodiment described above, a high-load pattern that places a high load on the driver and a low-load pattern that places a lower load on the driver than the high-load pattern are prepared. The high-load pattern is tested when the driver change is caused by an exit from the ODD, and the low-load pattern is tested when the driver change is caused by another unexpected event. By executing a test pattern that takes the driver load into consideration, it is possible to prevent unexpected situations from occurring due to the test.

[0262] Fifth Embodiment As shown in Fig. 11, the fifth embodiment is a modification of the first embodiment. The fifth embodiment will be described, focusing on the differences from the first embodiment.

[0263] In the fifth embodiment, both the test related to the driver changeover and the test related to the MRM are performed in the same period. The test software for this test may be a single software including both the test pattern related to the driver changeover and the test pattern related to the MRM.

[0264] The test software in this test may include one test software including a test pattern related to driver changeover and another test software including a test pattern related to MRM. When the test software is configured separately, the software distribution unit 97b may distribute the test software related to driver changeover and the test software related to MRM simultaneously or at different times.

[0265] The test pattern related to MRM may be a test pattern including a notification mode in which, in association with MRM, a notification is given to the driver through the HMI device 70 of the vehicle 1. In other words, the test may be a test of content displayed on the meter display 70b1, the CID 70b2, etc. during MRM.

[0266] Next, an example of a method for conducting a driving changeover test and an MRM test will be described using the flowchart in Figure 11. This series of processes from S501 to S511 may be realized, for example, by the processor 96b of the server 96 executing a computer program stored in the memory 96a, and the processor 57b of the software management unit 57 in each vehicle under test correspondingly executing a computer program stored in the memory 57a. A trigger for starting this process may be provided by an input operation by the test administrator to the server HMI 96d.

[0267] In S501, a test is planned in the server 96 (for example, the test management unit 97a). This test plan includes both a test plan related to operation changeover and a test plan related to MRM. The processes of S502 to S504 after the process of S501 are the same as S102 to S104 in the first embodiment. After the process of S504, the process proceeds to S505.

[0268] In S505, the software application unit MF1 determines whether a situation has arisen in which a handover of driving from the driving system 2 to the driver will occur. If the answer is Yes, the process proceeds to S506. If the answer is No, the process skips S506 and proceeds to S507.

[0269] In S506, the software application unit MF1 determines to execute a test pattern related to the driver changeover in the driver changeover that is about to be carried out. After the process of S506, the process proceeds to S507.

[0270] In S507, the software application unit MF1 determines whether or not a situation in which an MRM will occur is present, based on the mode management information from the mode management unit 23. If the determination is Yes, the process proceeds to S508. If the determination is No, the process skips S508 and proceeds to S509.

[0271] In S508, the software application unit MF1 determines to execute a test pattern relating to the MRM in the MRM to be executed from now on. After the process of S508, the process proceeds to S509.

[0272] In S509, the operation measurement unit MF2 evaluates the driving changeover and MRM executed using the test pattern with the driving metrics defined in S503. After the processing of S509, the series of processes ends with S510 to S511.

[0273] According to the fifth embodiment described above, in addition to a test related to a driving changeover, a test related to MRM is performed. That is, one or more test patterns are prepared in advance, including notification modes to be notified to the driver via the HMI device 70 of the vehicle 1 in association with MRM due to autonomous driving by the driving system 2 of the vehicle 1 or MRM due to driving by the driver. Then, the test pattern related to MRM is tested in a situation in which MRM occurs. By performing both a test related to a driving changeover and a test related to MRM, it is possible to improve the efficiency of the tests.

[0274] Sixth Embodiment As shown in Fig. 12, the sixth embodiment is a modification of the third embodiment. The sixth embodiment will be described, focusing on the differences from the third embodiment.

[0275] In the fifth embodiment, similarly to the third embodiment, the software application unit MF1 in the driving system 2 prepares an intermediate pattern in addition to a test pattern. Then, before testing the test pattern, the software application unit MF1 determines to test an intermediate pattern that is less changed from the current pattern than the test pattern.

[0276] The operation measurement unit MF2 measures the time required for the driver changeover for one or both of the intermediate pattern and the test pattern one by one. The time required for the driver changeover may be the time from the time when the system notifies the driver of the initial driver changeover request to the time when the driver has completed the driver changeover.

[0277] After testing the intermediate pattern, the software application unit MF1 determines whether to test the intermediate pattern again or to test a test pattern with the added changes at the next driver change based on the time required for the driver change. For example, if the time required for the driver change is equal to or greater than a preset time threshold, the intermediate pattern may be tested again.

[0278] The operation measurement unit MF2 may output a request to the recording device 55 to associate the test content (e.g., information specifying a test pattern) with the time required for the driver changeover each time a driver changeover is performed, and store the information in the storage medium 55c. As a result, information on the elapsed time required for the driver changeover when the test is performed multiple times is recorded in the storage medium 55c. Note that if the time required for the driver changeover is less than the above-mentioned time threshold, recording may be unnecessary.

[0279] Furthermore, the operation measurement unit MF2 may transmit information similar to that stored in the storage medium 55c to the server 96 via the communication system 43. The transmission may be performed each time a driving shift is performed, or may be performed all at once after a series of tests are completed. This allows the data collection unit 97c in the server 96 to accumulate the above-mentioned progress information.

[0280] Next, an example of a method for performing a test related to driver changeover and a test related to MRM will be described using the flowchart in Figure 12. The processing of S601 to S604 is the same as S301 to S302 and S304 to S305 in the third embodiment. After processing of S604, the process proceeds to S605.

[0281] In S605, the software application unit MF1 determines whether the time required for the driver changeover in S604 is equal to or longer than a preset time. If Yes, proceed to S606. If No, proceed to S607.

[0282] In S606, the action measurement unit MF2 causes the recording device 55 to record the time required for the driver changeover in S604, and also causes the recording device 55 to transmit the time to the server 96. After the processing of S606, the process returns to S604, and the intermediate pattern is executed again.

[0283] S607 is the same as S310. That is, a test pattern in which changes are added to the intermediate pattern is executed. After processing of S607, in S608, it is determined whether the time required for the driver change is equal to or longer than a preset time. If the answer is Yes, in S609, the same process as S606 is executed, and then the process returns to S607. If the answer is No, the process proceeds to S610.

[0284] In S610, the software application unit MF1 decides to complete the test and transmits a completion notification to the server 96 via the communication system 43. S611 after the processing of S610 is the same as S111. However, in S611, the test software evaluation unit 97d may evaluate the familiarity of the intermediate pattern and the test pattern based on the progress information generated by the processing of S606 and S609.

[0285] According to the sixth embodiment described above, if the driving changeover takes a time equal to or longer than a preset time threshold after testing the intermediate pattern, the intermediate pattern is tested again. On the other hand, if the driving changeover takes a time less than the threshold, the test pattern is tested. In other words, if the driving changeover is not performed smoothly, the same pattern is repeatedly performed without adding any additional changes, thereby suppressing the risk of adding changes and enabling verification of the process until the driver becomes accustomed to the pattern.

[0286] Seventh Embodiment As shown in Fig. 13, the seventh embodiment is a modification of the first embodiment. The seventh embodiment will be described, focusing on the differences from the first embodiment.

[0287] In the seventh embodiment, the driving system 2 is a driving system capable of performing automated driving at level 3 or higher. If the driving system 2 is unable to hand over driving to the driver within a preset time after a driving change request accompanied by a test pattern test, the driving planner 22 determines to execute MRM.

[0288] After the execution of the MRM, the software application unit MF1 decides to suspend the test. The suspension here may be a suspension without any time or conditional limits, i.e., a substantial suspension with no possibility of resumption. On the other hand, the software application unit MF1 sets the test resumption conditions.

[0289] The test resumption condition may be that the vehicle's engine switch is turned off and then turned on again. The test resumption condition may be that the vehicle 1 returns home. The test resumption condition may be that the date changes. In this way, it is preferable that the test resumption condition is a condition that restricts the implementation of the test using the test pattern if a driver change occurs again immediately after the MRM.

[0290] Next, an example of a method for performing a test related to driver changeover and a test related to MRM will be described using the flowchart in Figure 13. The processing of S701 to S704 is the same as S101 to S104 in the first embodiment. After the processing of S704, the process proceeds to S705.

[0291] In S705, it is determined whether the driver changeover accompanied by the execution of the test pattern was carried out smoothly. If the answer is Yes, the process proceeds to S706 to S708. S706 to S708 are the same as S109 to S111. If the answer is No, the process proceeds to S709.

[0292] In S709, the operation planning unit 22 determines to execute MRM. In S710 after the processing of S709, the software application unit MF1 determines to suspend the test. In S711 after the processing of S710, the software application unit MF1 sets a test restart condition.

[0293] In the subsequent step S712, the software application unit MF1 determines whether the test restart condition is satisfied. If the result is Yes, the process proceeds to step S713. If the result is No, the process returns to step S712 after a predetermined time has elapsed or a trigger has occurred.

[0294] In S713, the software application unit MF1 resumes the test, and the series of processes ends with S713.

[0295] According to the seventh embodiment described above, when the MRM is executed by the autonomous driving of the driving system 2 due to a failure in testing the test pattern related to the driving changeover, it is determined to suspend the test related to the driving changeover. This prevents the test from being repeated in a situation where the test is expected to be unsuccessful due to a high driver load. In other words, it is possible to prevent an unexpected situation caused by the test.

[0296] Eighth Embodiment As shown in Fig. 14, the seventh embodiment is a modification of the first embodiment. The seventh embodiment will be described, focusing on the differences from the first embodiment.

[0297] In the server 96 of the eighth embodiment, the test management unit 97a plans a plurality of tests (first test and second test) related to the driver changeover. These tests may correspond to a second example of so-called A / B testing in which a single test vehicle is assigned a plurality of test software programs to be evaluated and compared with each other.

[0298] On the other hand, the first test and the second test may not be compared with each other and may be tests with different purposes. For example, the first test is a test to change the specifications of the information presentation function regarding driver changeover, and the second test is a test to change the mode of vehicle control during driver changeover.

[0299] In the driving system 2, the software application unit MF1 divides the first test and the second test into two driving opportunities and tests them simultaneously in parallel. Specifically, in the first driving opportunity, the software application unit MF1 performs the first test and then the second test. In the second driving opportunity, the software application unit MF1 reverses the order and performs the second test and then the first test. Here, a driving opportunity may be defined as a period from when the vehicle 1 leaves a base until when it returns to the base. The base may be a home if the vehicle 1 is owned by an individual, or a company parking lot if the vehicle 1 is owned by a company.

[0300] Next, an example of a method for conducting a test related to a driver changeover and a test related to MRM will be described with reference to the flowchart in Fig. 14. In S801, the server 96 (particularly the test management unit 97a) plans a first test and a second test. In S802 after processing S801, the server 96 transmits test software corresponding to the first test and test software corresponding to the second test to the test target vehicle, and these test software are installed in the test target vehicle.

[0301] Thereafter, a driving opportunity occurs in S803, and in S804 the software application unit MF1 identifies a situation in which a driving changeover from the driving system 2 to the driver occurs, and determines to execute a first test pattern based on the first test for that driving changeover. The driving changeover is executed using the first test pattern. In S805 after processing of S804, the software application unit MF1 identifies a situation in which a driving changeover occurs, similar to S804, and determines to execute a second test pattern based on the second test for that driving changeover. The driving changeover is executed using the second test pattern. After both tests have been executed once each, the driving opportunity ends in S806.

[0302] Thereafter, a driving opportunity occurs in S807, and in S808, the software application unit MF1 identifies a situation in which a driving change will occur, similar to S804 and S805, and determines to execute a second test pattern based on the second test for that driving change. The driving change is executed using the second test pattern. In S809 after processing S808, the software application unit MF1 identifies a situation in which a driving change will occur, similar to S808, and determines to execute a first test pattern based on the first test for that driving change. The driving change is executed using the first test pattern. After both tests have been executed once each, the driving opportunity and tests end in S810.

[0303] According to the eighth embodiment described above, one or more test patterns are prepared in advance, where a first pattern and a second pattern are prepared. Then, in testing the test patterns, the first pattern is tested at a first driving opportunity, followed by the second pattern, and the second pattern is tested at a second driving opportunity after the first driving opportunity, followed by the first pattern. In this way, the two test patterns are tested in parallel over two driving opportunities, and the order of the tests is reversed at each driving opportunity. This makes it possible to efficiently test two test patterns in a short period of time while eliminating the influence of driver familiarity, compared to repeatedly testing the same test pattern.

[0304] Ninth Embodiment As shown in Fig. 15, the ninth embodiment is a modification of the first embodiment. The ninth embodiment will be described, focusing on the differences from the first embodiment.

[0305] In the ninth embodiment, similar to the first embodiment, the test of the test pattern is prohibited depending on whether or not a test permission condition is satisfied at the time of the driver changeover. In the first embodiment, the test permission condition was that the driver changeover was not caused by the exit of the ODD, but in the ninth embodiment, a different condition is adopted.

[0306] In the ninth embodiment, the condition for permitting the test is that the surrounding environment (also referred to as the external environment) at the time of the driver changeover indicates that the risk to the test is small. The risk here may be a risk evaluated by the risk confirmation unit 26. In this case, the risk may be expressed by a safety envelope, a safety distance, or the like, and the magnitude of the risk may be determined by, for example, comparing the safety distance with a threshold value.

[0307] The risk may be evaluated based on criteria implemented in the software application unit MF1 that are different from the risk evaluated by the risk confirmation unit 26. For example, the software application unit MF1 may refer to an environmental model and determine the risk based on the number or density of vehicles present around the vehicle 1, the number or density of VRUs present around the vehicle 1, road type, road shape, etc.

[0308] If the test is forcibly carried out when the risk of the test is high, the burden on the driver who takes over driving may exceed an unacceptable level. Therefore, the software application unit MF1 prohibits the test of the test pattern and decides to carry out the driving change using the current pattern. Note that if the driver has taken over driving a predetermined number of times (e.g., 50 times) or more, the driver is accustomed to taking over driving and may be allowed to carry out the test even if the risk of the test is high.

[0309] Next, an example of a method for conducting a test related to driver changeover and a test related to MRM will be described using the flowchart in FIG. 15. Steps S901 to S905 are the same as steps S101 to S105 in the first embodiment. If the answer is Yes in step S905, the software application unit MF1 determines in step S906 whether the test permission conditions are met. If the answer is Yes, the process proceeds to step S908. If the answer is No, the process proceeds to step S907. Steps S907 to S911 are the same as steps S107 to S111 in the first embodiment.

[0310] According to the ninth embodiment described above, when testing a test pattern, if a preset test permission condition is met, the test pattern is tested, and if the condition is not met, testing of the test pattern is prohibited.

[0311] Furthermore, according to the ninth embodiment, if the vehicle's surrounding environment at the time of driver change indicates that the risk to the test is small, the test pattern is tested, and if the risk is large, the test pattern is prohibited from being tested. Since a case where the risk to the test is large is detected and the execution of the test is prevented, it is possible to operate the test appropriately.

[0312] (Other Embodiments) Although multiple embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0313] In another embodiment, when a test using a test pattern is performed multiple times, the action measurement unit MF2 may statistically process the evaluation results and then transmit the results to the server 96. The statistical processing here may involve taking the average or median of the evaluation results. The action measurement unit MF2 may sort the evaluation results according to the driver's state, which is captured and analyzed by the in-vehicle monitor serving as the internal environment sensor 42. For example, the action measurement unit MF2 may exclude evaluation results when the driver is highly focused and transmit only evaluation results when the driver is less focused (e.g., sleepy) to the server 96. Evaluating the test pattern based on evaluation results when the driver is less focused can improve the robustness of the vehicle's system specifications regarding driver changes.

[0314] In another embodiment, the operation measurement unit MF2 may not set and calculate the evaluation metric itself to evaluate the driving change, but may transmit the measured data of the vehicle state to the server 96. In the server 96, the test software evaluation unit 97d may set the evaluation metric and evaluate the data of the vehicle state collected by the data collection unit 97c using the evaluation metric.

[0315] In another embodiment, the processes of S105 to S106 in FIG. 5 that were executed by the software application unit MF1 may be executed by the planning unit 20 in the main unit 51 instead of the software management unit 57.

[0316] As another example of the third embodiment, when both the intermediate pattern and the test pattern are tested and evaluated, the intermediate pattern may yield better evaluation results. In this case, the software including the intermediate pattern may be adopted as the official software.

[0317] As another embodiment of the fifth embodiment, the test related to the driver changeover may not be performed, and the test related to the MRM may be performed alone.

[0318] In other embodiments, the processing system 50 may have a configuration as shown in Figures 16 and 17. For example, Figure 16 shows a configuration including multiple domain controllers 451 to 454. Each of the domain controllers 451 to 454 may have the same hardware configuration as the processing system or ECU of the first embodiment.

[0319] The ADAS domain controller 451 aggregates functions related to ADAS (Advanced Driver-Assistance Systems). The ADAS domain controller 451 may comprehensively realize a portion of the recognition function, a portion of the judgment function, and a portion of the control function. The portion of the recognition function realized by the ADAS domain controller 451 may be, for example, a function corresponding to the fusion of information detected by the multiple sensors 40 in the detection unit 10 of the first embodiment, or a simplified function thereof. The portion of the judgment function realized by the ADAS domain controller 451 may be, for example, a function corresponding to the prediction unit 21 and the driving planner 22 of the first embodiment, or a simplified function thereof. The portion of the control function realized by the ADAS domain controller 451 may be, for example, a function corresponding to the motion control unit 31 of the first embodiment, that generates request information for the motion actuator 60.

[0320] The powertrain domain controller 452 aggregates functions related to the control of the powertrain. The powertrain domain controller 452 may compositely realize at least a part of the recognition function and at least a part of the control function. A part of the recognition function realized by the powertrain domain controller 452 may be, for example, a function corresponding to the internal recognition unit 13 in the first embodiment, which recognizes the driver's operation state with respect to the motion actuator 60. A part of the control function realized by the powertrain domain controller 452 may be, for example, a function corresponding to the motion control unit 31 in the first embodiment, which controls the motion actuator 60.

[0321] The cockpit domain controller 453 aggregates functions related to the cockpit. The cockpit domain controller 453 may compositely realize at least a part of the recognition function and at least a part of the control function. A part of the recognition function realized by the cockpit domain controller 453 may be, for example, a function of recognizing the switch state of the HMI device 70, which is included in the internal recognition unit 13 of the first embodiment. A part of the control function realized by the cockpit domain controller 453 may be, for example, a function corresponding to the HMI output unit 71 of the first embodiment.

[0322] The connectivity domain controller 454 aggregates connectivity-related functions. The connectivity domain controller 454 may comprehensively realize at least a part of the recognition function. Part of the recognition function realized by the connectivity domain controller 454 may be a function to organize and convert the global position data of the vehicle, V2X information, etc. acquired from the communication system 43 into a format usable by the ADAS domain controller 451 and the cockpit domain controller 453, for example.

[0323] In this embodiment, when software management is performed by each domain controller 451 to 454 that realizes the respective functions, the multiple domain controllers 451 to 454 may each correspond to a "software management unit" that constitutes the management system MS or the test system.

[0324] 17 employs a configuration including an integrated ECU 551 and multiple zone ECUs 551a to 551d. In this configuration, the multiple zone ECUs 551a to 551d control devices, modules, units, equipment, etc. that are located in specific zones of the vehicle 1 that they are responsible for.

[0325] For example, an external environment sensor 41 such as a camera arranged in the front of the vehicle 1, and an information presentation device 70b such as a CID 70b2 arranged in the cockpit, are controlled by a zone ECU 551a or 551b arranged in the front of the vehicle 1. For example, an external environment sensor 41 such as a millimeter wave radar arranged in the rear of the vehicle 1 is controlled by a zone ECU 551c or 551d arranged in the rear of the vehicle 1.

[0326] The integrated ECU 551 collects detection information and the like from each of the zone ECUs 551a to 551d and performs integrated control of the driving system 2 for each of the zone ECUs 551a to 551d. For example, the integrated ECU 551 may implement almost all of the planning function and software management function.

[0327] In another embodiment, the vehicles 1, 1A, and 1B equipped with the driving system 2 may be right-hand drive vehicles or left-hand drive vehicles. Furthermore, the traffic environment in which the vehicles 1, 1A, and 1B travel may be a traffic environment where traffic is assumed to be on the left side of the road, or a traffic environment where traffic is assumed to be on the right side of the road. The driving system 2 according to the present disclosure may be optimized as appropriate, taking into consideration the road traffic laws and customs of each country and region, as well as the forms of police investigations, prosecutions, criminal proceedings, and civil proceedings regarding traffic accidents.

[0328] The controller and methods described herein may be implemented by a special-purpose computer comprising a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and methods described herein may be implemented by special-purpose hardware logic circuitry. Alternatively, the apparatus and methods described herein may be implemented by one or more special-purpose computers comprising a processor executing a computer program in combination with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0329] (Disclosure of Technical Ideas) This specification discloses multiple technical ideas described in the following multiple clauses. Some clauses may be described in a multiple dependent form, where the subsequent clause alternatively cites the preceding clause. These multiple dependent clauses define multiple technical ideas.

[0330] <Technical Idea 1> A test execution device having at least one processor (57b) for conducting tests in a vehicle (1, 1A, 1B) that can participate in public road traffic, wherein the at least one processor: prepares in advance one or more test patterns related to a driving switch between autonomous driving by the vehicle's driving system (2) and driving by a driver; and tests the test pattern related to the driving switch in a situation where the driving switch from the driving system to the driver occurs.

[0331] <Technical Idea 2> The test execution device described in Technical Idea 1, wherein the at least one processor further performs the following: evaluating the driving change using the test pattern with a metric that evaluates whether the driving change was performed appropriately; and recording the evaluation result of the driving change using the metric in a storage medium (55c).

[0332] <Technical Idea 3> The test execution device described in Technical Idea 1 or 2, wherein the at least one processor further controls an HMI device (70) of the vehicle so that an explanation about the test regarding the driving change is given to the driver after the driver gets into the vehicle and before the autonomous driving starts.

[0333] <Technical Idea 4> The test execution device according to Technical Idea 1 or 2, wherein the at least one processor further controls an HMI device (70) of the vehicle so that, in a situation where the driving changeover from the driving system to the driver occurs, an explanation about the test regarding the driving changeover is given to the driver after a current pattern other than the test pattern is executed.

[0334] <Technical Idea 5> The test execution device described in Technical Idea 3 or 4, wherein the at least one processor further controls the HMI device of the vehicle to obtain consent from the driver for the driving change test, following the explanation.

[0335] <Technical Idea 6> The test execution device described in any one of Technical Ideas 1 to 5, wherein the at least one processor further performs the following: preparing an intermediate pattern generated by extracting a difference between the test pattern and a current pattern other than the test pattern, the intermediate pattern being obtained by applying a part of the difference to the current pattern; and testing the intermediate pattern in a situation where the driving changeover from the driving system to the driver occurs before testing the test pattern.

[0336] <Technical Idea 7> The test pattern includes changes to multiple notification contents associated with the driver change, which are notified to the driver through the vehicle's HMI device (70), and the intermediate pattern is a pattern obtained by applying some of the changes to the multiple notification contents from the current pattern, in the test execution device described in Technical Idea 6.

[0337] <Technical Idea 8> The test execution device according to Technical Idea 6 or 7, wherein the at least one processor further executes the following: after testing the intermediate pattern, if the driving changeover takes a time equal to or longer than a preset time threshold, testing the intermediate pattern again; and if the driving changeover takes a time less than the time threshold, testing the test pattern.

[0338] <Technical Idea 9> The test execution device described in Technical Idea 6 or 7, wherein the at least one processor further performs the following: evaluating the driving change using the intermediate pattern with a metric that evaluates whether the driving change was executed appropriately; and determining whether to test the test pattern depending on the evaluation result of the driving change using the metric.

[0339] <Technical Idea 10> The test execution device according to Technical Idea 9, wherein the at least one processor, in determining, decides to test the intermediate pattern again if the evaluation result indicates that the driving changeover was not performed smoothly.

[0340] <Technical Idea 11> The test execution device described in Technical Idea 9 or 10, wherein testing the intermediate pattern is executed multiple times, and the at least one processor, in determining, decides to stop the test regarding the driver change if the evaluation result shows that the driver change has not been executed smoothly a predetermined number of times or more.

[0341] <Technical Idea 12> The test execution device described in any one of Technical Ideas 1 to 11, wherein the at least one processor, when testing the test pattern, tests the test pattern if a preset test permission condition is met, and prohibits testing of the test pattern if the condition is not met.

[0342] <Technical Idea 13> The at least one processor, in testing the test pattern, tests the test pattern if the driver change is caused by an exit from an operation design area, and prohibits testing of the test pattern if the driver change is caused by any other sudden event, a test execution device described in any one of Technical Ideas 1 to 12.

[0343] <Technical Idea 14> The test execution device described in any one of Technical Ideas 1 to 12, wherein the at least one processor, when testing the test pattern, tests the test pattern if the surrounding environment of the vehicle at the time of the driver change indicates that the risk to the test is small, and prohibits testing of the test pattern if the risk indicates that the risk is large.

[0344] <Technical Idea 15> The at least one processor prepares one or more test patterns in advance, preparing a high-load pattern that places a high load on the driver and a low-load pattern that places a lower load on the driver than the high-load pattern, and in testing the test patterns, tests the high-load pattern if the driver change is caused by exiting an operation design area, and tests the low-load pattern if the driver change is caused by some other sudden event, a test execution device described in any one of Technical Ideas 1 to 5.

[0345] <Technical Idea 16> The at least one processor, in preparing one or more test patterns in advance, prepares a first pattern and a second pattern, and in testing the test patterns, tests the first pattern at a first running opportunity, then tests the second pattern, and tests the second pattern at a second running opportunity after the first running opportunity, this is the test execution device described in Technical Idea 1.

[0346] <Technical Idea 17> The test execution device described in any one of Technical Ideas 1 to 16, wherein the at least one processor further performs the following: preparing in advance one or more test patterns related to the MRM, including a notification mode to be notified to the driver through an HMI device (70) of the vehicle in association with an MRM caused by autonomous driving by the driving system or driving by the driver; and testing the test pattern related to the MRM in a situation in which the MRM occurs.

[0347] <Technical Idea 18> The test execution device described in any one of Technical Ideas 1 to 16, wherein the at least one processor further determines to interrupt the test related to the driving change when an MRM is executed by autonomous driving of the driving system due to a failure to test the test pattern related to the driving change.

[0348] <Technical Idea 19> A test execution device having at least one processor for conducting tests in a vehicle that can participate in public road traffic, wherein the at least one processor performs the following actions: prepares in advance one or more test patterns related to MRM, including a notification mode to be notified to the driver through an HMI device of the vehicle, in association with MRM caused by autonomous driving by the driving system of the vehicle or by driving by the driver; and tests the test pattern related to the MRM in a situation where the MRM occurs.

[0349] According to Technical Idea 19, it becomes possible to not miss MRM events that occur in vehicles that can participate in public road traffic and to use them for testing test patterns, thereby promoting V&V related to MRM and making it easier to appropriately improve the vehicle system specifications.

[0350] <Technical Idea 20> A test system comprising a plurality of vehicles (1, 1A, 1B) equipped with test execution devices (57) and capable of participating in public road traffic, and a server (96) communicably connected to the vehicles, wherein the server distributes one or more test patterns related to the driving switch to each of the vehicles based on a test plan related to the driving switch between autonomous driving by the driving system (2) of the vehicle and driving by a driver, and each of the test execution devices tests the test pattern related to the driving switch in a situation where the driving switch from the driving system to the driver occurs.

[0351] According to technical idea 20, it is possible to not miss any driver change events that occur in each vehicle that can participate in public road traffic and to use them to test test patterns, thereby promoting V&V regarding driver changes and making it easier to appropriately improve the vehicle system specifications.

[0352] <Technical Idea 21> The test system according to Technical Idea 20, wherein each of the test execution devices further performs the following: evaluating the driving change using the test pattern with a metric that evaluates whether the driving change was executed appropriately; and transmitting the evaluation result of the driving change using the metric to the server.

[0353] <Technical Idea 22> The test system described in Technical Idea 20, wherein each of the test execution devices further measures a vehicle state during the driving changeover using the test pattern and transmits the vehicle state data to the server, and the server further evaluates the driving changeover using the test pattern using a metric that evaluates whether the driving changeover was performed appropriately based on the received vehicle state data.

[0354] <Technical Idea 23> A method for generating data for improving a vehicle's system specifications, the method comprising: testing a test pattern related to the driving changeover in a situation where a driving changeover occurs from the vehicle's driving system to a driver; evaluating the driving changeover using the test pattern using a metric that evaluates whether the driving changeover was executed appropriately; and generating data indicating the evaluation results of the driving changeover evaluated using the metric.

[0355] According to technical idea 23, test patterns are tested during a driver changeover event that occurs in a vehicle, and the results are evaluated using evaluation metrics to generate data indicating the evaluation results obtained, thereby facilitating V&V regarding driver changes and making it easier to appropriately improve the vehicle's system specifications.

Claims

1. A test implementation device comprising at least one processor (57b) for conducting tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, The aforementioned at least one processor is One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where the driver change occurs from the driving system to the driver, the following is performed: Test the test pattern related to the driver change, The test pattern is a test pattern implemented by downloading test software, which is different from the current pattern implemented by the official software, and includes changes to the trajectory plan and behavior plan in the autonomous operation of the vehicle from immediately before the driver change to the driver change, in the test execution device.

2. A test implementation method for conducting tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. This includes testing test patterns related to the driver change in a situation where a driver change occurs from the driving system to the driver, The test pattern is a test pattern implemented by downloading test software, which is different from the current pattern implemented by the official software, and is a test pattern that includes changes to the trajectory plan and behavior plan in the autonomous operation of the vehicle from immediately before the driver change to the driver change.

3. A test implementation program that conducts tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, At least one processor (57b) One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where a driver change occurs from the aforementioned driving system to the aforementioned driver, the following is performed: Test the test pattern related to the driver change, The aforementioned test pattern is a test pattern implemented by downloading test software, which is different from the current pattern implemented by the official software, and is a test implementation program that includes changes to the trajectory plan and behavior plan in the autonomous operation of the vehicle from immediately before the driver change to the driver change.

4. A test implementation device comprising at least one processor (57b) for conducting tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, The aforementioned at least one processor is One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where the driver change occurs from the driving system to the driver, the test pattern related to the driver change is tested, A test execution device that controls the vehicle's HMI device (70) so that, in a situation where the driver change occurs from the driving system to the driver, after a current pattern different from the test pattern is executed, an explanation of the test regarding the driver change is given to the driver.

5. A test implementation device comprising at least one processor (57b) for conducting tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, The aforementioned at least one processor is One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where the driver change occurs from the driving system to the driver, the test pattern related to the driver change is tested, An intermediate pattern is generated by extracting the difference between the aforementioned test pattern and a current pattern that is different from the aforementioned test pattern, and an intermediate pattern is prepared in which a part of the aforementioned difference is applied to the current pattern. A test execution device that performs the following: testing the intermediate pattern in a situation in which the driver change from the driving system to the driver occurs, before testing the aforementioned test pattern.

6. A test implementation device comprising at least one processor (57b) for conducting tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, The aforementioned at least one processor is One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where the driver change occurs from the driving system to the driver, the following is performed: Test the test pattern related to the driver change, A test execution device comprising at least one processor, which, in testing the test pattern, tests the test pattern if the driver change is caused by exiting the operational design domain, and prohibits testing the test pattern if the driver change is caused by any other sudden event.

7. A test implementation device comprising at least one processor (57b) for conducting tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, The aforementioned at least one processor is One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where the driver change occurs from the driving system to the driver, the following is performed: Test the test pattern related to the driver change, The aforementioned at least one processor is In preparing one or more test patterns in advance, a high-load pattern that places a high load on the driver and a low-load pattern that places a lower load on the driver than the high-load pattern are prepared. A test device that, in testing the aforementioned test patterns, tests the high-load pattern when the driver change is caused by exiting the operational design domain, and tests the low-load pattern when the driver change is caused by other sudden events.

8. A test implementation device comprising at least one processor (57b) for conducting tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, The aforementioned at least one processor is One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where the driver change occurs from the driving system to the driver, the following is performed: Test the test pattern related to the driver change, The aforementioned at least one processor is In preparing one or more test patterns in advance, a first pattern and a second pattern are prepared, A test execution device for testing the aforementioned test patterns, wherein in a first running opportunity, the first pattern is tested, followed by the second pattern, and in a second running opportunity that occurs after the first running opportunity, the second pattern is tested, followed by the first pattern.

9. A test method for conducting tests in vehicles (1, 1A, 1B) capable of participating in public road traffic, using at least one processor (57b), One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where a driver change occurs from the aforementioned driving system to the aforementioned driver, the test pattern related to the driver change is tested, A test execution method comprising controlling the vehicle's HMI device (70) such that, in a situation where the driver change occurs from the driving system to the driver, an explanation of the test regarding the driver change is given to the driver after a current pattern different from the test pattern has been executed.

10. A test method for conducting tests in vehicles (1, 1A, 1B) capable of participating in public road traffic, using at least one processor (57b), One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where a driver change occurs from the aforementioned driving system to the aforementioned driver, the test pattern related to the driver change is tested, An intermediate pattern is generated by extracting the difference between the aforementioned test pattern and a current pattern that is different from the aforementioned test pattern, and an intermediate pattern is prepared in which a part of the aforementioned difference is applied to the current pattern. A test execution method comprising testing the intermediate pattern in a situation in which the driver change from the driving system to the driver occurs, before testing the aforementioned test pattern.

11. A test method for conducting tests in vehicles (1, 1A, 1B) capable of participating in public road traffic, using at least one processor (57b), One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. This includes testing test patterns related to the driver change in a situation where a driver change occurs from the driving system to the driver, A test implementation method further comprising testing the aforementioned test pattern, wherein the test pattern is tested if the driver change is caused by exiting the operational design domain, and testing of the test pattern is prohibited if the driver change is caused by any other sudden event.

12. A test method for conducting tests in vehicles (1, 1A, 1B) capable of participating in public road traffic, using at least one processor (57b), One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. This includes testing test patterns related to the driver change in a situation where a driver change occurs from the driving system to the driver, In preparing one or more test patterns in advance, a high-load pattern that places a high load on the driver and a low-load pattern that places a lower load on the driver than the high-load pattern are prepared. A test implementation method further comprising testing the aforementioned test patterns, wherein the high-load pattern is tested when the driver change is caused by exiting the operational design domain, and the low-load pattern is tested when the driver change is caused by any other sudden event.

13. A test method for conducting tests in vehicles (1, 1A, 1B) capable of participating in public road traffic, using at least one processor (57b), One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. This includes testing test patterns related to the driver change in a situation where a driver change occurs from the driving system to the driver, In preparing one or more test patterns in advance, a first pattern and a second pattern are prepared, A test method for testing the aforementioned test patterns, further comprising: testing the first pattern in a first driving opportunity, then testing the second pattern; and testing the second pattern in a second driving opportunity that occurs after the first driving opportunity, then testing the first pattern.

14. A test implementation program that conducts tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, At least one processor (57b) One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where a driver change occurs from the aforementioned driving system to the aforementioned driver, the test pattern related to the driver change is tested, A test execution program that controls the vehicle's HMI device (70) to perform the following actions when a driver change occurs from the driving system to the driver, after a current pattern different from the test pattern is executed, an explanation of the test regarding the driver change is given to the driver.

15. A test implementation program that conducts tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, At least one processor (57b) One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where a driver change occurs from the aforementioned driving system to the aforementioned driver, the test pattern related to the driver change is tested, An intermediate pattern is generated by extracting the difference between the aforementioned test pattern and a current pattern that is different from the aforementioned test pattern, and an intermediate pattern is prepared in which a part of the aforementioned difference is applied to the current pattern. A test execution program that, before testing the aforementioned test pattern, causes the program to test the intermediate pattern in a situation where the driver change occurs from the driving system to the driver.

16. A test implementation program that conducts tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, At least one processor (57b) One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where a driver change occurs from the aforementioned driving system to the aforementioned driver, the test pattern related to the driver change is tested, A test execution program that, in testing the aforementioned test pattern, performs the test pattern if the driver change is caused by exiting the operational design domain, and prohibits testing the test pattern if the driver change is caused by any other sudden event.

17. A test implementation program that conducts tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, At least one processor (57b) One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where a driver change occurs from the aforementioned driving system to the aforementioned driver, the test pattern related to the driver change is tested, In preparing one or more test patterns in advance, this includes preparing a high-load pattern that places a high load on the driver and a low-load pattern that places a lower load on the driver than the high-load pattern. A test execution program that, in testing the aforementioned test patterns, tests the high-load pattern when the driver change is caused by exiting the operational design domain, and tests the low-load pattern when the driver change is caused by any other sudden event.

18. A test implementation program that conducts tests on vehicles (1, 1A, 1B) that are capable of participating in public road traffic, At least one processor (57b) One or more test patterns are prepared in advance regarding the transition between autonomous driving by the vehicle's driving system (2) and driving by a driver. In a situation where a driver change occurs from the aforementioned driving system to the aforementioned driver, the test pattern related to the driver change is tested, In preparing one or more test patterns in advance, this involves preparing a first pattern and a second pattern, A test execution program that, in testing the aforementioned test patterns, performs the following actions: in a first driving opportunity, tests the first pattern, then tests the second pattern; and in a second driving opportunity that occurs after the first driving opportunity, tests the second pattern, then tests the first pattern.