Test system, test method, and software management device

Abstract

A management system (MS) that serves as a test system uses at least one vehicle (1) which can participate in public road traffic to test a driving system (2) that serves as a vehicle system mounted on the vehicle (1). The management system (MS) comprises: a test executing unit (MS1) that configures and executes a test; and a test stopping unit (MS2) that has both of an individual stop function (SFi) that individually stops the test in each participation unit and a complete stop function (SFw) that completely stops the test.

Description

Test system, test method and software management device CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2023-208762 filed in Japan on December 11, 2023, and the contents of the original application are incorporated by reference in their entirety.

[0002] TECHNICAL FIELD This disclosure relates to techniques for testing vehicle systems.

[0003] Patent Document 1 discloses a technique for testing a vehicle system installed in a vehicle using a vehicle participating in public road traffic.

[0004] U.S. Pat. No. 10,255,168

[0005] Now, vehicle systems are becoming more complex year by year, and in order to deal with these complex systems, it is becoming increasingly important to provide a V&V process for vehicles after they have been released to the market. In vehicle system tests using vehicles participating in public road traffic, such as those in Patent Document 1, the consideration of means for stopping the test can become a new issue from the standpoint of the validity and usability of the test operation.

[0006] One of the purposes of the disclosure of this specification is to provide a test system, a test method, and a software management device that are capable of appropriately stopping a test.

[0007] One aspect disclosed herein is a test system for testing vehicle systems installed in one or more vehicles capable of participating in public road traffic, comprising: a test execution unit that sets and executes the test; and a test stop unit that has both an individual stop function that stops the test individually on a participant basis and a global stop function that stops the test as a whole when the test is in a preparation state or execution state.

[0008] According to this aspect, it is possible to selectively use the individual stopping function for each participant and the overall stopping function, which makes it possible to stop the test in an appropriate manner, taking into consideration the validity of the test operation and usability.

[0009] Another aspect disclosed herein is a test method for testing vehicle systems installed in a plurality of vehicles capable of participating in public road traffic, comprising: determining, in each vehicle, using one or more metrics, whether to enable an individual stop function that stops the test individually on a vehicle-by-vehicle basis; and determining, based on the status of the individual stop functions being enabled in the plurality of vehicles, whether to enable a global stop function that stops the test as a whole.

[0010] According to this aspect, whether to stop the test is determined individually for each vehicle using metrics, and whether to stop the test as a whole is determined based on the status of multiple vehicles. By determining the stop function in this way in stages, it is possible to stop the test in an appropriate manner.

[0011] Another aspect disclosed herein is a software management device having at least one processor that manages software used in a vehicle system to test a vehicle system installed in one or more vehicles capable of participating in public road traffic, wherein the at least one processor performs the following operations: downloading test software from a server to the hardware of the vehicle system, separate from conventional software, in a test preparation state; leaving both the conventional software and the test software in the hardware in a state in which one or both of the software can be enabled; and when the vehicle in which the vehicle system is installed stops testing, enabling the conventional software and disabling the test software while leaving both software in the hardware.

[0012] According to this aspect, when the vehicle stops the test, the test software remains in the hardware of the vehicle system. This makes it easy to resume the test. Furthermore, since the software state at the time of the test is saved, it becomes easy to verify the test after the fact. Therefore, it becomes possible to stop the test in an appropriate manner.

[0013] Note that the symbols in parentheses included in the claims etc. are intended to exemplify the correspondence with the parts of the embodiments described below, and are not intended to limit the technical scope.

[0014] 1 is a diagram showing an example of the hardware configuration of a driving system, etc.; FIG. 1 shows the functional configuration of a driving system; FIG. 1 shows an example of the configuration of a cockpit; FIG. 2 shows an overview of a management system; FIG. 2 shows the functional configuration of a management system; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A diagram showing an example of notification using an HMI; A flowchart illustrating a test-related process; A diagram showing an example of notification using an HMI; A flowchart illustrating a test-related process; A flowchart illustrating a test-related process; A diagram showing an example of the hardware configuration of a processing system; A diagram showing an example of the hardware configuration of a processing system.

[0015] Hereinafter, several embodiments will be described with reference to the drawings. Note that corresponding components in each embodiment are given the same reference numerals, and redundant description may be omitted. When only a portion of the configuration is described in each embodiment, the configuration of another embodiment described previously can be applied to the remaining portion of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of several embodiments can also be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.

[0016] In the following embodiments, the contents of “Safety First for Automated Driving” by Aptiv, Audi, Baidu, BMW, Continental, Daimler, FCA, hereby, Infineon, Intel, and Volkswagen, Tech. Rep., 2019, the contents of ISO 21448:2022, and the contents of IEEE 2846-2022 are incorporated by reference in their entirety.

[0017] (Explanation of Terms) Terms related to the disclosure of this specification are explained below. This explanation is included in the embodiments of the specification.

[0018] A road user may be a traffic participant on or adjacent to an active road for the purpose of traveling from one location to another.

[0019] A dynamic driving task (DDT) may be a real-time operational and tactical function for operating a vehicle in traffic, and a DDT may be all real-time operational and tactical functions for operating a vehicle on a roadway.

[0020] An automated driving system (ADS) may be a collection of hardware and software capable of executing the entire DDT on a continuous basis, whether or not it is limited to a specific operational design domain.

[0021] An ADS feature may be the design-specific functionality of the ADS within a particular ODD at a given level of autonomous driving.

[0022] DDT fallback may be a response by a driver or automated system to either perform DDT or transition to a minimal risk state after a failure occurs or upon detection of a malfunction or potentially dangerous behavior. DDT fallback may also be a method of controlling the transition from autonomy to driver or other system control using takeover / fallback states and associated use cases. DDT fallback may also be a user response to perform DDT or achieve MRC after a system failure related to DDT performance occurs or upon deviation from ODD, or a response by an ADS to achieve MRC given the same circumstances.

[0023] A Minimal Risk Condition (MRC) may be a state of the vehicle to reduce the risk if a given trip cannot be completed, or may be a stable, stopped state that a user or ADS places on the vehicle after a DDT fallback is performed to reduce the risk of an accident if a given trip cannot or should not be continued.

[0024] An operational design domain (ODD) may be the specific conditions in which a given (automated) driving system is designed to function, or the operating conditions in which a given ADS or its functions are specifically designed to function, including, but not limited to, environmental, geographic, time-of-day restrictions, and / or the presence or absence of requirements for certain traffic and road characteristics.

[0025] Safety of the intended functionality (SOTIF) may be the absence of undue risk due to insufficient functionality of the intended functionality or its implementation.

[0026] A driving policy may be a strategy and rules that define control behavior at the vehicle level.

[0027] The situation is a factor that can affect the behavior of the system, and may include traffic conditions, weather, and the behavior of the vehicle itself.

[0028] A scenario may be a description of the temporal relationships between several scenes in a sequence of scenes, including the goals and values ​​in a specific situation influenced by actions and events, and a description of a continuous time sequence of activities that integrates a subject vehicle, all its external environments, and their interactions in the process of performing a specific driving task.

[0029] A safety-relevant object may be any dynamic or static object that may be relevant to the safety performance of the DDT.

[0030] Reasonably foreseeable may be technically reliable and have a reliable or measurable rate of occurrence.

[0031] A triggering condition may be a specific condition of a scenario that acts as a catalyst for subsequent system responses that contribute to unsafe behavior, failure to prevent, detect, and mitigate reasonably foreseeable indirect misuse.

[0032] A Minimal Risk Maneuver (MRM) may be a vehicle movement commanded by the automated driving system during DDT fallback to achieve an MRC.

[0033] Risk acceptance criteria / criterion are standards that represent the absence of unreasonable levels of risk, and may be, for example, physical parameters that define when a particular behavior is considered undesirable, a maximum number of accidents per hour, as low as reasonably practicable, etc.

[0034] A proper response may be an action that is significant to avoid or ameliorate a dangerous situation in a reasonably foreseeable scenario in which other safety-related objects are operating within expected bounds.

[0035] A safety-related model may be a representation of safety-related aspects of driving behavior based on assumptions about the reasonably foreseeable behavior of other road users. A safety-related model may be an on-board or off-board safety verification or analysis device, a mathematical model, a more conceptual set of rules, a set of scenario-based behaviors, or a combination of these.

[0036] A formal model may be a model expressed in a formal notation that is used to verify system performance.

[0037] A safety envelope may be a set of limits and conditions within which an (automated) driving system is designed to operate, subject to constraints or controls, in order to maintain operation within an acceptable level of risk. A safety envelope may be a general concept that can be used to accommodate all principles to which a driving policy can adhere, according to which an ego-vehicle operated by an (automated) driving system may have one or more boundaries around it.

[0038] Verification may be an activity to determine that operation of a vehicle equipped with an (autonomous) driving system achieves the safety of a defined (autonomous) driving system application in the intended environment.

[0039] Validation may be an activity to determine that a test object meets specified requirements.

[0040] A positive risk balance may be a criterion that demonstrates that a technical solution achieves an acceptable level of residual risk.

[0041] Object and event detection and response (OEDR) may be a subtask of DDT that involves monitoring the driving environment and executing appropriate responses to such objects and events.

[0042] Response time may be the time it takes a road user in a given scenario to perceive a particular stimulus and begin to execute a response (braking, steering, accelerating, stopping, etc.).

[0043] (First embodiment) <Driving system> The driving system 2 of the first embodiment shown in FIG. 1 realizes functions related to driving a vehicle 1. The driving system 2 may be a vehicle system itself, or may be a component that constitutes part of a vehicle system. Part or all of the driving system 2 is mounted on the vehicle 1. This vehicle 1 may be referred to as a subject vehicle, a host vehicle, or the like. The vehicle 1 may be configured to be able to communicate with other vehicles, etc., directly or indirectly via a communication infrastructure. The other vehicles may be referred to as target vehicles.

[0044] The vehicle 1 may be a road user capable of manual driving, such as a four-wheeled automobile or truck. The vehicle 1 may also be capable of automated driving. Autonomous driving may also be referred to as autonomous driving by the driving system 2. Driving is classified into levels according to the extent to which a human driver performs all dynamic driving tasks (DDTs). Automation levels are specified, for example, in SAE J3016. At levels 0 to 2, the driver performs some or all of the DDTs. Levels 0 to 2 may be classified as so-called manual driving. Level 0 indicates that driving is not automated. Level 1 indicates that the driving system 2 assists the driver. Level 2 indicates that driving is partially automated.

[0045] At levels 3 and above, while the ADS feature is activated, the driving system 2 performs all of the DDT. Levels 3 to 5 may be classified as so-called automated driving. A system capable of driving at level 3 or above may be called an automated driving system (ADS). A vehicle equipped with an automated driving system or a vehicle capable of driving at level 3 or above may be called an automated vehicle (AV).

[0046] Level 3 indicates conditional automation of driving. A level 3 automated driving system performs DDT but does not perform DDT fallback. That is, DDT fallback is performed by a driver who is ready for fallback. Level 4 indicates highly automated driving. A level 4 automated driving system performs DDT and DDT fallback. A level 4 automated driving system can hand over DDT to the driver after reaching a minimal risk condition (MRC) by performing DDT fallback, etc. Level 5 indicates fully automated driving. Taking over DDT between the driving system 2 and a human driver is also called delegation of authority.

[0047] The conditions for executing Level 3 and 4 autonomous driving may include some or all of the conditions indicated by the operational design domain (ODD). For example, the ADS function may be defined within the scope of the ODD. The driving system 2 described in this embodiment is a driving system capable of executing Level 3 or higher autonomous driving.

[0048] The driving system 2 provides functions such as automated driving to a vehicle user of the vehicle 1 that can participate in public road traffic. For example, the vehicle user may be a driver riding in the vehicle 1. The vehicle user may be a passenger riding in the vehicle 1. For example, if the vehicle 1 is a personally owned vehicle (POV), the vehicle user may be the owner of the vehicle 1. For example, if the vehicle 1 is used for MaaS (Mobility as a Service), the vehicle user may be an operations manager that manages the operation of the vehicle 1.

[0049] The architecture of the driving system 2 is selected to enable an efficient safety of the intended functionality (SOTIF) process. For example, the architecture of the driving system 2 may be configured based on a sense-plan-act model. The sense-plan-act model includes a sense element, a plan element, and an act element as major system elements. The sense element, plan element, and act element interact with each other. Here, sense may be replaced with perception, plan with determine, and act with control, respectively.

[0050] At the technical level (i.e., from a technical perspective), the driving system 2 implements at least a plurality of sensors 40 corresponding to sensing functions, at least one processing system 50 corresponding to planning functions, and a plurality of motion actuators 60 corresponding to acting functions. At the functional level (i.e., from a functional perspective), the sensing, planning, and acting functions are implemented (see also FIG. 2 ).

[0051] In detail, a detection unit 10 serving as a processing unit for realizing a detection function may be constructed in the driving system 2, mainly consisting of a plurality of sensors 40, a processing system 50 that processes detection information from the plurality of sensors 40, and the processing system 50 that generates an environmental model based on information from the plurality of sensors 40. A planning unit 20 and a risk confirmation unit 26 serving as processing units for realizing a planning function may be constructed in the driving system 2, mainly consisting of a plurality of motion actuators 60 and at least one processing system 50 that outputs operation signals for the plurality of motion actuators 60.

[0052] Here, the detection unit 10 may be realized in the form of a detection system serving as a subsystem provided so as to be distinguishable from the planner 20 and the action unit 30. The planner 20 may be realized in the form of a planning system serving as a subsystem provided so as to be distinguishable from the detection unit 10 and the action unit 30. The planning system may include a risk confirmation function. The risk confirmation function may be mounted in the operation system 2 independently of the detection unit 10, the planner 20, and the action unit 30. The action unit 30 may be realized in the form of an action system serving as a subsystem provided so as to be distinguishable from the detection unit 10 and the planner 20. The detection system, the planning system, and the action system may constitute components independent of each other. The subsystem referred to here may be replaced with a module, a unit, a device, etc.

[0053] The detection unit 10 is responsible for detection functions, including localization (e.g., location estimation) of road users such as the vehicle 1 and other vehicles. The detection unit 10 detects the external environment, internal environment, vehicle state, and the state of the driving system 2 of the vehicle 1. The detection unit 10 fuses the detected information to generate an environmental model. The environmental model may also be referred to as a world model. The planner 20 applies the objective and driving policy to the environmental model generated by the detection unit 10 to derive control actions. The behavior unit 30 executes the control actions derived by the planner 20.

[0054] <Physical Architecture> An example of the physical architecture of the driving system 2 will be described using Figure 1. The driving system 2 includes a plurality of sensors 40, a plurality of motion actuators 60, a plurality of HMI devices 70, and at least one processing system 50. These components can communicate with each other via one or both of wireless and wired connections. These components may also be able to communicate with each other through an in-vehicle network such as CAN (registered trademark).

[0055] The plurality of sensors 40 includes one or more external environment sensors 41. Furthermore, the plurality of sensors 40 may include at least one of one or more internal environment sensors 42, one or more communication systems 43, and a map database (DB) 44.

[0056] The external environment sensor 41 may detect targets present in the external environment of the vehicle 1. Target detection type external environment sensors 41 include, for example, a camera, LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging), laser radar, millimeter wave radar, ultrasonic sonar, etc. Typically, a combination of multiple types of external environment sensors 41 may be implemented to monitor the front, sides, and rear directions of the vehicle 1.

[0057] Furthermore, the external environment sensor 41 may detect atmospheric conditions and weather conditions in the environment outside the vehicle 1. The condition detection type external environment sensor 41 is, for example, an outside air temperature sensor, a temperature sensor, a raindrop sensor, or the like.

[0058] The internal environment sensor 42 may detect a specific physical quantity related to vehicle motion (hereinafter referred to as a motion physical quantity) in the internal environment of the vehicle 1. The motion physical quantity detection type internal environment sensor 42 is, for example, a speed sensor, an acceleration sensor, a gyro sensor, etc. The internal environment sensor 42 may detect the state of an occupant in the internal environment of the vehicle 1. The occupant detection type internal environment sensor 42 is, for example, an actuator sensor, a sensor and its system for monitoring a vehicle user (e.g., a driver), a biological sensor, a seating sensor, an in-vehicle equipment sensor, etc. Here, in particular, the actuator sensor is, for example, an accelerator sensor, a brake sensor, a steering sensor, etc., which detect the state of an occupant's operation of a motion actuator 60 related to the motion control of the vehicle 1.

[0059] The communication system 43 obtains communication data usable in the driving system 2 via wireless communication. The communication system 43 may receive positioning signals from artificial satellites of a global navigation satellite system (GNSS) that exist in the external environment of the vehicle 1. The positioning type communication device in the communication system 43 is, for example, a GNSS receiver.

[0060] The communication system 43 may transmit and receive communication signals to and from an external system (e.g., a server 96) present in the external environment of the vehicle 1. Examples of V2X-type communication devices in the communication system 43 include dedicated short range communications (DSRC) communication devices, cellular V2X (C-V2X) communication devices, etc. Examples of communication with a V2X system present in the external environment of the vehicle 1 include communication with a communication system of another vehicle (V2V), communication with infrastructure equipment such as a communication device installed in a traffic light or a roadside device (V2I), communication with a mobile terminal of a pedestrian (V2P), and communication with a network such as a cloud server (V2N). The architecture of V2X communication, including V2I communication, may be an architecture defined in ISO 21217, ETSI TS 102 940-943, IEEE 1609, etc.

[0061] Furthermore, the communication system 43 may transmit and receive communication signals to and from a mobile terminal 91, such as a smartphone, present inside the vehicle 1. Examples of terminal communication type communication devices in the communication system 43 include Bluetooth (registered trademark) devices, Wi-Fi (registered trademark) devices, infrared communication devices, etc. Furthermore, if the vehicle user's mobile terminal 91 is associated with the vehicle 1 in advance, the communication system 43 may transmit and receive communication signals to and from the mobile terminal present in the external environment.

[0062] The map DB 44 is a database that stores map data available to the driving system 2. The map DB 44 includes at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The map DB 44 may include a database of a navigation unit that navigates the vehicle 1 along a route to a destination. The map DB 44 may include a database of probe data (PD) maps generated using probe data (PD) collected from each vehicle. The map DB 44 may include a database of high-precision maps with a high level of accuracy that are primarily used in automated driving systems. The map DB 44 may also include a database of parking lot maps that include detailed parking lot information, such as parking space information, that is used in automated parking or parking assistance applications.

[0063] The map DB 44 suitable for the driving system 2 acquires and stores the latest map data, for example, by communicating with a map server via a V2X communication system 43. The map data is data representing the external environment of the vehicle 1, and is converted into two-dimensional or three-dimensional data. The map data may include road data representing at least one of the position coordinates, shape, road surface condition, and standard running route of a road structure. The map data may also include marking data representing at least one of the position coordinates and shape of road signs, road markings, and lane markings attached to a road. The marking data included in the map data may represent landmarks such as traffic signs, arrow markings, lane markings, stop lines, directional signs, landmark beacons, business signs, and changes in road line patterns. The map data may also include structure data representing at least one of the position coordinates and shape of buildings and traffic lights facing the road. The marking data included in the map data may represent landmarks such as street lights, road edges, reflectors, and poles.

[0064] The motion actuator 60 can control vehicle motion based on an input control signal. The drive-type motion actuator 60 is, for example, a power train including at least one of an internal combustion engine, a drive motor, etc. The braking-type motion actuator 60 is, for example, a brake actuator. The steering-type motion actuator 60 is, for example, a steering.

[0065] As shown in FIG. 3 , a plurality of HMI (Human Machine Interface) devices 70 may be mounted on the vehicle 1. The HMI devices 70 realize human-machine interaction, which is an interaction between a user of the vehicle 1 and the driving system 2. Of the plurality of HMI devices 70, a portion that realizes an operation input function by an occupant may be part of the detection unit 10. Of the plurality of HMI devices 70, a portion that realizes an information presentation function may be part of the behavior unit 30. On the other hand, the function realized by the HMI device 70 may be positioned as a function independent of the detection function, the planning function, and the behavior function.

[0066] The HMI device 70 may be an operation input device 70a that can input user operations to transmit the will or intention of the user of the vehicle 1 to the driving system 2. The operation input type HMI device 70, i.e., the operation input device 70a, is, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a turn signal lever, a mechanical switch, a touch panel of a navigation unit, etc. Of these, the accelerator pedal controls the powertrain as a motion actuator 60. The brake pedal controls a brake actuator as a motion actuator 60. The steering wheel controls a steering actuator as a motion actuator 60.

[0067] The HMI device 70 may be an information presentation device 70b that presents information such as visual information, auditory information, and cutaneous information to the user of the vehicle 1. The HMI device 70 of the visual information presentation type, i.e., the information presentation device 70b, is, for example, a meter display 70b1, a navigation unit, a CID (center information display) 70b2, a HUD (head-up display) 70b3, an illumination unit, etc.

[0068] The meter display 70b1 is a display device that is arranged, for example, in a driver-facing portion of the instrument panel that faces the driver's seat. The meter display 70b1 displays to the driver information necessary for driving, focusing on the vehicle status including the speed of the vehicle 1. The meter display 70b1 may be a graphic meter that displays all information using images, or may be a combination meter that combines an image display with an analog display using a device.

[0069] The CID 70b2 is a display device disposed, for example, in the center of the instrument panel. The CID 70b2 has the largest display screen of all the in-vehicle display devices mounted on the instrument panel. The CID 70b2 is capable of displaying images not only to the driver but also to passengers. The CID 70b2 may include a touch panel that can be operated by the vehicle user. In this case, the CID 70b2 also corresponds to the operation input device 70a.

[0070] The HUD 70b3 is a display device arranged on the instrument panel on the opposite side of the driver's seat from the meter display 70b1, i.e., on the rear side of the area facing the driver. The HUD 70b3 projects an image onto the front windshield of the vehicle 1, thereby displaying a virtual image VI that the driver can visually recognize as floating outside the vehicle.

[0071] Furthermore, instead of the CID 70b2 and the meter display 70b1, a pillar-to-pillar display arranged to cross the left and right A-pillars may be employed. Even in this case, the pillar-to-pillar display may be divided into several screens, each of which may be controlled in the same manner as the CID 70b2 and the meter display 70b1.

[0072] The auditory information presentation type HMI device 70 is, for example, a speaker, a buzzer, etc. The tactile information presentation type HMI device 70 is, for example, a steering wheel vibration unit, a driver's seat vibration unit, a steering wheel reaction force unit, an accelerator pedal reaction force unit, a brake pedal reaction force unit, an air conditioning unit, etc.

[0073] Furthermore, the HMI device 70 may realize an HMI function linked to a mobile terminal 91 such as a smartphone by mutually communicating with the terminal through the communication system 43. For example, as an alternative means of presenting information to the HMI device 70, information from the driving system 2 may be displayed on the screen of the vehicle user's smartphone through the communication system 43. On the other hand, the HMI device 70 may present information acquired from the smartphone to the user. Also, for example, an operation input to a smartphone may be an alternative means of inputting an operation to the HMI device 70.

[0074] Furthermore, the HMI device 70 may include, as the operation input device 70a, a feedback device 70a1 that receives feedback from the vehicle user. For example, the feedback device 70a1 includes a computer and a microphone. When a feedback function is selected from the CID 70b2, the feedback device 70a1 records the vehicle user's voice using the microphone for a predetermined period of time (e.g., 45 seconds). This allows the vehicle user to provide feedback such as praise or dissatisfaction about the vehicle 1 or the driving system 2. The feedback device 70a1 may then transmit the recorded vehicle user's voice to an external system in the external environment via the communication system 43. The external system may be a server 96, which will be described later. By collecting the vehicle user's feedback in the external system, it is possible to use the collected feedback to improve the vehicle 1 or the driving system 2.

[0075] At least one processing system 50 is provided. For example, the processing system 50 may be an integrated processing system that integrally executes processing related to the detection function, processing related to the planning function, and processing related to the action function. In this case, the integrated processing system 50 may further execute processing related to the HMI device 70, or a processing system dedicated to the HMI may be provided separately. For example, the processing system dedicated to the HMI may be an integrated cockpit system that integrally executes processing related to each HMI device 70. The processing system 50 may be provided by an in-vehicle platform that can be used generally for AVs.

[0076] For example, the processing system 50 may be configured to have at least one processing unit corresponding to processing related to the detection function, at least one processing unit corresponding to processing related to the planning function, and at least one processing unit corresponding to processing related to the behavioral function.

[0077] The processing system 50 has a communication interface to the outside and is connected to at least one type of element related to processing by the processing system 50, such as each sensor 40, motion actuator 60, and HMI device 70, via at least one type of interface, such as a LAN (Local Area Network), a wire harness, an internal bus, or a wireless communication circuit.

[0078] The processing system 50 includes at least one dedicated computer 51. The processing system 50 may combine multiple dedicated computers 51 to realize functions such as a sensing function, a planning function, and an action function.

[0079] For example, the dedicated computer 51 constituting the processing system 50 may be an integrated ECU that integrates the driving functions of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be a determination ECU that determines DDT. The dedicated computer 51 constituting the processing system 50 may be a monitoring ECU that monitors the driving of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an evaluation ECU that evaluates the driving of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be a navigation ECU that navigates the driving route of the vehicle 1.

[0080] Furthermore, the dedicated computer 51 constituting the processing system 50 may be a locator ECU that estimates the position of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an image processing ECU that processes image data detected by the external environment sensor 41. The dedicated computer 51 constituting the processing system 50 may be an actuator ECU that controls the motion actuator 60 of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may be an HCU (HMI Control Unit) that comprehensively controls the HMI device 70. The dedicated computer 51 constituting the processing system 50 may be at least one external computer provided in, for example, an external center or a mobile terminal 91 that can communicate via the communication system 43.

[0081] The dedicated computer 51 constituting the processing system 50 has at least one memory 51a and one processor 51b. The memory 51a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data that can be read by the processor 51b. Furthermore, the memory 51a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 51b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0082] The dedicated computer 51 constituting the processing system 50 may be a SoC (System on a Chip) in which the memory 51a, processor 51b and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer 51.

[0083] Furthermore, the processing system 50 may include at least one database for executing the DDT, which may include at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, and an interface for accessing the storage medium.

[0084] The database may be a scenario database (hereinafter referred to as a scenario DB) 59. The database may be a rule database (hereinafter referred to as a rule DB) 58. At least one of the scenario DB 59 and the rule DB 58 may not be provided in the processing system 50, but may be provided independently in the operation system 2. At least one of the scenario DB 59 and the rule DB 58 may be provided in an external system existing in the external environment, and may be configured to be accessible from the processing system 50 via the communication system 43.

[0085] The scenario DB 59 has a scenario catalog in which multiple scenarios used for driving the vehicle 1 are stored. The driving system 2 can, for example, apply a situation in which the vehicle 1 is placed to one scenario selected from the multiple scenarios or a combination of multiple scenarios. The scenario DB 59 may store multiple scenarios including at least one of a functional scenario, a logical scenario, and a concrete scenario. A functional scenario defines a top-level qualitative scenario structure. A logical scenario is a scenario in which quantitative parameter ranges are assigned to a structured functional scenario. A concrete scenario defines a safety judgment boundary that distinguishes between a safe state and an unsafe state.

[0086] The rule DB 58 stores a rule set used for driving the vehicle 1. The rule set may include multiple rules. The rule set may further include a priority structure for the rules, which is set based on the relative importance of the multiple rules. The rule set may be an implementation of guidelines for strategic driving of the vehicle 1.

[0087] The plurality of rules may include rules based on laws, regulations, or a combination thereof. The plurality of rules may include rules based on preferences that are not influenced by laws, regulations, or the like. The plurality of rules may include rules based on exercise behavior based on past experience. The plurality of rules may include rules based on characterization of the exercise environment. The plurality of rules may include rules based on ethical concerns. The plurality of rules may include rules based on basic principles of a safety model (e.g., the five principles of the RSS model). The plurality of rules may include traffic rules. The traffic rules may be rules specified in the Road Traffic Act or may be rules based on national or local customs.

[0088] The rules such as traffic rules stored in the rule DB 58 may be positioned as information provided from the detection unit 10 to the planning unit 20 by the detection function, similar to the map information acquired from the map DB 44 .

[0089] The processing system 50 may also include at least one recording device 55 that records at least one of the detection information, the plan information, and the action information of the operation system 2. The recording device 55 may include at least one large-capacity storage medium 55c. The storage medium 55c may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium.

[0090] The storage media 55c may be mounted on the board in a form that is not easily detachable or replaceable, and in this form, for example, an embedded multi-media card (eMMC) using a flash memory may be used. At least one of the storage media 55c may be detachable and replaceable from the recording device 55, and in this form, for example, an SD card may be used.

[0091] The recording device 55 may have a function of selecting information to be recorded from the detection information, the plan information, and the behavior information. In this case, the recording device 55 may have a dedicated computer.

[0092] The dedicated computer provided in the recording device 55 has at least one memory 55a and one processor 55b. The memory 55a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data that can be read by the processor 55b. Furthermore, the memory 55a may be a rewritable volatile storage medium, such as a random access memory (RAM). The processor 55b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0093] The dedicated computer may be a SoC (System on a Chip) in which the memory 55a, processor 55b, and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0094] The recording device 55 may access the storage medium 55c and perform recording in accordance with a data write command from each part of the driving system 2. The recording device 55 may determine information transmitted over the in-vehicle network, and, based on the judgment of the processor 55b provided in the recording device 55, access the storage medium 55c and perform recording.

[0095] Furthermore, the recording device 55 may not be provided in the processing system 50, but may be provided independently in the operation system 2. The recording device 55 may be provided in an external system present in the external environment, and configured to be accessible from the processing system 50 via the communication system 43.

[0096] Furthermore, the processing system 50 may include at least one risk confirmation unit 53. The risk confirmation unit 53 may be one aspect of on-board implementation of RSS (Responsibility Sensitive Safety) as a safety model. The risk confirmation unit 53 may be an on-board checker for the planning function realized by the dedicated computer 51. The risk confirmation unit 53 realizes the risk confirmation section 26, which realizes the risk confirmation function, by hardware independent of the planning section 20.

[0097] The risk confirmation unit 53 may be primarily configured as a dedicated computer having at least one memory 53a and one processor 53b. The memory 53a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data readable by the processor 55b. Furthermore, the memory 55a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 55b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0098] The dedicated computer may be a SoC (System on a Chip) in which the memory 53a, processor 53b, and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0099] As described above, the processing system 50 includes memories 51a, 53a, and 55a storing software. The processors 51b, 53b, and 55b are configured to operate the software to realize automated driving, allowing authority to be transferred between the system itself and the user. The software here may include the computer program itself used in the driving system 2. The software here may include an algorithm in the computer program used in the driving system 2. The software here may include parameters in the computer program used in the driving system 2. The software here may include AI or a trained model used in the driving system 2, for example, realized by a neural network. Furthermore, the software may include data stored in a database referenced by the processing system 50, data stored in the map DB 44, and the like. One piece of software may correspond to one application or multiple applications, may be part of one application, or may be software commonly used by multiple applications.

[0100] Furthermore, the processing system 50 may include at least one software management unit 57. The software management unit 57 implements software management functions. The software management unit 57 may also be referred to as a software management device.

[0101] The software management unit 57 manages various software used in the processing system 50, such as the computer 51, the risk confirmation unit 53, the recording device 55, the rule DB 58, and the scenario DB 59. The software management unit 57 may further manage software used in the driving system 2 outside the processing system 50. For example, the software management unit 57 may manage data stored in the map DB 44, software used for drawing processing by the information presentation device 70b, software used for communication processing by the communication system 43, etc.

[0102] Software management may include software version management, download and installation processes, uninstallation processes, etc. Software management may also include software testing.

[0103] The software management unit 57 may be configured primarily as a dedicated computer having at least one memory 57a and one processor 57b to realize the software management function. The memory 57a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data readable by the processor 57b. Furthermore, the memory 57a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 57b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0104] The dedicated computer may be a SoC (System on a Chip) in which the memory 57a, processor 57b, and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0105] The above architecture is merely an example, and various configurations can be adopted as the hardware configuration of the driving system 2.

[0106] <Logical Architecture in Autonomous Driving> Next, an example of a logical architecture in the driving system 2 will be described using Figure 2. The description here will focus on processing by a computer program executed during autonomous driving at level 3 or higher. The detection unit 10 may include an environment recognition unit 11, a self-location recognition unit 12, and an internal recognition unit 13 as processing units for realizing sub-functions obtained by further classifying the detection function by the processor 51b executing a computer program.

[0107] The environment recognition unit 11 individually processes information (sometimes referred to as sensor data) related to the external environment acquired from each sensor 40, and realizes a function of recognizing the external environment including targets, other road users, etc. The environment recognition unit 11 individually processes detection data detected by each external environment sensor 41. The detection data may be detection data provided by, for example, millimeter-wave radar, sonar, LiDAR, etc. The environment recognition unit 11 may generate relative position data including the direction, size, and distance of an object relative to the vehicle 1 from the raw data detected by the external environment sensors 41.

[0108] The detection data may be image data provided by, for example, a camera, LiDAR, or the like. The environment recognition unit 11 processes the image data and extracts objects reflected within the angle of view of the image. The object extraction may include estimating the direction, size, and distance of the object relative to the vehicle 1. The object extraction may also include classifying the object using, for example, semantic segmentation.

[0109] Furthermore, the environment recognition unit 11 processes information acquired through the V2X function of the communication system 43. The environment recognition unit 11 processes information acquired from the map DB 44.

[0110] The environment recognition unit 11 may be further divided into a plurality of sensor recognition units each optimized for a single sensor group. When a sensor recognition unit is associated with recognizing information from a single sensor group, the sensor recognition unit may fuse information from the single sensor group.

[0111] The self-location recognition unit 12 performs localization of the vehicle 1. The self-location recognition unit 12 acquires global position data of the vehicle 1 from the communication system 43 (e.g., a GNSS receiver). In addition, the self-location recognition unit 12 may acquire position information of targets extracted by the environment recognition unit 11. The self-location recognition unit 12 also acquires map information from the map DB 44. The self-location recognition unit 12 integrates this information to estimate the position of the vehicle 1 on the map.

[0112] The internal recognition unit 13 processes the detection data detected by each internal environment sensor 42 and realizes a function of recognizing the vehicle state. The vehicle state may include the state of the physical quantities of motion of the vehicle 1 detected by a speed sensor, an acceleration sensor, a gyro sensor, etc. The vehicle state may also include at least one of the user state, the user's operation state of the motion actuator 60, and the switch state of the HMI device 70.

[0113] The planning unit 20 may include a prediction unit 21, an operation planning unit 22, and a mode management unit 23 as processing units for realizing sub-functions that are further classified into planning functions by having processors 51b, 53b execute computer programs.

[0114] The prediction unit 21 acquires information on the external environment recognized by the environment recognition unit 11 and the self-position recognition unit 12, the vehicle state recognized by the internal recognition unit 13, etc. The prediction unit 21 may interpret the environment based on the acquired information and estimate the current situation of the vehicle 1. The situation here may be an operational situation or may include the operational situation.

[0115] The prediction unit 21 may interpret the environment and predict the behavior of objects such as other road users. The objects may be safety-relevant objects. The behavior prediction may include at least one of predicting the object's speed, acceleration, and trajectory. The behavior prediction may be performed based on reasonably foreseeable assumptions. Furthermore, the prediction unit 21 may estimate the user's intention based on the predicted behavior, predicted potential hazards, and the acquired vehicle state.

[0116] The driving planning unit 22 plans autonomous driving of the vehicle 1 based on at least one of the estimated information of the vehicle 1's position on a map by the self-position recognition unit 12, the prediction information and user intention estimation information by the prediction unit 21, and the function constraint information by the mode management unit 23.

[0117] The driving planner 22 realizes a route planning function, a behavior planning function, and a trajectory planning function. The route planning function is a function of planning at least one of a route to a destination and a mid-range lane plan based on estimated information about the position of the vehicle 1 on a map. The route planning function may further include a function of determining at least one of a lane change request and a deceleration request based on the mid-range lane plan. Here, the route planning function may be a mission / route planning function in a strategic function, and may be a function of outputting a mission plan and a route plan.

[0118] The behavior planning function is a function that plans the behavior of the vehicle 1 based on at least one of the route to the destination planned by the route planning function, a mid-distance lane plan, a lane change request and a deceleration request, prediction information and user intention estimation information by the prediction unit 21, and function constraint information by the mode management unit 23. The behavior planning function may include a function that generates conditions related to state transitions of the vehicle 1. The conditions related to state transitions of the vehicle 1 may correspond to triggering conditions. The conditions related to state transitions may include fallback conditions for executing DDT fallbacks.

[0119] The behavior planning function may include a function for determining state transitions of an application that realizes the DDT and further state transitions of driving actions based on the conditions. As a result, the driving planner 22 plans the execution of a DDT fallback. If this does not involve authority delegation, the driving planner 22 may further execute a Minimal Risk Maneuver (MRM) together with the motion control unit 31 to transition the vehicle 1 to a minimal risk state. The MRM plan may be realized by the behavior planning function or the trajectory planning function.

[0120] The behavior planning function may also include a function for determining, based on the information on these state transitions, longitudinal constraints on the path of the vehicle 1 and lateral constraints on the path of the vehicle 1. The behavior planning function may be a tactical behavior plan in the DDT function, and may output tactical behavior.

[0121] The trajectory planning function is a function that plans a driving trajectory of the vehicle 1 based on the judgment information by the prediction unit 21, longitudinal constraints on the path of the vehicle 1, and lateral constraints on the path of the vehicle 1. The trajectory planning function may include a function that generates a path plan. The path plan may include a speed plan, or the speed plan may be generated as a plan independent of the path plan. The trajectory planning function may include a function that generates multiple path plans and selects an optimal path plan from the multiple path plans, or a function that switches between path plans. The trajectory planning function may further include a function that generates backup data of the generated path plan. The trajectory planning function may be a trajectory planning function in the DDT function, and may output a trajectory plan.

[0122] The mode management unit 23 monitors the driving system 2 and sets constraints on driving-related functions. The mode management unit 23 may manage the autonomous driving mode, for example, the state of the automation level. The management of the automation level may include management of switching between manual driving and autonomous driving, i.e., management of the transfer of authority between the user and the driving system 2, in other words, management of the takeover of driving. The mode management unit 23 may monitor the state of the subsystem related to the driving system 2 and determine a system malfunction (e.g., an error, an unstable operation state, a system failure, or a malfunction). The mode management unit 23 may determine a mode based on the user's intention based on the user's intention estimation information generated by the internal recognition unit 13. The mode management unit 23 may set constraints on driving-related functions based on at least one of the system malfunction determination result, the mode determination result, the vehicle state determined by the internal recognition unit 13, the sensor abnormality (or sensor failure) signal output from the sensor 40, the application state transition information and the trajectory plan determined by the driving planner 22, etc.

[0123] Furthermore, the mode management unit 23 may have a comprehensive function of determining, in addition to constraints on driving functions, longitudinal constraints on the path of the vehicle 1 and lateral constraints on the path of the vehicle 1. In this case, the operation planning unit 22 plans behavior and trajectories in accordance with the constraints determined by the mode management unit 23.

[0124] In addition, when the risk confirmation function is implemented as part of the planning unit 20, the risk confirmation function may be implemented as part of the functions realized by the prediction unit 21, the operation planning unit 22, and the mode management unit 23.

[0125] The risk confirmation function is a function that acquires an environmental model, sensor data, etc. from the detection unit 10, evaluates a risk based on this information, and outputs a response based on the risk to the behavior unit 30 or the operation planning unit 22. This series of functions or processes may be referred to as risk confirmation or risk monitoring.

[0126] More specifically, the risk confirmation function outputs a situation based on information acquired from the detection unit 10. The risk confirmation function confirms whether the situation is safe or dangerous. This confirmation may include confirmation of an estimated result of a collision risk between the vehicle 1 and a surrounding object. In this confirmation, an index such as a collision probability may be used to take uncertainty into account. The risk confirmation function may compare an acceptable collision risk threshold with the estimated collision risk value to determine whether the situation is dangerous. The acceptable collision risk threshold may be set in advance based on risk acceptance criteria / criterion, which will be described in detail later.

[0127] The risk confirmation function derives a proper response based on the result of this confirmation. The proper response may be provided to the behavior unit 30 or the driving plan unit 22 only if the situation is determined to be a dangerous situation. The proper response may be a restriction on the control command of the motion actuator 60. The proper response may be a response to return the vehicle 1 to a safe state.

[0128] The risk confirmation function is realized by implementing a safety model. The safety model may be referred to as a safety-related model. The safety model may be a formal model. For example, an RSS model may be adopted as the safety model, but other models such as an SFF model, a more generalized model, or a composite model combining multiple models may also be adopted. SFF stands for Safety Force Field. The safety model monitors the risk of the vehicle 1 based on a driving policy. In other words, risk monitoring can also be said to be monitoring the driving policy.

[0129] In the RSS model, for example, the longitudinal and lateral safety distances from other road users are used as indicators for determining the collision risk. The safety distances are an example of a geometric approach such as a safety envelope.

[0130] The behavior unit 30 may include a motion control unit 31 and an HMI output unit 71 as processing units for realizing sub-functions obtained by further classifying the behavior functions by the processor 51b executing a computer program. The motion control unit 31 controls the motion of the vehicle 1 based on the trajectory plan (e.g., a path plan and a speed plan) acquired from the driving plan unit 22. Specifically, the motion control unit 31 generates accelerator request information, shift request information, brake request information, and steering request information according to the trajectory plan, and outputs them to the motion actuator 60.

[0131] Here, the motion control unit 31 can directly obtain the vehicle state recognized by the detection unit 10 (particularly the internal recognition unit 13), such as at least one of the current speed, acceleration, and yaw rate of the vehicle 1, from the detection unit 10 and reflect this in the motion control of the vehicle 1.

[0132] The HMI output unit 71 outputs information related to the HMI based on at least one of prediction information and user intention estimation information from the prediction unit 21, application state transition information and trajectory planning from the operation planning unit 22, and function constraint information from the mode management unit 23. The HMI output unit 71 may manage vehicle interactions. The HMI output unit 71 may generate a notification request based on the management state of the vehicle interactions and control an information presentation function of the HMI device 70. Furthermore, the HMI output unit 71 may generate control requests for wipers, a sensor washing device, headlights, and an air conditioning device based on the management state of the vehicle interactions and control these devices.

[0133] <Testing in Public Road Traffic> A verification and validation (V&V) process is required for the driving system 2 described above. The V&V here may be V&V of the intended functions of the software used in the driving system 2 or V&V of SOTIF. Scenarios that the vehicle 1 may encounter can be classified into known dangerous scenarios, known non-hazardous scenarios, unknown dangerous scenarios, and unknown non-hazardous scenarios. The V&V process may be a process for reducing the risks of known dangerous scenarios and unknown dangerous scenarios among these scenarios.

[0134] The V&V of the operation system 2 includes verification to satisfy the safety requirements at each technology level and verification to safely integrate each element. The verification to satisfy the safety requirements at each technology level may include evaluation of at least one, and preferably all, of the following functions and capabilities. The verification may also include evaluation of other functions and capabilities.

[0135] For example, the detection unit 10 may evaluate the functionality of the sensors 40 or external data sources (eg, map data sources), the functionality of the sensor algorithms that model the environment, and the reliability of the infrastructure and communication systems 43 .

[0136] For example, an evaluation target related to the planning unit 20 is the capability of a decision algorithm. The capability of the decision algorithm is the ability to safely handle potential functional deficiencies, and the ability to make appropriate decisions according to an environmental model, a driving policy, a current destination, etc. Also, for example, an evaluation target related to the planning unit 20 may include one of the following: absence of unreasonable risks due to dangerous behavior of the intended functions, the capability of the system to safely handle use cases of the ODD, robust performance of the execution of the driving policy across the ODD, suitability of the DDT fallback, and suitability of the minimum risk state.

[0137] Furthermore, for example, the evaluation target may include not only the nominal performance of the system or function but also its robust performance, such as the system's robust performance against adverse environmental conditions affected by various disturbances, the appropriateness of system operation against known trigger conditions, the sensitivity of the intended function, and the monitoring capability against various scenarios.

[0138] V&V may be performed with the goal that the automated driving performed by the driving system 2 achieves a positive risk balance. A positive risk balance can be said to be a primary measure of an ethically acceptable risk level.

[0139] More specifically, V&V may be performed with the goal of achieving a risk tolerance criterion that may be set based on a positive risk balance. A quantitative criterion for the risk tolerance criterion may be, for example, that the probability of occurrence of harm is below a threshold. The risk tolerance criterion may be set by combining a statistical approach, such as traffic statistics, with a scenario-based approach.

[0140] When verifying or testing software related to at least one of behavior planning and trajectory planning by the driving planning unit 22, it is necessary to confirm that the driving system 2 behaves at least safer than a competent and careful driver or an experienced and attentive driver. When testing software in a virtual environment, a verification method based on software-in-the-loop (SiL) may be performed using, for example, a reference data set including scenarios stored in a scenario DB.

[0141] When verifying or testing software related to mode management by the mode management unit 23, particularly verification or testing related to ODD determination and management of operating and non-operating states, may be performed in SiL, hardware-in-the-loop (HiL), or both.

[0142] Furthermore, when verifying or testing software related to the HMI using the HMI output unit 71 or the like, the verification or testing may be performed using HiL and driver-in-the-loop (DiL). DiL testing may be performed on vehicle users who have no prior experience or knowledge of the driving system 2 and are unfamiliar with automated driving.

[0143] In this way, verification methods such as SiL, HiL, and DiL may be selected depending on the target and purpose in verifying or testing software in the driving system 2. The loops used in SiL, HiL, and DiL may be open loops or closed loops.

[0144] In order to manage unacceptable risks and improve the driving system 2, it is preferable to have a robust management system MS for the driving system 2 of vehicles 1 participating in public road traffic after they are shipped to the market. For example, the management system MS shown in FIG. 4 performs software changes to a vehicle population over the air (OTA). The management system MS includes a vehicle population including multiple vehicles 1A, 1B, ... and a server 96. The vehicles 1A and 1B managed by the management system MS may have the same configuration as the vehicle 1 equipped with the driving system 2 described above. However, as long as there is at least compatibility in the software specifications, some of the hardware specifications, such as vehicle type and model, may differ from each other.

[0145] The testing in the change process may use data collected while each vehicle 1A, 1B belonging to the vehicle population is traveling on public roads. Furthermore, safety-related indicators may be used in the testing in the change process. The results of validation using safety-related indicators do not have to be used solely to determine whether or not the test software is applicable. For example, the results may be used to set the ODD itself and to set parameter limits by the ODD. The setting here may include specification changes due to software updates.

[0146] In the following, we may distinguish between software used for temporary testing as test software and software used officially (permanently) as official software.

[0147] Furthermore, the test by the management system MS is not limited to software updates, and may be conducted to improve the environment in which the driving system 2 is used. That is, the test results may be fed back to urban planning. For example, the road shapes at intersections, merging points, etc. may be changed to shapes that provide greater safety as a result of the test. Also, for example, the control of traffic lights at intersections, etc. may be changed to control that reduces the frequency of traffic congestion as a result of the test. In this case, the test may not involve the application of test software.

[0148] The safety index may be a value that indexes the risk confirmed by the risk confirmation unit 26. Examples of risk indexes include the above-mentioned risk value, safety envelope, safety distance, violation metric (degree of violation), etc. The violation metric (degree of violation) is a value that evaluates the degree of violation of the rules stored in the rule set by the vehicle 1 (test vehicle).

[0149] The safety indicator may be a safety metric of the automated driving system. The safety metric may refer to a quantifiable measure based on collision rates. Examples of safety metrics include collision severity and frequency, citable offense severity and frequency, longitudinal and lateral distance, longitudinal and lateral acceleration, longitudinal and lateral jerk, and OEDR reaction time. The collision severity may be rated on a six-point scale using, for example, the Abbreviated Injury Scale (AIS). The longitudinal and lateral distances are indicators of maintaining a safety envelope or safety distance.

[0150] Furthermore, the evaluation of the safety index or the like may be a human evaluation. The human here may be the vehicle user of the vehicle being tested. The human here may be a vehicle user who is unfamiliar with automated driving. In a DiL test, the evaluation index may be an evaluation input by the driver, who is the vehicle user, through the operation input device 70a (e.g., the feedback device 70a1) of the vehicle being tested, or may be an evaluation input through the mobile terminal 91 owned by the driver. The human here may be another VRU, such as a pedestrian who encounters the vehicle being tested. In this case, the evaluation may be an evaluation transmitted by another VRU who senses danger from the vehicle being tested to the driving systems 2A, 2B of the vehicles 1A, 1B or the server 96 using their own mobile terminal 91 or the like.

[0151] The management system MS may use the vehicle population to test software that may be used in the driving systems 2A, 2B. The management system MS may apply the software that has been validated through testing to each driving system 2A, 2B in the vehicle population. In this way, the management system MS functions as a test system that tests the validity of the driving systems 2A, 2B and the environment in which the driving systems 2A, 2B are used.

[0152] As shown in FIG. 5, the management system MS may be configured with a test execution unit MS1, a test stop unit MS2, and a software management unit MS3 as processing units realized through cooperation between the server 96 and the operation systems 2A and 2B.

[0153] The test execution unit MS1 sets a test plan in the server 96 and executes tests using the driving systems 2A and 2B of some or all of the vehicles 1A and 1B in the vehicle population. Processing related to the overall test plan may be executed mainly by the test management unit 97a of the server 96. Processing related to the execution of the test may be executed mainly by the test management unit 97a, software distribution unit 97b, data collection unit 97c, and test software evaluation unit 97d of the server 96, as well as the participation status management unit 81, software application unit 82, and operation measurement unit 83 of the driving systems 2A and 2B.

[0154] The test stop unit MS2 has both an individual stop function SFi that stops the test individually on a participation basis and an overall stop function SFw that stops the test as a whole during the test preparation state or implementation state, and executes processing related to the individual stop function SFi and the overall stop function SFw. The participation unit in the test may be, for example, a single vehicle. The term "stop" here refers to the stop of the test implementation state, and is a concept that encompasses both temporary and permanent stop. A positional stop may be referred to as an interruption, and a permanent stop may be referred to as a cancellation. The processing related to the individual stop function SFi may be executed mainly by, for example, the participation status management unit 81 of the driving systems 2A and 2B, and the test management unit 97a of the server 96 may also be involved. The processing related to the overall stop function SFw may be executed mainly by, for example, the test management unit 97a of the server 96.

[0155] The software management unit MS3 distributes test software to the test vehicle in accordance with the test and manages the operation of the software. The software management unit MS3 rewrites software, switches software, etc. based on the enabled states of the individual stop function SFi and the global stop function SFw. The software management process may be mainly performed by, for example, the software distribution unit 97b of the server 96 and the software application unit 82 of the driving systems 2A and 2B.

[0156] 1 and 4, the server 96 is installed in an external environment relative to the vehicles 1A and 1B. The server 96 is communicatively connected to each of the vehicles 1A and 1B, for example, by V2X communication via a communication infrastructure. The server 96 may be connected to an operation terminal operated by a human operator, for example, and may form a remote management center, together with the operation terminal, that manages the vehicle population.

[0157] As shown in FIG. 1, the server 96 may be primarily configured as a dedicated computer having at least one memory 96a and one processor 96b. The memory 96a may be at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer programs and data that can be read by the processor 96b. Furthermore, the memory 96a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 96b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0158] The dedicated computer may be a SoC (System on a Chip) in which the memory 96a, processor 96b, and interface are integrated into a single chip, or may have at least one SoC as a component of the dedicated computer.

[0159] The server 96 may further have a management database (hereinafter referred to as management DB) 96c. The management DB 96c may store information for identifying the vehicles 1A, 1B that are to be managed by the management system MS. The management DB 96c may store information related to the specifications of each vehicle 1A, 1B. The management DB 96c may store various information collected from each vehicle 1A, 1B. The various information may include information for performing evaluation in the test described below. The various information may also include a status indicating whether the test participating vehicle is undergoing testing or is stopped.

[0160] The server 96 implements a software improvement function based on a V&V process suitable for an operating system. As shown in Fig. 4, the server 96 may include a test management unit 97a, a software distribution unit 97b, a data collection unit 97c, and a test software evaluation unit 97d as processing units for realizing functions by a processor 96b executing a computer program.

[0161] The test management unit 97a manages tests using a vehicle population. A single type of test software may be prepared as an improved version of the software currently being used by the vehicle population (hereinafter referred to as conventional software), and tests may be conducted using this software. When two types of test software are prepared and compared, this test is called an AB test. The multiple test software may be three or more types of software that have similar functions and can be compared with each other.

[0162] The following description focuses on a typical example of A / B testing, in which one of multiple test software programs is assigned to one test vehicle. However, various testing methods can be adopted, and for example, it is also possible to perform A / B testing by assigning multiple test software programs to one test vehicle.

[0163] The test software is provided, for example, by a human administrator of the server 96 (hereinafter referred to as the test administrator). In this case, the test is conducted for the purpose of selecting the most suitable software from among the release candidate software. On the other hand, if the test is conducted to optimize parameters used in a computer program, the test software may be software automatically generated by the test management unit 97a. For example, the test software may be automatically generated in a form that changes parameters such as the judgment threshold, upper limit, and lower limit of an existing program.

[0164] The test management unit 97a manages the scale and duration of the test. The scale and duration may be set to values ​​entered into the server 96 by the test administrator, or may be automatically set by the test management unit 97a. The test management unit 97a determines test target vehicles suitable for conducting the test from among the vehicles 1A, 1B belonging to the vehicle population based on the scale and duration. If the number of vehicles incorporated into the management system MS is smaller than the optimal scale for conducting the test, all of the vehicles 1A, 1B belonging to the vehicle population may be designated as test target vehicles.

[0165] The test management unit 97a assigns one of multiple test software programs to each of the vehicles 1A, 1B belonging to the vehicle population as a test target vehicle. In an A / B test comparing test software A and test software B, for example, half of the test target vehicles test test software A, and the other half test test software B. The test management unit 97a may randomly assign test software programs to each of the vehicles 1A, 1B using pseudo-random numbers. The test management unit 97a may refer to the vehicle information stored in the management DB 96c and assign test software programs in a way that reduces bias in the conditions between the groups testing each test software program.

[0166] The test management unit 97a may impose restrictions on the content of the test. For example, the test management unit 97a may limit the area in which the test is conducted to, for example, a specific country or region. The test management unit 97a may limit the time period in which the test is conducted to, for example, only daytime hours.

[0167] Furthermore, the test management unit 97a sets at least one evaluation index for evaluating the test software. The evaluation index may be set to an index determined by an input operation by the test administrator to the server 96 based on the functions and characteristics of the test software or the test intention. Alternatively, the evaluation index may be set by the test management unit 97a based on the functions and characteristics of the test software. The evaluation index may include an index related to safety.

[0168] The test management unit 97a may also manage at least one of a method for obtaining consent from the vehicle user regarding the application of the test software to each test target vehicle and the content of the consent. The test management unit 97a may leave the management of at least one of a method for obtaining consent and the content of the consent to the driving systems 2A and 2B of each test target vehicle.

[0169] The consent may include information regarding whether or not a participant can withdraw from the test during the test (i.e., a cancellation policy). For example, if the test content or its evaluation index requires continuity of the test, the consent may include prohibiting withdrawal from the test due to the vehicle user for the entire test period or a certain period of the test period. Examples of tests that require continuity of the test include a test to verify the time it takes for a driver to become accustomed to a new user interface, a test to verify the time it takes for a passenger to become accustomed to new vehicle controls, etc.

[0170] If the vehicle user can select whether or not to prohibit the vehicle user from withdrawing from the test for a period of time, an incentive may be provided to the vehicle user who agrees to the agreement including the withdrawal prohibition. The incentive may be, for example, a function upgrade of the vehicle 1, the issuance of a discount coupon for an optional function of the vehicle 1, or the awarding of electronic money or points. For example, a function upgrade of the vehicle 1 may be to unlock a seat heater function that is installed in the vehicle 1 but whose function has been frozen, during the test period. Then, at the end of the test, the vehicle user can use the discount coupon to purchase the seat heater function at a discounted price and make it available for permanent use.

[0171] The test management unit 97a determines whether the test should be started as a whole in the test preparation state. The suspension of the start of the test as a whole can be considered to be the activation of the overall stop function SFw. Furthermore, the test management unit 97a determines whether the test should be stopped as a whole in the test implementation state. The stopping of the test as a whole can be considered to be the activation of the overall stop function SFw. When the overall stop function SFw is activated, the test management unit 97a may determine whether the stop is permanent or temporary. When the overall stop function SFw is activated and the activated stop is temporary, the test management unit 97a may disable the overall stop function SFw and determine whether to resume the implementation of the test.

[0172] The test management unit 97a may manage the individual stop functions SFi in addition to managing the overall stop function SFw. The test management unit 97a may acquire information regarding the activation and deactivation of the individual stop functions SFi from the driving systems 2A and 2B of each test target vehicle and aggregate the information. The test management unit 97a may determine whether to individually stop the test of each test target vehicle and request control based on this determination from the driving systems 2A and 2B.

[0173] The software distribution unit 97b distributes the test software to the driving systems 2 of the vehicle to be tested based on the test software allocation set by the test management unit 97a. Information about the test plan may be distributed together with the distribution of the test software. The information about the test plan includes information about the test period and evaluation indexes, and may also include information about the scale of the test. As a result, the distributed test software is temporarily applied to each driving system 2A, 2B, and the test begins.

[0174] The data collection unit 97c collects, as probe data, information about the operation or operation results of the test software from each driving system 2A, 2B to which the test software is temporarily applied. The collected information may include at least one of the safety indicators themselves and information for deriving the safety indicators. The data collection unit 97c stores the data sequentially collected from each vehicle 1A, 1B in the management DB 96c.

[0175] The test software evaluation unit 97d compares multiple test software programs. The test software evaluation unit 97d compares the evaluation indexes set by the test management unit 97a between the test software programs. If multiple evaluation indexes exist, the test software evaluation unit 97d compares each evaluation index between the test software programs.

[0176] Specifically, the test software evaluation unit 97d statistically processes the data from each vehicle 1A, 1B stored in the management DB 96c. For example, if the evaluation index can be expressed as a rate such as an occurrence rate, the test software evaluation unit 97d calculates the occurrence rate per vehicle and / or per unit time from the occurrence frequency in the data from each vehicle 1A, 1B.

[0177] Furthermore, in evaluations using safety metrics as evaluation indices, it is preferable to consider severity potential. When evaluating the severity and frequency of collisions for test software, even if the test software is temporarily applied to multiple vehicles 1A and 1B, if the actual number of collisions is small, it is difficult to statistically evaluate. For this reason, the test software evaluation unit 97d may estimate the rate of serious collisions with high severity from the occurrence rates of appropriate precursor events, such as near collisions and low-severity collisions.

[0178] Furthermore, since the distribution and sensitivity of crash types may differ between an automated driving system and a human driver, the statistical evaluation of the safety metrics for the test software may be performed by classifying the vehicle 1 to which the test software is applied into an automated driving vehicle of level 3 or higher and a vehicle 1 driven by a human user.

[0179] Furthermore, the test software evaluation unit 97d may have a function of selecting one optimal test software from multiple test software. The optimal test software may be the test software with the highest safety. The test management unit 97a may decide that the test software selected by the test software evaluation unit 97d is the official software to be officially adopted.

[0180] If the multiple test software programs are improved versions of conventional software currently officially applied to each vehicle 1A, 1B, the test software evaluation unit 97d may perform a relative evaluation of the selected test software against the conventional software. The test software evaluation unit 97d may determine that the selected test software does not have higher performance than the conventional software. In this case, the test management unit 97a may exclude any of the multiple test software programs from the official software to be officially adopted.

[0181] On the other hand, the test software evaluation unit 97d does not need to have a function for selecting optimal test software. In this case, the test management unit 97a presents the comparison results of the evaluation indexes to the test manager via the HMI of the server 96 and accepts the test manager's input operation of the selection results of the official software to be officially adopted. Based on this input operation, the test management unit 97a determines the official software.

[0182] Based on the determination of the official software, the software distribution unit 97b may distribute the official software to each vehicle 1A, 1B belonging to the vehicle population. The distribution destination vehicles may include vehicles belonging to the vehicle population that were not selected as test vehicles. Furthermore, the software distribution unit 97b may request other management systems to adopt the selected software or may recommend the selected software to other management systems.

[0183] <Configuration example of driving system in management system> Each vehicle 1A, 1B belonging to a vehicle population is equipped with an individual driving system 2A, 2B. The driving systems 2A, 2B are equipped with a software management function in addition to a recognition function, a planning function, and an action function. The driving systems 2A, 2B may each include a participation status management unit 81, a software application unit 82, a behavior measurement unit 83, and an HMI linkage unit 84 as processing units for realizing functions by the processor 57b of the software management unit 57 executing a computer program.

[0184] The participation status management unit 81 manages the test participation status of the vehicles 1A, 1B equipped with the respective driving systems 2A, 2B. The test participation status may be either a state in which the vehicle is participating in the test or a state in which the vehicle is not participating in the test. Furthermore, the test participation status may be classified into a state in which the test is being conducted, a state in which the overall test stop function SFw is enabled, and a state in which the individual test stop function SFi is enabled.

[0185] If the participation status management unit 81 is unable to obtain consent to participate in the test (hereinafter referred to as test consent) from the vehicle user, the participation status management unit 81 sets the test participation status to a state in which the vehicle is not participating in the test. Under such a setting condition, the participation status management unit 81 prohibits the vehicle 1A, 1B equipped with the participation status management unit 81 from participating in the test.

[0186] If the participation status management unit 81 has obtained test consent from the vehicle user, the participation status management unit 81 sets the test participation status to a state in which the vehicle is participating in the test. The participation status management unit 81 then determines whether the overall stop function SFw is enabled by obtaining information about the overall stop function SFw of the test from the server 96 via the communication system 43. The participation status management unit 81 also controls the enabling and disabling of the individual stop functions SFi of the vehicles 1A, 1B in which the participation status management unit 81 is installed, and reflects this in the test participation status.

[0187] The software application unit 82 manages the application of software implemented in the driving systems 2A and 2B. The application of software here may include downloading and installing the software, i.e., preparation for making the software available for use. The management of software application may include defense against external attacks that exploit security vulnerabilities, management of privacy in the software, and management of software updates. The management of updates may include software version management.

[0188] When update information is obtained from the server 96 and official software is distributed from the server 96, the software application unit 82 permanently applies the official software to the operation systems 2A, 2B. "Permanent application" here means application until the next update of the official software.

[0189] Furthermore, managing the application of software may include managing the application of test software. When the vehicles 1A and 1B are selected as test target vehicles, the software application unit 82 temporarily applies the test software downloaded from the server 96 to the driving systems 2A and 2B, and starts verification of the test software.

[0190] Below, two examples of methods for applying test software when testing test software that is an improved version of conventional software already applied to the hardware of the operation systems 2A and 2B are shown. Here, the hardware may be, for example, the memory 51a of the computer 51, the memory 53a of the risk confirmation unit 532, etc. When performing tests related to rule sets, scenario structures, etc., the hardware may be the rule DB 58, the scenario DB 59, etc.

[0191] The first method involves rewriting the existing software with test software as the test is carried out. This method is suitable when the operating systems 2A and 2B do not have sufficient hardware resources. This method also allows the test software to be tested under conditions that are almost identical to those during actual operation.

[0192] In the first method, when the test software is applied and at least one of the overall stop function SFw and the individual stop function SFi is enabled, the software application unit 82 downloads the conventional software from the server 96 and writes the test software back to the conventional software.

[0193] The second method is to leave the conventional software and download new test software so that they coexist. In this case, the hardware that stores the test software may be the same hardware as the conventional software, or may be different hardware. The different hardware may be the memory 57a of the software management unit 57, or may be hardware dedicated to testing (e.g., hardware dedicated to shadow mode). This method is suitable when the operating systems 2A and 2B have relatively ample hardware resources.

[0194] In the second method, when the conventional software and the test software coexist, it is preferable to configure the system so that one software is enabled and the other software is disabled by a software switching function. For example, when both the overall stop function SFw and the individual stop function SFi are disabled, the software application unit 82 disables the conventional software and enables the test software. When at least one of the overall stop function SFw and the individual stop function SFi is enabled, the software application unit 82 enables the conventional software and disables the test software. The software switching function can be easily realized by incorporating a conditional branch instruction, such as an IF statement, into a computer program that controls the software.

[0195] Here, the driving systems 2A and 2B may not execute the calculations of the disabled software at all, or may execute the calculations of the disabled software but not reflect the calculation results in other functions that affect the behavior of the vehicles 1A and 1B, such as the DDT.

[0196] Testing using the shadow mode may be performed while the conventional software is enabled and the test software is disabled. In this case, the calculation results of the disabled test software may be stored exclusively in the storage medium 55c of the recording device 55 or the management DB 96c for verifying the test software, without being reflected in other functions that affect the behavior of the vehicles 1A, 1B, such as the DDT. The following description will be continued assuming that the second method is adopted.

[0197] The operation measurement unit 83 measures the operation results of the temporarily applied test software. The measurement target is at least one of the evaluation index specified by the server 96 and the data required to calculate the evaluation index. When a calculation is performed in the disabled test software as described above, the measurement target may include the calculation results of the disabled test software. Measurement of the operation results may be performed continuously depending on the characteristics of the evaluation index, or may be performed only on predetermined occasions based on a preset trigger.

[0198] Furthermore, the operation measurement unit 83 transmits the measured test results to the server 96. The test results may be transmitted sequentially, including the progress of the test, or may be transmitted all at once at the end of the test period. The test results include at least one of the participation status set by the participation status management unit 81, test software application information to the operation systems 2A and 2B by the software application unit 82, and measurement information measured by the operation measurement unit 83. The test software application information may include the date and time and period when the test software was applied to the operation systems 2A and 2B, the method of applying the test software to the operation systems 2A and 2B, etc. The measurement information is information related to the measurement target described above.

[0199] The operation measurement unit 83 may also record the test results in the storage medium 55c. A transmission log to the server 96 may also be recorded in the storage medium 55c together with the test results.

[0200] In this way, the operation measurement unit 83 selects the indices or data to be measured and the source from which to obtain them, i.e., the measurement target, based on the evaluation index information received from the server 96. The operation measurement unit 83 then generates the measured indices or data in the form of measurement information according to a preset format. This format may be specified by the server 96 in the evaluation index information. The generated measurement information may be transmitted to the server 96 as probe data, as described above, and may also be recorded in the storage medium 55c.

[0201] The HMI linking unit 84 links with the HMI device 70 to realize the functions of obtaining the above-mentioned consent, presenting information related to software management, and receiving feedback from the vehicle user.

[0202] The HMI linking unit 84 may attempt to obtain consent to the test from the vehicle user (e.g., the driver) in response to a request from the participation status management unit 81. Specifically, the HMI linking unit 84 uses the information presentation device 70b to issue a notification to the vehicle user (e.g., the driver) requesting consent to the test. The HMI linking unit 84 accepts an operation input from the driver to the operation input device 70a in response to the notification and confirms the driver's willingness to consent. The HMI linking unit 84 provides information on the consent obtained from the driver to the software application unit 82. The HMI linking unit 84 also issues notifications to the vehicle user as necessary during the test, such as when the test starts, is in progress, and is completed. These notifications can be easily recognized by the vehicle user if they are displayed using, for example, the CID 70b2 of the HMI device 70.

[0203] The HMI linking unit 84 also receives feedback from the vehicle user during the test and at the end of the test period. The feedback may be received by an input operation to the CID 70b2 or by the vehicle user's voice input to the feedback device 70a1.

[0204] In the following, several examples of using the overall shutdown function SFw and the individual shutdown function SFi by the management system MS will be described.

[0205] <Example of Response to Vehicle User's Operation Error> As shown in the flowchart of Figure 6, the management system MS uses the individual stop function SFi of the test stop unit MS2 to appropriately respond to an operation error by a vehicle user who expresses their intention to participate in a test. The operation error here refers to an operation by a vehicle user who erroneously agrees to the test. The operation of a vehicle user who agrees to the test is an example of an operation expressing their intention to participate in a test.

[0206] 6, for example, the processor 96b of the server 96 executes a computer program stored in the memory 96a. Correspondingly, in the driving systems 2A and 2B, the processor 53b of the software management unit 57 executes a computer program stored in the memory 53a. A trigger for starting this process may be provided by an input operation to a user interface of the server 96 by the test administrator.

[0207] In the first step S101, the server 96 (e.g., the test management unit 97a) sets the preparation status of the planned test. Based on this, the server 96 and each of the operation systems 2A and 2B start processing to execute the test. After processing S101, the process proceeds to S102. Hereinafter, a specific operation system 2 will be described.

[0208] In S102, the driving system 2 (e.g., the participation status management unit 81 and the HMI linkage unit 84) attempts to obtain the vehicle user's consent to the test and determines whether or not the consent to the test has been obtained. If the answer is Yes, proceed to S103. If the answer is No, proceed to S103.

[0209] In S103, the driving system 2 (for example, the participation status management unit 81) sets the participation status of the vehicle 1 to "participating in the test." After the processing of S103, the process proceeds to S104.

[0210] In S104, the driving system 2 (e.g., the participation status management unit 81) enables the individual stop function SFi of the corresponding vehicle 1. This prevents a test from starting in this vehicle 1 until the individual stop function SFi is disabled. After processing S104, the process proceeds to S105.

[0211] In S105, the driving system 2 (e.g., the participation status management unit 81) determines whether a preset grace period has elapsed. The grace period is a preset period, e.g., one minute, starting from the timing when the vehicle user performs an operation to agree to the test. If the answer is No, proceed to S106. If the answer is Yes, proceed to S108.

[0212] In S106, the driving system 2 (e.g., the participation status management unit 81 and the HMI linkage unit 84) accepts an operation to cancel consent from the vehicle user. After processing S106, the process proceeds to S107. In S107, the driving system 2 (e.g., the participation status management unit 81) determines whether or not a consent cancellation operation has been performed. If the answer is Yes, the process proceeds to S110. If the answer is No, the process returns to S105.

[0213] In S108, the driving system 2 (for example, the participation state management unit 81) disables the individual stop function SFi of the corresponding vehicle 1. After the processing of S108, the process proceeds to S109.

[0214] In S109, the server 96 (e.g., the test management unit 97a) or the driving system 2 (e.g., the participation status management unit 81) places the vehicle 1 in a test implementation state in which the individual stop function has been disabled. That is, the test is started in the vehicle 1. The series of processes ends with S109.

[0215] On the other hand, if consent to the test is not obtained in S102, or if it is determined in S107 that consent has been cancelled, the driving system 2 (e.g., the participation status management unit 81) sets the participation status of the vehicle 1 to "not participating in the test" in S110. In other words, it is determined that the test will not be conducted in the vehicle 1. The series of processes ends with S110.

[0216] <Example of Responding to a Vehicle User's Change of Mind> As shown in the flowchart of FIG. 7 , the management system MS uses the individual stop function SFi of the test stop unit MS2 to appropriately respond to a change of mind of the vehicle user after the start of the test. In the series of processes in the method of FIG. 7 , for example, on the driving system 2 side, the processor 53b of the software management unit 57 executes a computer program stored in the memory 53a. In response to this, the processor 96b of the server 96 may execute a computer program stored in the memory 96a. When this process starts, a test is being performed, and the individual stop function SFi for the specific driving system 2 being processed is disabled. A trigger for starting this process may be provided by an input operation by the vehicle user to the operation input device 70a.

[0217] In the first step S201, the driving system 2 (e.g., the participation status management unit 81) determines whether the vehicle user wishes to leave the test, for example, based on the input operation described above. If the determination is Yes, the process proceeds to S202. If the determination is No, the process ends.

[0218] In S202, the driving system 2 (e.g., the participation status management unit 81) refers to the consent details. If the consent details are managed by the test management unit 97a, the consent details are acquired from the server 96 via the communication system 43. After processing S202, the process proceeds to S203.

[0219] In S203, the driving system 2 (e.g., the participation status management unit 81) determines whether the consent content allows for withdrawal due to the vehicle user. If the answer is Yes, proceed to S204. If the answer is No, proceed to S206.

[0220] In S204, the driving system 2 (for example, the participation status management unit 81) enables the individual stop function SFi. After the processing of S204, the process proceeds to S205.

[0221] In S205, the operation system 2 (for example, the software application unit 82) disables the test software and enables the conventional software in response to the individual stop function SFi being enabled. After S205, the series of processes ends.

[0222] In S206, the driving system 2 (e.g., the HMI linkage unit 84) notifies the vehicle user via the information presentation device 70b that the test withdrawal is not permitted. In this notification, the actual consent content may be presented to the vehicle user. After S206, the series of processes ends.

[0223] <Example of response to conventional functions or conventional performance that cannot be performed due to test software> As shown in the flowchart of Figure 8, when there is a conventional function or conventional performance that cannot be performed due to test software, the management system MS uses the individual stop function SFi of the test stop unit MS2 to perform the conventional function or conventional performance of the vehicle 1.

[0224] For example, if the conventional software complies with the road traffic laws of multiple countries across borders, while the test software complies with the road traffic laws of only one country, and the test management unit 97a limits the test area itself, in this case, when the vehicle 1 moves outside the test area, the functions of the conventional software are required.

[0225] In the series of processes in the method of FIG. 8 , for example, on the driving system 2 side, the processor 53b of the software management unit 57 executes a computer program stored in the memory 53a. In response to this, the processor 96b of the server 96 may execute a computer program stored in the memory 96a. At the start of this process, a test is being performed, and the individual stop function SFi for the specific driving system 2 being processed is disabled. In other words, the test software is enabled, and the conventional software is disabled. Furthermore, the test management unit 97a functioning as the test execution unit MS1 limits the area in which the test is performed to a specific country. The specific country is the country in which the driver of the vehicle 1 resides.

[0226] In the first step S301, the operation system 2 (for example, the participation status management unit 81 functioning as the test stop unit MS2) acquires information about the test implementation area set by the test management unit 97a. After the processing of S301, the process proceeds to S302.

[0227] In S302, the driving system 2 (e.g., the participation status management unit 81) determines whether the vehicle 1 has moved outside the implementation area. If the determination is Yes, the process proceeds to S303. If the determination is No, the process of S302 is executed again after a predetermined time.

[0228] In S303, the driving system 2 (for example, the participation status management unit 81) enables the individual stop function SFi. After the processing of S303, the process proceeds to S304.

[0229] In S304, the operation system 2 (for example, the software application unit 82) disables the test software and enables the conventional software in response to the activation of the individual stop function SFi. After the processing of S304, the process proceeds to S305.

[0230] In S305, the driving system 2 (e.g., the participation status management unit 81) determines whether the vehicle 1 has returned to the implementation area. If the determination is Yes, the process proceeds to S306. If the determination is No, the process of S305 is executed again after a predetermined time.

[0231] In S306, the driving system 2 (for example, the participation status management unit 81) disables the individual stop function SFi. After the processing of S306, the process proceeds to S307.

[0232] In S307, the driving system 2 (e.g., the software application unit 82) disables the conventional software and enables the test software in response to the activation of the individual stop function SFi. The series of processes ends with S307. Note that, during the test, the process of changing the individual stop function SFi according to the position of the vehicle 1 is repeated. That is, after the process of S307, the process of S302 may be executed again after a predetermined time.

[0233] The test area may be set based on legal restrictions or other circumstances regarding the implementation of market tests. The test area may be limited to special economic zones where market tests are permitted. Highly sensitive areas such as military facilities may be excluded from the test area. If the test software is not suitable for testing in urban areas with heavy traffic, the test area may be limited to suburban areas with low traffic.

[0234] <Example of Response to Pending Issues in Test Software> The management system MS selectively uses the overall halt function SFw and the individual halt function SFi of the test halt unit MS2 to appropriately respond to pending issues in the test software. In the management system MS, the test halt unit MS2 monitors one or more metrics for determining the halt function when testing the test software on a public road. This metric may be the same as or different from the evaluation index evaluated by the test software evaluation unit 97d. Since this metric is an index for determining whether the test can be safely continued, it is preferably the safety-related index described above.

[0235] The test stop unit MS2 then sets one or more thresholds in advance for one or more monitored metrics. The thresholds define the allowable range of the metrics. The thresholds may be set, for example, based on a boundary that distinguishes between a safe state and an unsafe state. The test stop unit MS2 determines that the metrics are outside the allowable range based on the thresholds, and selectively activates the overall stop function SFw and the individual stop functions SFi in accordance with this determination. The fact that the metrics are outside the allowable range based on the thresholds is an example of a stop condition.

[0236] The stop conditions may be separated into individual stop conditions and overall stop conditions, which may be operated separately. To selectively enable the overall stop function SFw and the individual stop function SFi, for example, each vehicle 1, which is a test target vehicle, may monitor a metric for determining the individual stop condition based on the individual stop condition for enabling the individual stop function SFi for each vehicle. The determination of whether to enable the individual stop function SFi may be made by self-diagnosis by the participation status management unit 81 of each vehicle 1, or may be made by the test management unit 97a of the server 96 that receives the monitoring results. The metrics and thresholds used for self-diagnosis may be common among test target vehicles participating in the same test. Meanwhile, the server 96 may monitor a metric for determining the overall stop condition based on the overall stop condition for enabling the overall stop function SFw.

[0237] Here, the method shown in the flowcharts of Figures 9 and 10 will be described below as an example of how to address the pending issue. In the series of processes shown in Figure 9, the processor 53b of the software management unit 57 of the driving system 2 in each vehicle 1 functions as the test stop unit MS2 and executes a computer program stored in the memory 53a. The series of processes shown in Figure 9 is started, for example, in accordance with the timing of the start of testing for each vehicle 1. In the series of processes shown in Figure 10, the processor 96b of the server 96 functions as the test stop unit MS2 and executes a computer program stored in the memory 96a. The series of processes shown in Figure 10 is started, for example, in accordance with the timing when the server 96 determines that testing as a whole should be started.

[0238] 9, in the first step S401, the driving system 2 (e.g., the participation status management unit 81) sets one or more metrics and their thresholds to be used as conditions for individual stops. This setting may be performed by acquiring the settings of the metrics and their thresholds determined by the test management unit 97a of the server 96. This setting may be obtained by acquiring the settings determined by the test management unit 97a and partially modifying them based on the individual specifications and individual driving environments of the vehicle 1. After processing S401, the process proceeds to S402.

[0239] The one or more metrics may include the number of software errors. Software errors may include software failures and software defects. Software errors may be the number of times the software itself outputs a diagnostic code, or the number of times the driving system 2 detects an abnormality in the software. The number of software errors may be the number of times errors occur in the test software only, or the number of times errors occur including software directly related to the test software, or the number of times errors occur in all software in the driving system 2. Software directly related to the test software is, for example, software that directly references the output of the test software to execute processing. Instead of the number of errors, a function weighting error occurrences by one or both of the importance and urgency of the error may be used as a metric.

[0240] One or more metrics may include an error indicating a decrease in performance. The performance here may be general performance. If the test software is software that electronically controls a damper included in the motion actuator 60, the error may be a value indicating the occurrence of abnormal shaking of the vehicle 1 when traveling around a curve. If the test software is software that realizes the image recognition function of a camera as the external environment sensor 41, the error may be a value indicating a decrease in the object recognition rate due to disturbances such as bad weather.

[0241] The one or more metrics may be the number of violations of the safety determination. The violation of the safety determination may be a violation of the safety envelope or a violation of the safety distance. When the violation of the safety determination is determined to include the collision risk exceeding a risk threshold, a function weighted by one or both of the importance and urgency of the violation may be adopted as the metric instead of the number of violations. When the violation of the safety determination is determined to include the vehicle 1 violating a rule stored in the rule set, a function weighted by one or both of the importance and urgency of the violation may be adopted as the metric instead of the number of violations. In other words, the above-mentioned violation metric may be adopted as a metric used as a condition for an individual stop. Furthermore, the one or more metrics may include the number of times monitoring of the behavior of the vehicle 1 operated by the driving system 2 according to the driving policy fails.

[0242] In S402, the driving system 2 (for example, the participation state management unit 81) continuously monitors the metric set in S401. After the processing of S402, the process proceeds to S403.

[0243] In S403, the driving system 2 (e.g., the participation status management unit 81) determines whether the monitored metric exceeds the threshold set in S401. In other words, it determines whether the individual stop condition is met. If multiple metrics are set, it is set in advance in S401 whether all metrics must exceed the corresponding threshold, or whether it is sufficient for one or some of the metrics to exceed the corresponding threshold, and the determination is made according to this condition setting. If the answer is Yes, proceed to S404. If the answer is No, return to S402, and the loop processing of S402 to S403 is repeated until the end of the test period.

[0244] In S404, the driving system 2 (for example, the participation status management unit 81) enables the individual stop function SFi. After the processing of S404, the process proceeds to S405.

[0245] In S405, the operation system 2 (for example, the software application unit 82) disables the test software and enables the conventional software in response to the individual stop function SFi being enabled. After the processing of S405, the process proceeds to S406.

[0246] In S406, the driving system 2 (e.g., the participation status management unit 81) transmits to the test management unit 97a of the server 96 information that the test has been individually stopped in the vehicle 1 in which it is installed. The reason for the individual test stop may also be transmitted, and metric monitoring data may also be transmitted. S406 marks the end of the series of processes.

[0247] Next, the processing in the server 96 corresponding to the processing in the vehicle 1 in Figure 9 will be described with reference to Figure 10. In the first step S501, the server 96 (e.g., the test management unit 97a) sets one or more metrics and their thresholds to be used as conditions for overall shutdown. The metric may be, for example, the number of vehicles 1 that have been individually stopped during testing due to self-diagnosis. Instead of the number of vehicles, the metric may be the ratio of vehicles 1 that have been individually stopped to all tested vehicles, i.e., the individual shutdown rate. In the following, the description will be continued assuming that the metric is the number of vehicles. After processing in S501, the process proceeds to S502.

[0248] In S502, the server 96 (for example, the test management unit 97a) counts the number of vehicles that have individually stopped the test based on the information received from each vehicle 1 in the process of S406. After the process of S502, the process proceeds to S503.

[0249] In S503, the server 96 (e.g., the test management unit 97a) determines whether the number of vehicles counted in S502 exceeds a threshold value. In other words, it determines whether the condition for total stop is met. If the answer is Yes, the process proceeds to S504. If the answer is No, the process returns to S502, and the loop process of S502 to S503 is repeated until the end of the test period.

[0250] In S504, the server 96 (for example, the test management unit 97a) enables the entire shutdown function SFw. After the process of S504, the process proceeds to S505.

[0251] In S505, the server 96 (e.g., the test management unit 97a) sends a message to all test target vehicles informing them that the test is to be stopped as a whole. In each vehicle 1 that receives the message in S505, the participation status management unit 81 activates the overall stop function SFw.

[0252] The transmission destination may include a vehicle 1 in which the individual stop function is already enabled. If the individual stop function SFi is disabled again based on some trigger, and the participation status management unit 81 of the vehicle 1 does not recognize that the whole stop function SFw has been enabled, the test may be restarted in that vehicle 1. The series of processes ends at S505.

[0253] According to the first embodiment described above, the individual stop function SFi and the overall stop function SFw can be selectively used for each participant. This makes it possible to stop a test in an appropriate manner, taking into consideration the validity of the test operation and usability.

[0254] Furthermore, in the first embodiment, particularly according to the first method described above, in the test preparation state, the conventional software applied to the hardware of the driving system 2 as a vehicle system corresponding to the participating unit is rewritten with test software downloaded from the server 96 to the driving system 2. Then, when a specific participating unit individually stops the test using the individual stop function SFi, the conventional software is downloaded from the server 96, and the test software is rewritten as the conventional software. Rewriting and rewriting in response to the start and stop of the test makes it possible to perform and stop the test using fewer hardware resources.

[0255] Furthermore, in the first embodiment, particularly according to the second method described above, in the preparation state, test software is downloaded from the server 96 to the hardware of the driving system 2 corresponding to the participating unit, separately from the conventional software. Then, both the conventional software and the test software are left in the hardware, with one or both of them being activatable. Furthermore, when the participating unit stops the test, the conventional software is activated and the test software is deactivated, while both remain in the hardware. In this way, when the vehicle 1 stops the test, the test software remains in the hardware of the driving system 2, making it easy to resume the test. Furthermore, since the software state at the time of the test is saved, it becomes easy to verify the test after the fact. Therefore, it is possible to stop the test in an appropriate manner.

[0256] Furthermore, according to the first embodiment, in the individual stop function SFi in the preparation state, when a participating unit performs an operation to indicate its intention to participate in a test through the HMI device 70, the test of the participating unit is stopped without starting until the grace period during which the participation intention indication operation can be canceled has elapsed. Therefore, even if a participating unit erroneously performs the operation to indicate its intention to participate, it is possible to prevent the test from affecting the behavior of the vehicle 1 against its actual intention.

[0257] According to the first embodiment, a consent is obtained from a specific participant unit that the test participation cannot be cancelled for a predetermined period of time, and the individual stop function SFi for the specific participant unit is fixed to a disabled state for the predetermined period of time. By avoiding the easy activation of the individual stop function SFi, the test can be conducted efficiently, leading to the prompt improvement of the operation system 2.

[0258] Furthermore, according to the first embodiment, a test implementation area is set. Then, when a specific vehicle among multiple vehicles moves out of the implementation area during the test, the individual stop function SFi is used to individually stop the test at the participating unit corresponding to the specific vehicle. In other words, it is possible to prevent the entire test from being stopped due to the movement of only some vehicles out of the implementation area. Therefore, the test can be operated efficiently, leading to rapid improvements to the driving system 2.

[0259] Furthermore, according to the first embodiment, when a specific vehicle among a plurality of vehicles satisfies the individual stop condition, the individual stop function SFi is used to individually stop the test for the participating unit corresponding to the specific vehicle. When the overall stop condition is met, the overall stop function is used to stop the entire test. The individual stop function SFi and the overall stop function can be used separately based on different activation conditions, making it possible to stop the test in an appropriate manner.

[0260] Furthermore, according to the first embodiment, the condition for stopping the entire vehicle is that the number or percentage of vehicles for which testing has been stopped individually is equal to or greater than a predetermined threshold value among the plurality of vehicles 1. By associating the condition for stopping the entire vehicle with the individual stopping function, testing can be stopped in stages, thereby improving the validity of the testing process.

[0261] Furthermore, according to the first embodiment, whether or not to stop the test is determined individually for each vehicle using metrics, and whether or not to stop the test as a whole is determined based on the status of multiple vehicles 1. By determining the stop function in stages in this way, it becomes possible to stop the test in an appropriate manner.

[0262] 11 and 12, the second embodiment is a modification of the first embodiment. The second embodiment will be described, focusing on the differences from the first embodiment.

[0263] In the second embodiment, the HMI linking unit 84 links with the participation status management unit 81 to notify the vehicle user at an appropriate timing. Therefore, the software management unit 57 functions as a notification control device that controls the implementation of notifications regarding the test.

[0264] In the series of processes shown in the flowchart of Fig. 11, the processor 53b of the software management unit 57 of the driving system 2 functions as the test stop unit MS2 and executes a computer program stored in the memory 53a. The series of processes in Fig. 11 are repeatedly executed at predetermined intervals during the test.

[0265] In the first step S601, the driving system 2 (e.g., the participation status management unit 81) acquires the conditions for individual stops for the test. The conditions for the individual stops here include the test implementation area, and the test is to be conducted only on expressways. The conditions for the individual stops may be linked to the ODD. For example, if the conditions for implementing level 3 autonomous driving are limited to expressways, the test implementation area will substantially match the ODD. After processing S601, the process proceeds to S602.

[0266] In S602, the driving system 2 (e.g., the participation status management unit 81) determines whether the conditions for individual stop will be met soon. For example, if the vehicle 1 is traveling on a highway, it determines whether the vehicle plans to get off the highway soon. "Soon" may mean within a preset time, such as five minutes from now. In other words, it is sufficient to determine whether the possibility that the conditions for individual stop will be met within the preset time is equal to or greater than a predetermined threshold. If the answer is Yes, proceed to S603. If the answer is No, the series of processes ends.

[0267] In S603, the driving system 2 (e.g., the HMI linkage unit 84) uses the information presentation device 70b to notify the vehicle user (e.g., the driver) that the test will be interrupted. The test interruption notification may include information that the test will be interrupted soon and the reason for the interruption.

[0268] For example, as shown in Fig. 12, text content such as "The test in progress will be interrupted soon because you will be exiting the expressway in 3 minutes" may be displayed on the screen of CID 70b2 together with the route guidance image. After processing S603, the process proceeds to S604.

[0269] In S604, the driving system 2 (for example, the participation status management unit 81) enables the individual stop function SFi. After the processing of S604, the process proceeds to S605.

[0270] In S605, the operation system 2 (for example, the software application unit 82) disables the test software and enables the conventional software in response to the individual stop function SFi being enabled. After the processing of S605, the process proceeds to S606.

[0271] In S606, the driving system 2 (e.g., the driving planner 22 and the motion controller 31) executes vehicle control that causes the individual stop condition to be satisfied. For example, the vehicle 1 is autonomously guided to the highway exit and proceeds onto an ordinary road. The series of processes ends with S606.

[0272] The test interruption notification may be issued before the vehicle control that causes the stop condition to be satisfied is executed and the test is actually interrupted. Therefore, if it is determined in S602 that the vehicle control in S606 is likely to be executed before the notification in S603, the participation status management unit 81 may output a request to suspend or delay the vehicle control to the driving planner 22, and adjust the test interruption notification to be executed before the vehicle control. For example, the driving planner 22 may control the speed of the vehicle 1 to be slower, thereby delaying the timing of exiting the expressway.

[0273] According to the second embodiment described above, the notification that the test will be stopped is made to the vehicle user before the control of the vehicle 1 that will cause the individual stop condition to be satisfied is executed. This adjustment allows the vehicle user to be aware that the test will be stopped before a situation arises in which the test must be stopped, thereby improving the validity of the test process.

[0274] 13 and 14, the third embodiment is a modification of the second embodiment. The third embodiment will be described, focusing on the differences from the second embodiment.

[0275] In the third embodiment, the HMI linkage unit 84 issues a test interruption notification at a timing that avoids the vehicle 1 being in motion. When a test is being conducted while the driver is manually driving, avoiding the vehicle being in motion reduces the adverse effect of the notification on the driving. The test while the driver is manually driving may be, for example, a test of an HMI that provides route guidance to the driver to a destination.

[0276] In the series of processes shown in the flowchart of Fig. 13, the processor 53b of the software management unit 57 of the driving system 2 functions as the test stop unit MS2 and executes a computer program stored in the memory 53a. The series of processes in Fig. 13 are repeatedly executed at predetermined intervals during the test.

[0277] In the first step S701, the driving system 2 (e.g., the participation status management unit 81) acquires the conditions for individual stop of the test. After processing of S701, in step S702, the driving system 2 (e.g., the participation status management unit 81) determines whether the conditions for individual stop are met. If the answer is Yes, the process proceeds to step S703. If the answer is No, the process proceeds to step S704.

[0278] In S703, the driving system 2 (for example, the participation state management unit 81) enables the individual stop function SFi. After the processing of S703, the process proceeds to S704.

[0279] In S704, the operation system 2 (e.g., the participation status management unit 81) acquires the status of the entire stop function SFw from the test management unit 97a of the server 96 and determines whether the entire stop function SFw is enabled. If the answer is Yes, the process proceeds to S705. If the answer is No, the process ends.

[0280] In S705, the operation system 2 (for example, the software application unit 82) disables the test software and enables the conventional software in response to the activation of the individual stop function SFi or the whole stop function SFw. After the processing of S705, the process proceeds to S706.

[0281] In S706, the driving system 2 (for example, the HMI linkage unit 84) determines whether the vehicle 1 is traveling. If the determination is Yes, the determination in S706 is repeated after a predetermined time has elapsed. If the determination is No, the process proceeds to S707.

[0282] In S707, the driving system 2 (e.g., the HMI linkage unit 84) uses the information presentation device 70b to notify the vehicle user (e.g., the driver) that the test has been interrupted. The test interruption notification may include information that the test has been interrupted and the reason for the interruption.

[0283] 14 shows an example of a notification when the overall stop function SFw is enabled and the test is stopped. The screen of CID 70b2 may display text content such as "Route guidance test during manual driving has been stopped" and text content such as "(Reason) This is because there has been an increase in cases where drivers have misinterpreted the route guidance display and made unreasonable lane changes." The process of S707 ends the series of processes.

[0284] The HMI linking unit 84 may further adjust the timing of the notification in more detail. For example, even if the HMI linking unit 84 allows a notification when the vehicle 1 stops temporarily, the notification may be made when the vehicle stops at a stop signal, avoiding the timing when the vehicle stops temporarily at a stop line. This is because, when the vehicle stops temporarily at a stop line, it is possible that the driver is concentrating on monitoring the surroundings in preparation for restarting, and the timing of restarting is difficult to predict. Furthermore, the HMI linking unit 84 may postpone making a notification until the vehicle 1 starts traveling again after parking, and may make a notification when the driver gets inside the vehicle to resume traveling, or when the power switch (ignition switch) of the vehicle 1 is turned on.

[0285] According to the third embodiment described above, when a participant unit stops a test, the notification that the test will be stopped is given at a timing that avoids the vehicle 1 being in motion. Since the notification is given at a timing when the vehicle user is likely to be focused, the vehicle user is more likely to be aware of the notification.

[0286] 15 and 16, the fourth embodiment is a modification of the second embodiment. The fourth embodiment will be described, focusing on the differences from the second embodiment.

[0287] In the fourth embodiment, the test agreement includes a policy that allows the vehicle user to temporarily withdraw from the test at will. Therefore, the participation status management unit 81 and the HMI linkage unit 84, which function as the test stop unit MS2, provide a function that allows the vehicle user to set a mode for temporarily suspending the test. The temporary suspension here corresponds to one mode of the individual suspension function SFi.

[0288] For example, as shown in FIG. 15 , the HMI linking unit 84 provides a user interface UI1, for example, using a CID 70 b 2, for the vehicle user to set the type of temporary suspension of the test. The temporary suspension type includes a period-based type and a situation-based type. In the period-based type, the vehicle user can directly set the period for temporarily suspending the test. For example, the vehicle user may be allowed to set the start date and end date of the period.

[0289] In the situation-based mode, the vehicle user individually sets whether or not the test should be paused when a corresponding situation occurs. Therefore, it is preferable that multiple situations can be individually set. For example, in FIG. 15, the interface provides configurable situations such as "when a passenger is present," "during commuting," and "nighttime."

[0290] For example, if the vehicle user sets the trip to be temporarily suspended "when there is a passenger," the participation status management unit 81 may acquire sensor data such as a seat occupancy sensor to determine whether there is a passenger. Also, if the vehicle user sets the trip to be temporarily suspended "during commuting," the participation status management unit 81 may acquire departure point information and destination information in the route plan to determine whether the departure point or destination is a workplace registered in advance by the vehicle user.

[0291] If the suspension period continues for a predetermined period or longer, the participation status management unit 81 and the HMI linkage unit 84 may issue a reminder to remind the vehicle user of the test. For example, in the series of processes shown in the flowchart of Figure 16, the processor 53b of the software management unit 57 of the driving system 2 functions as the test suspension unit MS2 and executes a computer program stored in the memory 53a. The series of processes shown in Figure 16 is repeatedly executed in the suspension state set by the vehicle user.

[0292] In the first step S801, the driving system 2 (e.g., the participation status management unit 81) acquires the duration of the enabled state of the individual stop function SFi. In step S802 after processing of step S801, the driving system 2 (e.g., the participation status management unit 81) determines whether the duration acquired in step S801 exceeds a preset threshold value (e.g., one week). If the answer is Yes, the process proceeds to step S803. If the answer is No, the process ends.

[0293] In S803, the driving system 2 (e.g., the HMI linkage unit 84) issues a reminder notification. The reminder notification may include a notification prompting the vehicle user to decide whether to continue participating in the test or to cancel the test. The reminder notification may also include a notification that re-explains the test content.

[0294] According to the fourth embodiment described above, the user interface UI1 allows the vehicle user to set a temporary suspension mode for the test on a participant basis. The activation of the individual suspension function SFi is determined based on the setting in the user interface UI1. This improves the usability of the test.

[0295] Furthermore, according to the fourth embodiment, if the duration of the temporary stop based on the setting in the user interface UI1 exceeds a preset period, a reminder notification is issued to remind the vehicle user of the test. This increases the likelihood that the user who remembers the test will return to the test, thereby enabling the test to be conducted efficiently and leading to rapid improvement of the driving system 2.

[0296] Fifth Embodiment As shown in Fig. 17, the fifth embodiment is a modification of the first embodiment. The fifth embodiment will be described, focusing on the differences from the first embodiment.

[0297] In the fifth embodiment, a vehicle used for MaaS, such as a robot taxi, an autonomous bus, or a ride-sharing vehicle, is assumed as the vehicle 1. In the fifth embodiment, the unit of participation in the test is not a vehicle unit but a passenger unit. The passenger unit may be a single passenger or a group of passengers.

[0298] The test on a passenger basis may be a test on the HMI used by the passenger while riding in the vehicle 1, or a test on a new payment service. The function to be tested may include a function provided in cooperation with a mobile terminal 91 such as a smartphone owned by the passenger via the communication system 43 or the like.

[0299] In the fifth embodiment, the participation status management unit 81 of the operation system 2 confirms with each passenger whether they wish to refuse the test, and then conducts the test on each passenger based on the confirmation. For example, in the series of processes shown in the flowchart of Fig. 17, the processor 53b of the software management unit 57 of the operation system 2 functions as the test stop unit MS2 and executes a computer program stored in the memory 53a. The series of processes shown in Fig. 17 are repeatedly executed at predetermined intervals during the test.

[0300] In the first step S901, the operation system 2 (for example, the participation status management unit 81) determines whether a new passenger has boarded the vehicle 1. If the answer is Yes, the process proceeds to S902. If the answer is No, the series of processes ends.

[0301] In S902, the operation system 2 (e.g., the participation status management unit 81) determines whether the new passenger has refused the test. This determination may be made by acquiring a preset intention from the passenger in an application used by the MssS. This determination may also be made based on information obtained when the new passenger confirms their intention using the in-vehicle HMI when boarding the vehicle 1. If the answer is Yes, proceed to S903. If the answer is No, proceed to S905.

[0302] In S903, the operation system 2 (e.g., the participation status management unit 81) enables the individual stop function SFi for the passenger who is the subject of the determination in S902. In S904 after the processing of S903, the operation system 2 (e.g., the software application unit 82) sets the passenger who is the subject of the determination in S902 to not be provided with the test function but to be provided with only the conventional function. The series of processes ends with S904.

[0303] In S905, the operation system 2 (e.g., the participation status management unit 81) disables the individual stop function SFi for the passenger who is the subject of the determination in S902. In S906 after the processing of S905, the operation system 2 (e.g., the software application unit 82) sets the test function to be provided to the passenger who is the subject of the determination in S902. The series of processes ends with S904.

[0304] According to the fifth embodiment described above, the participation unit is set to a passenger unit based on the passenger. The individual stop function SFi is enabled for passenger units whose intention to refuse the test is confirmed, and the test function is provided to passenger units whose intention not to refuse the test. Since it is possible to conduct the test in the vehicle 1 while respecting the wishes of individual passengers, usability can be improved.

[0305] (Other Embodiments) Although multiple embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0306] In another embodiment, when a test is conducted using a POV, which is a vehicle 1 that can participate in public road traffic, the unit of participation in the test may be an individual. In this case, the individual may include not only the driver but also passengers.

[0307] For example, suppose a test is conducted on the display layout of a vehicle 1 equipped with displays for the driver's seat and passenger seat, and displays for the rear seats. If a passenger in the rear seat declines to participate in the test, the test may be conducted only on the displays for the driver's seat and passenger seat. Furthermore, for example, in a test on the temperature control of a seat heater, the test may be conducted only in the seats occupied by passengers who wish to participate.

[0308] In another embodiment related to the fifth embodiment, the test system may be configured to eliminate the need for the server 96 and to be completed within a vehicle used for one MaaS. In this case, the driving system 2 may have all the functions of the test execution unit MS1, the test stop unit MS2, and the software management unit MS3. The test stop unit MS2 included in the driving system 2 may control both the individual stop function SFi and the overall stop function.

[0309] In other embodiments, the processing system 50 may have a configuration as shown in Figures 17 and 18. For example, Figure 18 shows a configuration including multiple domain controllers 451 to 454. Each of the domain controllers 451 to 454 may have the same hardware configuration as the processing system or ECU of the first embodiment.

[0310] The ADAS domain controller 451 aggregates functions related to ADAS (Advanced Driver-Assistance Systems). The ADAS domain controller 451 may comprehensively realize a portion of the recognition function, a portion of the judgment function, and a portion of the control function. The portion of the recognition function realized by the ADAS domain controller 451 may be, for example, a function corresponding to the fusion of information detected by the multiple sensors 40 in the detection unit 10 of the first embodiment, or a simplified function thereof. The portion of the judgment function realized by the ADAS domain controller 451 may be, for example, a function corresponding to the prediction unit 21 and the driving planner 22 of the first embodiment, or a simplified function thereof. The portion of the control function realized by the ADAS domain controller 451 may be, for example, a function corresponding to the motion control unit 31 of the first embodiment, that generates request information for the motion actuator 60.

[0311] The powertrain domain controller 452 aggregates functions related to the control of the powertrain. The powertrain domain controller 452 may compositely realize at least a part of the recognition function and at least a part of the control function. A part of the recognition function realized by the powertrain domain controller 452 may be, for example, a function corresponding to the internal recognition unit 13 in the first embodiment, which recognizes the driver's operation state with respect to the motion actuator 60. A part of the control function realized by the powertrain domain controller 452 may be, for example, a function corresponding to the motion control unit 31 in the first embodiment, which controls the motion actuator 60.

[0312] The cockpit domain controller 453 aggregates functions related to the cockpit. The cockpit domain controller 453 may compositely realize at least a part of the recognition function and at least a part of the control function. A part of the recognition function realized by the cockpit domain controller 453 may be, for example, a function of recognizing the switch state of the HMI device 70, which is included in the internal recognition unit 13 of the first embodiment. A part of the control function realized by the cockpit domain controller 453 may be, for example, a function corresponding to the HMI output unit 71 of the first embodiment.

[0313] The connectivity domain controller 454 aggregates connectivity-related functions. The connectivity domain controller 454 may comprehensively realize at least a part of the recognition function. Part of the recognition function realized by the connectivity domain controller 454 may be a function to organize and convert the global position data of the vehicle, V2X information, etc. acquired from the communication system 43 into a format usable by the ADAS domain controller 451 and the cockpit domain controller 453, for example.

[0314] In this embodiment, when software management is performed by each domain controller 451 to 454 that realizes the respective functions, the multiple domain controllers 451 to 454 may each correspond to a "software management unit" that constitutes the management system MS or the test system.

[0315] 19 employs a configuration including an integrated ECU 551 and multiple zone ECUs 551a to 551d. In this configuration, the multiple zone ECUs 551a to 551d control devices, modules, units, equipment, etc. that are located in specific assigned zones of the vehicle 1.

[0316] For example, an external environment sensor 41 such as a camera arranged in the front of the vehicle 1, and an information presentation device 70b such as a CID 70b2 arranged in the cockpit, are controlled by a zone ECU 551a or 551b arranged in the front of the vehicle 1. For example, an external environment sensor 41 such as a millimeter wave radar arranged in the rear of the vehicle 1 is controlled by a zone ECU 551c or 551d arranged in the rear of the vehicle 1.

[0317] The integrated ECU 551 collects detection information and the like from each of the zone ECUs 551a to 551d and performs integrated control of the driving system 2 for each of the zone ECUs 551a to 551d. For example, the integrated ECU 551 may implement almost all of the planning function and software management function.

[0318] In another embodiment, the vehicles 1, 1A, and 1B equipped with the driving systems 2, 2A, and 2B may be right-hand drive vehicles or left-hand drive vehicles. Furthermore, the traffic environment in which the vehicles 1, 1A, and 1B travel may be a traffic environment where left-hand traffic is assumed, or a traffic environment where right-hand traffic is assumed. The driving systems 2, 2A, and 2B according to the present disclosure may be optimized as appropriate, taking into consideration the road traffic laws and customs of each country and region, as well as the forms of police investigations, prosecutions, criminal proceedings, and civil proceedings regarding traffic accidents.

[0319] The controller and methods described herein may be implemented by a special-purpose computer comprising a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and methods described herein may be implemented by special-purpose hardware logic circuitry. Alternatively, the apparatus and methods described herein may be implemented by one or more special-purpose computers comprising a processor executing a computer program in combination with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0320] (Disclosure of Technical Ideas) This specification discloses multiple technical ideas described in the following multiple clauses. Some clauses may be described in a multiple dependent form, where the subsequent clause alternatively cites the preceding clause. These multiple dependent clauses define multiple technical ideas.

[0321] <Technical Idea 1> A test system that uses one or more vehicles (1, 1A, 1B) that can participate in public road traffic to test vehicle systems (2, 2A, 2B) installed in the vehicles, the test system comprising: a test execution unit (MS1) that sets and executes the test; and a test stop unit (MS2) that has both an individual stop function (SFi) that stops the test individually on a participant basis and an overall stop function (SFw) that stops the test overall while the test is in a preparation state or an execution state.

[0322] <Technical Idea 2> The test system according to Technical Idea 1, wherein the test involves changing the software used in the vehicle system, and further comprises a software management unit (MS3) that manages the application of the software to the vehicle, wherein the software management unit, in the preparation state, rewrites the conventional software that was applied to the hardware (51 a, 53 a, 58, 59) of the vehicle system corresponding to the participating unit with test software downloaded from a server (96) to the vehicle system, and when a specific participating unit individually stops the test using the individual stop function, downloads the conventional software from the server and rewrites the test software to the conventional software.

[0323] <Technical Idea 3> The test system according to Technical Idea 1, wherein the test involves changing software used in the vehicle system, and further comprises a software management unit (MS3) that manages application of the software to the vehicle, wherein the software management unit: in the preparation state, downloads test software from a server (96) to hardware (51a, 53a, 58, 59) equipped in the vehicle system corresponding to the participating unit, separately from conventional software, leaves both the conventional software and the test software on the hardware in a state where one or both of the software can be enabled, and when the participating unit stops the test, leaves both software on the hardware, enables the conventional software, and disables the test software.

[0324] <Technical Idea 4> The test system described in any one of Technical Ideas 1 to 3, wherein, in the individual stop function in the ready state, when the participating unit executes an operation to express its intention to participate in the test through an HMI device, the test stop unit stops the test of the participating unit without starting it until a grace period during which the operation to express its intention to participate can be canceled has elapsed.

[0325] <Technical Idea 5> A test system described in any one of Technical Ideas 1 to 4, wherein the test stop unit obtains consent from a specific participating unit that participation in the test cannot be canceled for a predetermined period of time, and fixes the individual stop function for the specific participating unit in a disabled state for the predetermined period of time.

[0326] <Technical Idea 6> A test system described in any one of Technical Ideas 1 to 5, wherein the one or more vehicles are a plurality of vehicles, the test implementation unit sets an implementation area for the test, and the test stop unit, when a specific vehicle among the plurality of vehicles moves outside the implementation area in the implementation state, individually stops the test of the participation unit corresponding to the specific vehicle as the individual stop function.

[0327] <Technical Idea 7> The one or more vehicles are a plurality of vehicles, and the test stop unit sets an individual stop condition and an overall stop condition for the test, and when a specific vehicle among the plurality of vehicles satisfies the individual stop condition, stops the test of the participation unit corresponding to the specific vehicle individually as the individual stop function, and when the overall stop condition is satisfied, stops the test as a whole as the overall stop function, a test system described in any one of Technical Ideas 1 to 6.

[0328] <Technical Idea 8> The test system according to Technical Idea 7, wherein the condition for stopping the entire test is that the number or percentage of vehicles that have individually stopped the test among the plurality of vehicles is equal to or greater than a predetermined threshold value.

[0329] <Technical Idea 9> The test system described in any one of Technical Ideas 1 to 8, wherein the test stop unit sets a condition for an individual stop in the test, and adjusts so that a notification that the test will be stopped is made to a vehicle user before control of the vehicle that causes the condition for the individual stop to be satisfied is executed.

[0330] <Technical Idea 10> The test system according to any one of Technical Ideas 1 to 8, wherein, when the participating unit stops the test, the test stop unit issues a notification that the test will be stopped at a timing that avoids the vehicle being in motion.

[0331] <Technical Idea 11> The test system described in any one of Technical Ideas 1 to 10, wherein the test stop unit provides a user interface (UI1) that allows a vehicle user to set a form for temporarily stopping the test in the participation unit, and determines whether to enable the individual stop function based on the setting in the user interface.

[0332] <Technical Idea 12> The test system described in Technical Idea 11, wherein the test stop unit issues a reminder notification to remind the vehicle user of the existence of the test when the duration of the temporary stop based on the setting in the user interface exceeds a preset period.

[0333] <Technical Idea 13> A test system described in any one of Technical Ideas 1 to 12, wherein the vehicle is a vehicle provided for providing services to passengers, the test is a test in which the participation unit is a passenger unit based on the passenger, and the test stop unit enables the individual stop function for the passenger unit whose intention to refuse the test is confirmed, and provides the test function to the passenger unit whose intention not to refuse the test.

[0334] <Technical Idea 14> A test system that uses a plurality of vehicles (1, 1A, 1B) that can participate in public road traffic to test vehicle systems (2, 2A, 2B) installed in each of the vehicles, the test system including: each of the vehicle systems; and a server (96) communicatively connected to each of the vehicle systems, wherein each of the vehicle systems uses one or more metrics to determine, in each of the vehicles, whether to enable an individual stop function that stops the test individually on a vehicle-by-vehicle basis; and the server determines, based on the status of activation of the individual stop functions in the plurality of vehicles, whether to enable a global stop function that stops the test as a whole.

[0335] According to this technical concept, each vehicle system uses metrics to determine whether to stop the test individually, and the server determines whether to stop the test as a whole based on the status of multiple vehicles. By determining the stop function in this way, it is possible to stop the test in an appropriate manner.

[0336] <Technical Idea 15> A vehicle system configured to be mounted on a vehicle (1, 1A, 1B) capable of participating in public road traffic and comprising at least one processor (57b) for conducting tests related to the vehicle, wherein the at least one processor performs the following operations: determining one or more metrics and thresholds thereof; monitoring the one or more metrics and determining, based on a relationship between the one or more metrics and thresholds, whether to enable an individual stop function that stops tests individually on a vehicle-by-vehicle basis; and transmitting information indicating that the individual stop function has been enabled to a server (96) that manages the entire test.

[0337] According to this technical concept, each vehicle system uses metrics to individually determine whether to stop the test. Therefore, even if there is an urgent reason for stopping the test, the test can be stopped quickly on a vehicle-by-vehicle basis. When an individual test is stopped, that information is sent to the server, allowing the server to objectively manage the test process.

[0338] <Technical Idea 16> A server that is communicatively connected to each of a plurality of vehicles (1, 1A, 1B) that can participate in public road traffic and that includes at least one processor (96b) in order to test vehicle systems (2, 2A, 2B) installed in each of the vehicles, wherein the at least one processor performs the following operations: receiving information from the vehicles indicating that an individual stop function that stops a test individually on a vehicle-by-vehicle basis through self-diagnosis has been enabled; and determining to enable a global stop function that stops the test as a whole, based on the status of the enablement of the individual stop function in each of the vehicles.

[0339] According to this technical concept, each vehicle determines whether to stop the test individually using metrics through self-diagnosis, and the server determines whether to stop the test as a whole based on the status of multiple vehicles. By determining the stop function in this way in stages, it becomes possible to stop the test in an appropriate manner.

[0340] <Technical Idea 17> A notification control device configured to be mounted on a vehicle (1, 1A, 1B) that can participate in public road traffic and having at least one processor (57b) for controlling the implementation of notifications about tests related to the vehicle, wherein the at least one processor: refers to an individual stop condition under which the vehicle will individually stop the test; determines whether the individual stop condition will soon be met; and if it is determined that the individual stop condition will soon be met, adjusts so that a notification about the test being stopped is made to a vehicle user before control of the vehicle that will cause the individual stop condition to be met is executed.

[0341] According to this technical concept, the notification that the test will be stopped is adjusted to be sent to the vehicle user before the vehicle control that will cause the individual stop condition to be satisfied is executed. This adjustment allows the vehicle user to be aware of the test stopping before the situation becomes such that the test has to be stopped, thereby improving the validity of the test process.

[0342] <Technical Idea 18> A notification control device configured to be mounted on a vehicle (1, 1A, 1B) capable of participating in public road traffic, and comprising at least one processor (57b) for controlling the implementation of notifications regarding tests related to the vehicle, wherein the at least one processor, when stopping the test performed while the vehicle is manually driven, determines whether the vehicle is moving or not, and, based on the determination, issues a notification regarding the test being stopped at a timing that avoids the vehicle being in motion.

[0343] According to this technical concept, when the vehicle stops the test, the notification that the test will be stopped is given at a timing that avoids the vehicle being in motion. Since the notification is given at a timing when the vehicle user is likely to be focused, the vehicle user is more likely to be aware of the notification.

Claims

1. A test system for testing vehicle systems (2, 2A, 2B) installed in one or more vehicles (1, 1A, 1B) capable of participating in public road traffic, comprising: a test execution unit (MS1) that sets and executes the test; and a test stop unit (MS2) that has both an individual stop function (SFi) that stops the test individually on a participation unit basis and a global stop function (SFw) that stops the test as a whole when the test is in a preparation state or execution state.

2. The test system of claim 1, further comprising a software management unit (MS3) that manages the application of the software to the vehicle, wherein the test involves a change to the software used in the vehicle system, and wherein the software management unit: in the preparation state, rewrites the conventional software applied to the hardware (51a, 53a, 58, 59) of the vehicle system corresponding to the participating unit with test software downloaded from a server (96) to the vehicle system; and when a specific participating unit individually stops the test using the individual stop function, downloads the conventional software from the server and rewrites the test software to the conventional software.

3. The test system of claim 1, further comprising a software management unit (MS3) for managing the application of the software to the vehicle, wherein the test involves a change to software used in the vehicle system, and wherein the software management unit: in the preparation state, downloads test software from a server (96) to the vehicle system, separately from the conventional software, to hardware (51a, 53a, 58, 59) equipped in the vehicle system corresponding to the participating unit; leaves both the conventional software and the test software on the hardware in a state in which one or both of the software can be enabled; and when the participating unit stops the test, enables the conventional software and disables the test software while leaving both software on the hardware.

4. The test system of claim 1, wherein, in the individual stop function in the ready state, when the participating unit executes an operation indicating its intention to participate in the test through an HMI device, the test stop unit stops the test of the participating unit without starting it until a grace period during which the operation indicating its intention to participate can be canceled has elapsed.

5. The test system of claim 1, wherein the test stop unit obtains consent from a specific participating unit that participation in the test cannot be canceled for a specified period of time, and fixes the individual stop function for the specific participating unit in a disabled state for the specified period of time.

6. The test system of claim 1, wherein the one or more vehicles are a plurality of vehicles, the test implementation unit sets an implementation area for the test, and the test stop unit, when a specific vehicle among the plurality of vehicles moves outside the implementation area in the implementation state, individually stops the test of the participation unit corresponding to the specific vehicle as the individual stop function.

7. The test system of claim 1, wherein the one or more vehicles are a plurality of vehicles, and the test stop unit sets an individual stop condition and an overall stop condition for the test, and when a specific vehicle among the plurality of vehicles satisfies the individual stop condition, stops the test of the participation unit corresponding to the specific vehicle individually as the individual stop function, and when the overall stop condition is satisfied, stops the test as a whole as the overall stop function.

8. The test system of claim 7, wherein the condition for stopping the entire test is that the number or percentage of vehicles among the plurality of vehicles that have individually stopped the test is equal to or greater than a predetermined threshold value.

9. The test system of claim 1, wherein the test stop unit sets conditions for an individual stop in the test, and adjusts so that a notification that the test will be stopped is sent to a vehicle user before control of the vehicle that causes the condition for the individual stop to be satisfied is executed.

10. The test system according to claim 1, wherein when the participating unit stops the test, the test stop unit issues an announcement that the test will be stopped at a timing that avoids the vehicle being in motion.

11. The test system of claim 1, wherein the test stop unit provides a user interface (UI1) that enables a vehicle user to set a form for temporarily stopping the test at the participation unit, and determines whether to enable the individual stop function based on the setting in the user interface.

12. The test system of claim 11, wherein the test stop unit issues a reminder notification to remind the vehicle user of the existence of the test when the duration of the temporary stop based on the setting in the user interface exceeds a preset period.

13. The test system of claim 1, wherein the vehicle is a vehicle provided for service to passengers, the test is a test in which the participation unit is a passenger unit based on the passenger, and the test stop unit enables the individual stop function for the passenger unit that has confirmed its intention to refuse the test, and provides the test function to the passenger unit that does not.

14. A test method for testing vehicle systems (2, 2A, 2B) installed in a plurality of vehicles (1, 1A, 1B) capable of participating in public road traffic, comprising: determining, in each of the vehicles, the activation of an individual stop function that stops the test individually on a vehicle-by-vehicle basis using one or more metrics; and determining, based on the activation status of the individual stop function in the plurality of vehicles, the activation of a global stop function that stops the test as a whole.

15. A software management device having at least one processor (57b) for managing software used in a vehicle system (2, 2A, 2B) mounted on one or more vehicles (1, 1A, 1B) capable of participating in public road traffic, in order to test the vehicle system (2, 2A, 2B) mounted on the vehicle, wherein the at least one processor performs the following operations in a test preparation state: downloading test software from a server (96) to hardware (51a, 53a, 58, 59) equipped with the vehicle system, separately from conventional software; leaving both the conventional software and the test software in the hardware in a state in which one or both of the conventional software and the test software can be enabled; and, when the vehicle in which the vehicle system is mounted stops the test, enabling the conventional software and disabling the test software while leaving both software in the hardware.

Images