Test system, test method and software management device

Abstract

The present application relates to a test system, a test method, and a software management device. A management system (MS) as a test system uses one or more vehicles (1) capable of participating in road traffic to test a driving system (2) as a vehicle system mounted on the vehicle (1). The management system (MS) has a test implementation unit (MS1) that sets and implements the test, and a test stop unit (MS2) that has both a separate stop function (SFi) that separately stops the test by participating units and a global stop function (SFw) that globally stops the test in a preparation state or an implementation state of the test.

Description

[0001] Cross-references to related applications

[0002] This application is based on Japanese Patent Application No. 2023-208762, filed on December 11, 2023, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The disclosures in this specification relate to techniques for testing vehicle systems. Background Technology

[0004] Patent document 1 discloses the following technology: testing a vehicle system mounted on a vehicle that participates in road traffic.

[0005] Patent Document 1: US Patent No. 10255168

[0006] However, vehicle systems are becoming increasingly complex each year, and the importance of providing vehicle and vehicle inspection (V&V) procedures for vehicles after they are put into the market is increasing in order to cope with these complex systems. In the testing of vehicle systems using vehicles involved in road traffic, as in Patent Document 1, the exploration of methods for stopping testing may become a new topic from the perspective of the appropriateness and usability of test application. Summary of the Invention

[0007] One of the purposes of this specification is to provide a test system, test method, and software management device that can appropriately stop testing.

[0008] One method disclosed here is a test system that uses one or more vehicles capable of participating in road traffic to test vehicle systems mounted on those vehicles, wherein the system includes:

[0009] The Test Implementation Department sets up and implements tests; and

[0010] The test stop unit has both individual stop functions (allowing individual participants to stop the test individually, and overall stop functions (allowing the entire test to stop) in the test preparation or implementation states.

[0011] This approach allows for the differentiation between individual stop functions for participating units and overall stop functions. Therefore, it enables the appropriate cessation of testing, taking into account both the suitability and usability of the testing process.

[0012] Another method disclosed here is a test method that uses multiple vehicles capable of participating in road traffic to test the vehicle system mounted on the vehicles, which includes the following steps:

[0013] In each vehicle, more than one metric is used to determine the effectiveness of the individual stop function, which is tested individually on a vehicle-by-vehicle basis.

[0014] Based on the effectiveness of individual stop functions in multiple vehicles, the effectiveness of the overall stop function in the overall stop test was determined.

[0015] Based on this approach, metrics are used within a single vehicle unit to determine whether to stop testing individually, while the overall testing is stopped based on the condition of multiple vehicles. By making this phased decision to stop the function, testing can be stopped in an appropriate manner.

[0016] Another approach disclosed herein is to have at least one processor, and a software management device for managing the software used in the vehicle system in order to test the vehicle system mounted on the vehicle using one or more vehicles capable of participating in road traffic, wherein...

[0017] The at least one processor performs the following processing:

[0018] In the test preparation state, in addition to the usual software, the test software is downloaded from the server to the vehicle system from the hardware available in the vehicle system;

[0019] To ensure that one or both of the existing software and the test software are effective, the software of both should remain on the hardware; and

[0020] When the vehicle equipped with the vehicle system stops testing, the software of both parties is retained in the hardware, the previous software is made valid, and the test software is made invalid.

[0021] In this way, when the vehicle stops testing, the test software remains in the hardware of the vehicle system. Therefore, restarting the test becomes easy. Furthermore, since the software state during the test is saved, it is easy to verify the test afterwards. Thus, testing can be stopped in an appropriate manner.

[0022] Furthermore, the symbols enclosed in parentheses in the claims and the like illustratively indicate a correspondence with parts of the embodiments described later, and are not intended to limit the scope of the technology. Attached Figure Description

[0023] Figure 1 This is a diagram illustrating an example of the hardware structure of a driving system, etc.

[0024] Figure 2 It is a diagram showing the functional structure of the driving system.

[0025] Figure 3 This is a diagram illustrating an example of the structure of a cockpit.

[0026] Figure 4 It is a diagram that shows an overview of the management system.

[0027] Figure 5 It is a diagram representing the functional structure of a management system.

[0028] Figure 6 This is a flowchart illustrating the processes related to testing.

[0029] Figure 7 This is a flowchart illustrating the processes related to testing.

[0030] Figure 8 This is a flowchart illustrating the processes related to testing.

[0031] Figure 9 This is a flowchart illustrating the processes related to testing.

[0032] Figure 10 This is a flowchart illustrating the processes related to testing.

[0033] Figure 11 This is a flowchart illustrating the processes related to testing.

[0034] Figure 12 This is a diagram illustrating an example of a notification using an HMI.

[0035] Figure 13 This is a flowchart illustrating the processes related to testing.

[0036] Figure 14 This is a diagram illustrating an example of a notification using an HMI.

[0037] Figure 15 This is a diagram illustrating an example of a notification using an HMI.

[0038] Figure 16 This is a flowchart illustrating the processes related to testing.

[0039] Figure 17 This is a flowchart illustrating the processes related to testing.

[0040] Figure 18 This is a diagram illustrating an example of the hardware architecture of a processing system.

[0041] Figure 19 This is a diagram illustrating an example of the hardware architecture of a processing system. Detailed Implementation

[0042] Hereinafter, several embodiments are described based on the accompanying drawings. It should be noted that in each embodiment, repeated descriptions are sometimes omitted by using the same symbols to denote corresponding constituent elements. In each embodiment, where only a portion of the structure is described, structures from other previously described embodiments can be applied to the remaining parts of that structure. Furthermore, not only combinations of structures explicitly shown in the descriptions of each embodiment are possible, but structures from multiple embodiments can also be partially combined with each other, even if not explicitly shown, as long as they do not particularly hinder combination.

[0043] In the following embodiments, the contents of "SafetyFirst for Automated Driving" Tech.Rep., 2019, by Aptiv, Audi, Baidu, BMW, Continental, Daimler, FCA, Infineon, Intel, and Volkswagen, ISO 21448:2022, and IEEE 2846-2022 are incorporated herein by reference in their entirety.

[0044] (Explanation of terminology)

[0045] The following describes the terms relevant to the disclosure in this specification. This description is included in the embodiments of the specification.

[0046] A road user can be a traffic participant on or adjacent to an activity road with the purpose of moving from one place to another.

[0047] Dynamic driving task (DDT) can be any real-time operational and tactical function used to operate a vehicle in traffic. Alternatively, dynamic driving task can encompass all real-time operational and tactical functions used to operate a vehicle on a road.

[0048] An automated driving system (ADS) can be a complete set of hardware and software that can continuously execute the overall DDT regardless of whether it is confined to a specific operating design domain.

[0049] An ADS feature can be an inherent feature of the design of an ADS within a specific ODD under a given level of autonomous driving.

[0050] DDT fallback can be a response by the driver or automated system to either execute DDT or transition to a minimum-risk state after a malfunction occurs, a functional deficiency is detected, or a potentially hazardous behavior is detected. Alternatively, DDT fallback can be a method of transferring control from autonomy to driver or other system control using takeover / fallback states and related use cases. Furthermore, DDT fallback can be a user-initiated response to execute DDT or achieve MRC after a system malfunction related to DDT performance or an ODD deviation, or an ADS-initiated response to achieve MRC under similar circumstances.

[0051] Minimum Risk Condition (MRC) can be the vehicle's state designed to mitigate the risk of failing to complete a specified journey. Additionally, Minimum Risk Condition can be the stable stopping state of the vehicle after a DDT rollback is executed, designed to reduce the risk of accidents in situations where a specified journey cannot or should not continue.

[0052] An operational design domain (ODD) can be the specific conditions under which a (autonomous) driving system is designed to function. Additionally, an operational design domain can be the specific operational conditions under which an ADS (Autonomous Driving System) or its functions are specifically designed to operate, including, but not limited to, environmental, geographical, time-related limitations, and / or the presence or absence of certain traffic / road characteristics.

[0053] The safety of the intended functionality (SOTIF) can be defined as the absence of unreasonable risks arising from the inadequacy of the intended functionality or its implementation.

[0054] Driving policy can be a strategy and set of rules that define vehicle-level control actions.

[0055] A situation can be a factor that may affect the behavior of a system, including traffic conditions, weather, and the behavior of the vehicle itself.

[0056] A scenario can be a description of the temporal relationships between several scenarios within a series of scenarios that include objectives and values ​​under specific conditions influenced by actions and events. Alternatively, a scenario can be a description of the continuous temporal sequence of activities that unify the vehicle, its external environment, and their interactions within the process of performing a specific driving task.

[0057] A safety-relevant object can be any dynamic or static object that may be related to the safety performance of DDT.

[0058] Reasonably foreseeable can be technically credible and have a credible or measurable incidence rate.

[0059] A triggering condition can be a subsequent system response, a specific condition that functions as the initiation of a response to an indirect misuse of behavior that is unpreventable, undetectable, and unmitigable, and which is reasonably foreseeable.

[0060] Minimal Risk Maneuver (MRM) can be the movement of a vehicle instructed by an automated driving system during DDT rollback in order to achieve MRC.

[0061] Risk acceptance criteria (criterion) are benchmarks that indicate there is no unreasonable level of risk. For example, they could be physical parameters that define when a particular behavior is considered an opinion behavior, the maximum number of incidents per hour, or the lowest level that is reasonably feasible.

[0062] A proper response is an important action taken to avoid or mitigate a hazardous situation in a scenario where other safety-related objects are acting within a reasonable and foreseeable range.

[0063] A safety-related model can be a representation of the safety-related aspects of driving actions based on assumptions about the reasonably foreseeable behavior of other road users. A safety-related model can also be an onboard or offboard safety verification or analysis device, a mathematical model, a more conceptual set of rules, a scenario-based set of behaviors, or a combination of these.

[0064] A formal model is a model that is represented in a formal way to verify the performance of a system.

[0065] A safety envelope is a set of constraints and conditions by which a (autonomous) driving system is designed to act as a constraint or control in order to keep operations within an acceptable level of risk. A safety envelope can be a general concept applicable to all principles that a corresponding driving strategy can follow, according to which the vehicle operated by the (autonomous) driving system can have one or more boundaries around it.

[0066] Verification can be an activity used to determine whether a vehicle equipped with an autonomous driving system has achieved the safety of a defined autonomous driving system application in the intended environment.

[0067] Validation can be an activity used to determine whether an object being inspected meets specified requirements.

[0068] A positive risk balance can be a benchmark for demonstrating that a tolerable level of residual risk has been achieved through technical solutions.

[0069] OEDR (object and event detection and response) can be a subtask that includes monitoring the driving environment and performing appropriate responses (DDT) to such objects and events.

[0070] Reaction time can be the time required for a road user to detect a specific stimulus in a given scenario and begin to execute a response (braking, steering, accelerating, stopping, etc.).

[0071] (First Implementation)

[0072] Driving System

[0073] Figure 1 The driving system 2 of the first embodiment shown implements functions related to driving the vehicle 1. The driving system 2 may be the vehicle system itself or a component forming part of the vehicle system. Part or all of the driving system 2 is mounted on the vehicle 1. This vehicle 1 is sometimes referred to as the main vehicle, the primary vehicle, etc. The vehicle 1 may be configured to communicate directly or indirectly with other vehicles, etc., via communication infrastructure. Other vehicles are sometimes referred to as target vehicles.

[0074] Vehicle 1 can be, for example, a four-wheeled car or truck, a road user capable of manual driving. Vehicle 1 can also perform autonomous driving. Autonomous driving can also be referred to as autonomous driving performed by driving system 2. Driving is classified according to the extent performed by the human driver in all dynamic driving tasks (DDTs). Automation levels are specified, for example, by SAE J3016. In levels 0 to 2, the driver performs part or all of the DDT. Levels 0 to 2 can also be classified as so-called manual driving. Level 0 indicates that driving is not automated. Level 1 indicates that driving system 2 assists the driver. Level 2 indicates that driving is partially automated.

[0075] At Level 3 and above, during the operation of the ADS feature, Driving System 2 performs all of DDT. Levels 3-5 can also be classified as so-called automated driving. Systems capable of performing Level 3 and above driving can also be called automated driving systems (ADS). Vehicles equipped with automated driving systems or capable of performing Level 3 and above driving can also be called automated vehicles (AVs).

[0076] Level 3 indicates conditional automation of driving. Level 3 automated driving systems perform DDT (Driver-Driven Technology) but not DDT fallback. That is, DDT fallback is performed by the driver after preparation for fallback. Level 4 indicates high automation of driving. Level 4 automated driving systems perform both DDT and DDT fallback. After reaching the minimum risk condition (MRC) through DDT fallback, etc., Level 4 automated driving systems allow the driver to take over DDT. Level 5 indicates full automation of driving. The handover of DDT between the driving system and the human driver is also known as a transfer of authority.

[0077] The conditions for performing Level 3 or 4 automated driving can include some or all of the conditions represented by the Operational Design Domain (ODD). For example, the ADS function can be defined within the scope of the ODD. The driving system 2 described in this embodiment is a driving system capable of performing Level 3 or higher automated driving.

[0078] Driving system 2 provides functions such as autonomous driving to vehicle users of vehicle 1 that can participate in highway traffic. For example, a vehicle user can be a driver riding in vehicle 1. A vehicle user can be a passenger riding in vehicle 1. For example, if vehicle 1 is a POV (Personally Owned Vehicle), a vehicle user can be the owner of vehicle 1. For example, if vehicle 1 is used for MaaS (Mobility as a Service), a vehicle user can be an operations manager who manages the operation of vehicle 1.

[0079] The architecture of Driving System 2 is chosen to achieve the safety of the intended functionality (SOTIF) process effectively. For example, the architecture of Driving System 2 can be based on a sense-plan-act model. The sense-plan-act model, as the main system element, includes sense, plan, and act elements. These elements interact with each other. Here, sense can be replaced by perception, plan by determination, and act by control.

[0080] In such a driving system 2, from a technical perspective (in other words, from a technical point of view), at least a plurality of sensors 40 corresponding to the detection function, at least one processing system 50 corresponding to the planning function, and a plurality of motion actuators 60 corresponding to the action function are installed. From a functional perspective (in other words, from a functional point of view), detection function, planning function, and action function (also refer to...) are installed. Figure 2 ).

[0081] Specifically, a detection unit 10, serving as a processing unit for implementing detection functions, can be constructed within the driving system 2, primarily consisting of multiple sensors 40, a processing system 50 for processing the detection information from the multiple sensors 40, and a processing system 50 for generating an environment model based on the information from the multiple sensors 40. A planning unit 20 and a risk assessment unit 26, serving as processing units for implementing planning functions, can be constructed within the driving system 2, primarily consisting of multiple motion actuators 60 and at least one processing system 50 that outputs action signals from the multiple motion actuators 60.

[0082] Here, the detection unit 10 can also be implemented as a detection system that can be distinguished from the planning unit 20 and the action unit 30. The planning unit 20 can also be implemented as a planning system that can be distinguished from the detection unit 10 and the action unit 30. The planning system may include a risk assessment function. The risk assessment function can be independently mounted on the driving system 2, separate from the detection unit 10, the planning unit 20, and the action unit 30. The action unit 30 can also be implemented as an action system that can be distinguished from the detection unit 10 and the planning unit 20. The detection system, the planning system, and the action system can constitute independent components. The term "subsystem" here can also be replaced with modules, units, devices, etc.

[0083] The detection unit 10 is responsible for detecting the location (e.g., position estimation) of road users, including vehicle 1 and other vehicles. The detection unit 10 detects the external environment, internal environment, and vehicle status of vehicle 1, and consequently detects the status of the driving system 2. The detection unit 10 fuses the detected information to generate an environment model, which can also be called a world model. The planning unit 20 applies its purpose and driving policy to the environment model generated by the detection unit 10, deriving control actions. The action unit 30 executes the control actions derived by the planning unit 20.

[0084] <Physical Architecture>

[0085] use Figure 1 An example of the physical architecture in driving system 2 will be described below. Driving system 2 includes multiple sensors 40, multiple motion actuators 60, multiple HMI devices 70, and at least one processing system 50, etc. These components can communicate with each other via one or both of wireless and wired connections. These components can also communicate with each other, for example, via an in-vehicle network based on CAN (registered trademark).

[0086] The plurality of sensors 40 includes one or more external environment sensors 41. Furthermore, the plurality of sensors 40 may also include at least one of one or more internal environment sensors 42, one or more communication systems 43, and a map database 44.

[0087] External environment sensor 41 can also detect objects present in the external environment of vehicle 1. Object detection type external environment sensor 41 includes, for example, cameras, LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging), lidar, millimeter-wave radar, ultrasonic sonar, etc. Typically, multiple types of external environment sensors 41 can be combined to monitor the front, sides, and rear of vehicle 1.

[0088] Furthermore, the external environment sensor 41 can also detect the state of the atmosphere and the weather in the external environment of the vehicle 1. Examples of external environment sensors 41 that detect these conditions include external air temperature sensors, temperature sensors, and rain sensors.

[0089] The interior environment sensor 42 can also detect specific physical quantities (hereinafter, motion physical quantities) related to vehicle motion within the interior environment of vehicle 1. Examples of interior environment sensors 42 that detect motion physical quantities include speed sensors, acceleration sensors, and gyroscope sensors. The interior environment sensor 42 can also detect the state of the occupants within the interior environment of vehicle 1. Examples of interior environment sensors 42 that detect occupants include actuator sensors, sensors and systems for monitoring vehicle users (e.g., drivers), biosensors, seating sensors, and in-vehicle equipment sensors. Here, actuator sensors, in particular, detect the occupant's operational state on the motion actuator 60 related to the motion control of vehicle 1, such as acceleration sensors, braking sensors, and steering control sensors.

[0090] The communication system 43 acquires communication data that can be used in the driving system 2 via wireless communication. The communication system 43 can also receive positioning signals from GNSS (global navigation satellite system) satellites existing in the external environment of the vehicle 1. Positioning-type communication devices in the communication system 43 may include, for example, a GNSS receiver.

[0091] Communication system 43 can also send and receive communication signals with external systems (e.g., server 96) existing in the external environment of vehicle 1. V2X type communication devices in communication system 43 include, for example, DSRC (dedicated short-range communications) communicators, cellular V2X (C-V2X) communicators, etc. Examples of V2X communication with external systems existing in the external environment of vehicle 1 include communication with other vehicles' communication systems (V2V), communication with infrastructure equipment such as communicators installed in traffic lights or roadside units (V2I), communication with pedestrian mobile terminals (V2P), and communication with networks such as cloud servers (V2N). The architecture of V2X communication including V2I communication can adopt the architecture specified by ISO 21217, ETSI TS 102 940-943, IEEE 1609, etc.

[0092] Furthermore, the communication system 43 can also send and receive communication signals with the interior environment of the vehicle 1, such as a mobile terminal 91 such as a smartphone located inside the vehicle. The communication devices used in the communication system 43 for terminal communication types include, for example, Bluetooth devices, Wi-Fi devices, and infrared communication devices. Additionally, if the vehicle user's mobile terminal 91 has been pre-associated with the vehicle 1, the communication system 43 can also send and receive communication signals with that mobile terminal located in the external environment.

[0093] Map DB44 is a database storing map data that can be used in driving system 2. Map DB44 is configured to include at least one non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media. Map DB44 may include a database of navigation units that navigate the driving path to the destination of vehicle 1. Map DB44 may include a database of PD maps generated using probe data (PD) collected from each vehicle. Map DB44 may include a database of high-precision maps with a high level of accuracy primarily used for applications in autonomous driving systems. Map DB44 may include a database of parking lot maps containing detailed parking information, such as parking space information, used for applications of automatic parking or parking assistance.

[0094] The map DB44, suitable for the driving system 2, obtains and stores the latest map data, for example, through communication with a map server via a V2X-type communication system 43. The map data, representing the external environment of the vehicle 1, is digitized in a two-dimensional or three-dimensional manner. The map data may also include, for example, road data representing at least one of the following: the location coordinates, shape, road surface condition, and standard driving route. The map data may also include, for example, labeling data representing at least one of the following: the location coordinates and shape of road signs, road markings, and lane lines attached to the road. The labeling data included in the map data may also represent, for example, landmarks such as traffic signs, arrows, lane markings, stop lines, directional signs, landmark beacons, commercial signs, and road alignment changes. The map data may also include, for example, structure data representing, for example, the location coordinates and shape of at least one of the following: buildings and traffic lights facing the road. The labeling data included in the map data may also represent, for example, landmarks such as streetlights, road edges, reflectors, and poles.

[0095] The motion actuator 60 can control vehicle movement based on an input control signal. The drive-type motion actuator 60 is, for example, a powertrain including at least one of an internal combustion engine, a drive motor, etc. The braking-type motion actuator 60 is, for example, a brake actuator. The steering-type motion actuator 60 is, for example, a steering wheel.

[0096] like Figure 3 As shown, multiple HMI (Human Machine Interface) devices 70 can also be mounted on the vehicle 1. The HMI devices 70 enable human-machine interaction between the user of the vehicle 1 and the driving system 2. The portion of the multiple HMI devices 70 that implements occupant input functions can be part of the detection unit 10. The portion of the multiple HMI devices 70 that implements information prompting functions can be part of the action unit 30. On the other hand, the functions implemented by the HMI devices 70 can also be positioned as functions independent of detection, planning, and action functions.

[0097] HMI device 70 can be an operation input device 70a that can input operations performed by the user and is used to transmit the user's intentions or goals from vehicle 1 to driving system 2. Operation input type HMI device 70, i.e., operation input device 70a, includes, for example, an accelerator pedal, brake pedal, gear shift lever, steering wheel, turn signal stalk, mechanical switch, touch panel of navigation unit, etc. Accelerator pedal control serves as the powertrain of motion actuator 60. Brake pedal control serves as the brake actuator of motion actuator 60. Steering wheel control serves as the steering actuator of motion actuator 60.

[0098] HMI device 70 can be an information prompting device 70b that provides visual, auditory, and tactile information to the user of vehicle 1. Visual information prompting type HMI devices 70, i.e., information prompting devices 70b, include, for example, instrument display 70b1, navigation unit, CID (center information display) 70b2, HUD (head-up display) 70b3, lighting unit, etc.

[0099] The instrument display 70b1 is, for example, a display device located in the dashboard, opposite the driver's seat. The instrument display 70b1 displays information necessary for driving, centered on the vehicle's status, including the vehicle's speed. The instrument display 70b1 can be a graphical instrument that displays all information as images, or a combination instrument that combines image display and instrument-based analog display.

[0100] CID70b2 is, for example, a display device located in the center of the dashboard. Among in-vehicle display devices mounted on the dashboard, CID70b2 has the largest display screen. CID70b2 can display images not only to the driver but also to passengers. CID70b2 may include a touch panel that vehicle users can operate; in this case, CID70b2 also functions as an input device 70a.

[0101] HUD70b3 is a display device located in the dashboard, on the side opposite to the instrument display 70b1 across the driver's seat, i.e., inside the driver's seat. HUD70b3 can project an image onto the windshield of the vehicle 1, thereby displaying to the driver a virtual image VI that is visually perceived as appearing outside the vehicle.

[0102] Alternatively, a pillar-to-pillar display can be used instead of CID70b2 and instrument display 70b1, arranged across the left and right A-pillars. Even in this case, the pillar-to-pillar display can be divided into several screens, each controlled in the same way as CID70b2 and instrument display 70b1.

[0103] HMI devices 70 that provide auditory information prompts include, for example, speakers and buzzers. HMI devices 70 that provide tactile information prompts include, for example, steering wheel vibration units, driver's seat vibration units, steering wheel reaction force units, accelerator pedal reaction force units, brake pedal reaction force units, and air conditioning units.

[0104] Furthermore, the HMI device 70 can also communicate with a mobile terminal 91, such as a smartphone, via the communication system 43, thereby enabling HMI functions in cooperation with that terminal. For example, as an alternative to providing information to the HMI device 70, information from the driving system 2 can be displayed on the screen of the vehicle user's smartphone via the communication system 43. On the other hand, the HMI device 70 can also provide information obtained from the smartphone to the user. Additionally, for example, operation input to the smartphone can also be used as an alternative to operation input to the HMI device 70.

[0105] Furthermore, the HMI device 70 can also function as an operation input device 70a, including a feedback device 70a1 for receiving feedback from vehicle users. For example, the feedback device 70a1 may include a computer and a microphone, and when the feedback function is selected from CID 70b2, the microphone records the vehicle user's voice for a specified period (e.g., 45 seconds). This allows the vehicle user to provide feedback such as praise or dissatisfaction regarding vehicle 1 and even the driving system 2. The feedback device 70a1 can then transmit the recorded vehicle user's voice to an external system located in the external environment via communication system 43. This external system could also be a server 96, described later. By aggregating vehicle user feedback in an external system, improvements to vehicle 1 and even the driving system 2 can be achieved.

[0106] The processing system 50 includes at least one component. For example, the processing system 50 may be an integrated processing system that comprehensively performs processing related to detection functions, planning functions, and action functions. In this case, the integrated processing system 50 may further perform processing related to the HMI device 70, and a separate HMI-specific processing system may also be provided. For example, the HMI-specific processing system may be an integrated cockpit system that comprehensively performs processing related to each HMI device 70. The processing system 50 may also be provided via an onboard platform that can be universally used in AV systems.

[0107] Alternatively, for example, the processing system 50 may also have a structure having at least one processing unit corresponding to the processing related to the detection function, at least one processing unit corresponding to the processing related to the planning function, and at least one processing unit corresponding to the processing related to the action function.

[0108] The processing system 50 has an external communication interface, such as via at least one of LAN (Local Area Network), wiring harness, internal bus and wireless communication circuit, to connect to at least one element of each sensor 40, motion actuator 60 and HMI device 70 that is associated with processing based on the processing system 50.

[0109] The processing system 50 is configured to include at least one dedicated computer 51. The processing system 50 may also combine multiple dedicated computers 51 to realize functions such as detection, planning, and action.

[0110] For example, the dedicated computer 51 constituting the processing system 50 may be an integrated ECU that integrates the driving functions of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may also be a determination ECU that determines DDT (Discharge Delay). The dedicated computer 51 constituting the processing system 50 may also be a monitoring ECU that monitors the driving of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may also be an evaluation ECU that evaluates the driving of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may also be a navigation ECU that navigates the driving path of the vehicle 1.

[0111] Alternatively, the dedicated computer 51 constituting the processing system 50 may also be a positioner ECU that estimates the position of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may also be an image processing ECU that processes image data detected by the external environment sensor 41. The dedicated computer 51 constituting the processing system 50 may also be an actuator ECU that controls the motion actuator 60 of the vehicle 1. The dedicated computer 51 constituting the processing system 50 may also be an HCU (HMI Control Unit) that comprehensively controls the HMI device 70. The dedicated computer 51 constituting the processing system 50 may also be at least one external computer, for example, installed in an external center or mobile terminal 91 capable of communication via the communication system 43.

[0112] The dedicated computer 51 constituting the processing system 50 has at least one memory 51a and one processor 51b. The memory 51a may be at least one non-temporary physical storage medium, such as semiconductor memory, magnetic media, and optical media, that non-temporarily stores computer programs and data that can be read by the processor 51b. Furthermore, the memory 51a may be configured with a rewritable, volatile storage medium, such as RAM (Random Access Memory). The processor 51b may include at least one of the following as its core: CPU (Central Processing Unit), GPU (Graphics Processing Unit), and RISC (Reduced Instruction Set Computer)-CPU.

[0113] The dedicated computer 51 constituting the processing system 50 may also be a SoC (System on a Chip) that integrates memory 51a, processor 51b and interface on a single chip, or may have at least one SoC as a component of the dedicated computer 51.

[0114] Furthermore, the processing system 50 may also include at least one database for performing DDT. The database may also be configured to include, for example, at least one non-transitory real storage medium such as semiconductor memory, magnetic media, and optical media, and an interface for accessing the storage medium.

[0115] The database may be a scenario database (hereinafter, Scenario DB) 59. The database may be a rule database (hereinafter, Rule DB) 58. At least one of Scenario DB 59 and Rule DB 58 may also be located independently in the driving system 2, rather than in the processing system 50. At least one of Scenario DB 59 and Rule DB 58 may also be located in an external system existing in the external environment, configured to be accessible from the processing system 50 via the communication system 43.

[0116] Scenario DB59 has a directory storing multiple scenarios for driving vehicle 1. Driving system 2, for example, can match the situation of vehicle 1 with one or more scenarios selected from the multiple scenarios. Scenario DB59 can store multiple scenarios including at least one of functional scenarios, logical scenarios, and concrete scenarios. Functional scenarios define the highest-level qualitative scenario structure. Logical scenarios assign quantitative parameter ranges to structured functional scenarios. Concrete scenarios define the boundaries for safety determinations that distinguish between safe and unsafe states.

[0117] Rule DB58 stores a set of rules for driving vehicle 1. The rule set can contain multiple rules. The rule set can further include a priority structure for a series of rules based on their relative importance. The rule set can be a set of guidelines for strategic driving of vehicle 1.

[0118] Multiple rules can include rules based on laws, regulations, or combinations thereof. Multiple rules can include preference-based rules unaffected by laws or regulations. Multiple rules can include rules based on movement behavior derived from past experience. Multiple rules can include rules characterized by the movement environment. Multiple rules can include rules based on ethical concerns. Multiple rules can include rules based on the fundamental principles of safety models (e.g., the five principles of the RSS model). Multiple rules can include traffic rules. Traffic rules can be those stipulated in road traffic laws or rules based on national or regional customs.

[0119] The traffic rules and other rules stored in rule DB58 can also be located in the same way as the map information obtained from map DB44, as the information provided to the planning department 20 by the detection department 10 through the detection function.

[0120] Additionally, the processing system 50 may also include at least one recording device 55 for recording at least one of the detection information, planning information, and action information of the driving system 2. The recording device 55 may include at least one high-capacity storage medium 55c. The storage medium 55c may also be at least one non-transitory physical storage medium, such as semiconductor memory, magnetic media, and optical media.

[0121] The storage medium 55c can also be mounted relative to the substrate in a manner that is not easily removable or replaceable, in which case, for example, an eMMC (embedded Multi Media Card) using flash memory can be used. At least one of the storage media 55c can also be removable and replaceable relative to the recording device 55, in which case, for example, an SD card can be used.

[0122] The recording device 55 may also have the function of selecting information to be recorded from detection information, planning information, and action information. In this case, the recording device 55 may also have a dedicated computer.

[0123] The dedicated computer provided in the recording device 55 has at least one memory 55a and one processor 55b. The memory 55a may be at least one non-temporary physical storage medium, such as semiconductor memory, magnetic medium, and optical medium, that non-temporarily stores computer programs and data that can be read by the processor 55b. Furthermore, the memory 55a may be provided with a rewritable volatile storage medium, such as RAM (Random Access Memory). The processor 55b includes at least one of the following as its core: CPU (Central Processing Unit), GPU (Graphics Processing Unit), and RISC (Reduced Instruction Set Computer)-CPU.

[0124] A dedicated computer can also be a System on a Chip (SoC) that integrates memory 55a, processor 55b, and interface into a single chip, or it can have at least one SoC as a component of a dedicated computer.

[0125] The recording device 55 can also access the storage medium 55c and perform recording according to write commands for data from various parts of the driving system 2. The recording device 55 can also determine information flowing into the vehicle network, and access the storage medium 55c and perform recording based on the determination made by the processor 55b located in the recording device 55.

[0126] Alternatively, the recording device 55 may be installed independently in the driving system 2 instead of in the processing system 50. The recording device 55 may also be installed in an external system existing in the external environment, configured to be accessible from the processing system 50 via the communication system 43.

[0127] Furthermore, the processing system 50 may also include at least one risk verification unit 53. The risk verification unit 53 may be an onboard installation as part of the Responsibility Sensitive Safety (RSS) safety model. The risk verification unit 53 may be an onboard verification device for planning functions implemented via a dedicated computer 51. The risk verification unit 53 implements a risk verification unit 26, which performs risk verification functions, through hardware independent of the planning unit 20.

[0128] The risk assessment unit 53 may also be primarily constructed as a dedicated computer having at least one memory 53a and one processor 53b. The memory 53a may be at least one non-temporary physical storage medium, such as semiconductor memory, magnetic media, and optical media, that non-temporarily stores computer programs and data that can be read by the processor 53b. Furthermore, the memory 53a may be a rewritable, volatile storage medium, such as RAM (Random Access Memory). The processor 53b may include at least one of the following as its core: CPU (Central Processing Unit), GPU (Graphics Processing Unit), and RISC (Reduced Instruction Set Computer)-CPU.

[0129] A dedicated computer can also be a System on a Chip (SoC) that integrates memory 53a, processor 53b, and interface into a single chip, or it can have at least one SoC as a component of a dedicated computer.

[0130] Thus, the processing system 50 includes memories 51a, 53a, and 55a storing software. Furthermore, it is configured to enable autonomous driving by having the software act via processors 51b, 53b, and 55b, thereby allowing the transfer of permissions between the system itself and the user. The software here may include the computer program itself used in the driving system 2. The software here may include algorithms within the computer program used in the driving system 2. The software here may include parameters within the computer program used in the driving system 2. The software here may include AI or even learned models used in the driving system 2, such as those implemented through neural networks. Furthermore, the software may include data stored in a database referenced in the processing system 50, data stored in map DB44, etc. A piece of software may correspond to one application, multiple applications, be part of an application, or be software used in multiple applications.

[0131] Furthermore, the processing system 50 may also include at least one software management unit 57. The software management unit 57 implements software management functions. The software management unit 57 may also be referred to as a software management device.

[0132] The software management unit 57 manages various software used in the processing system 50, including the computer 51, risk assessment unit 53, recording device 55, rule database 58, scene database 59, etc. The software management unit 57 can also further manage software external to the processing system 50 and used in the driving system 2. For example, the software management unit 57 can also manage data stored in the map database 44, software used in the drawing process of the information prompting device 70b, and software used in the communication process of the communication system 43.

[0133] Software management can include version control, download and installation processing, and uninstallation. Furthermore, software management can also include software testing.

[0134] To implement software management functions, the software management unit 57 can also be primarily configured as a dedicated computer having at least one memory 57a and one processor 57b. The memory 57a can be at least one non-temporary physical storage medium, such as semiconductor memory, magnetic media, and optical media, that non-temporarily stores computer programs and data that can be read by the processor 57b. Furthermore, the memory 57a can also be a rewritable volatile storage medium, such as RAM (Random Access Memory). The processor 57b may include at least one of the following as its core: CPU (Central Processing Unit), GPU (Graphics Processing Unit), and RISC (Reduced Instruction Set Computer)-CPU.

[0135] A dedicated computer can also be a System on a Chip (SoC) that integrates memory 57a, processor 57b, and interface into a single chip, or it can have at least one SoC as a component of a dedicated computer.

[0136] The above architecture is just an example; various structures can be adopted as the hardware structure of the driving system 2.

[0137] Logical Architecture in Autonomous Driving

[0138] Next, use Figure 2 An example of the logical architecture in driving system 2 will be explained. The explanation here focuses on computer program-based processing performed in autonomous driving at level 3 and above. The detection unit 10 may also include an environment recognition unit 11, a self-position recognition unit 12, and an internal recognition unit 13, serving as a processing unit for implementing sub-functions that further classify the detection function by executing a computer program through the processor 51b.

[0139] The environment recognition unit 11 performs the following functions: it independently processes information (sometimes referred to as sensor data) obtained from various sensors 40 related to the external environment, and identifies the external environment, including landmarks, other road users, etc. The environment recognition unit 11 independently processes detection data detected by each external environment sensor 41. Detection data may be, for example, detection data provided by millimeter-wave radar, sonar, LiDAR, etc. The environment recognition unit 11 can also generate relative position data, including the direction, size, and distance of objects relative to the vehicle 1, based on the raw data detected by the external environment sensors 41.

[0140] Additionally, the detection data can be image data provided by a camera, LiDAR, etc. The environment recognition unit 11 processes the image data and extracts objects within the viewpoint reflected in the image. Object extraction may include estimation of the object's orientation, size, and distance relative to the vehicle 1. Furthermore, object extraction may also include, for example, object classification using semantic segmentation.

[0141] Furthermore, the environmental identification unit 11 processes information obtained via the V2X function of the communication system 43. The environmental identification unit 11 also processes information obtained from the map DB44.

[0142] The environmental identification unit 11 can be further classified into multiple sensor identification units optimized for each sensor group. When a sensor identification unit establishes a correspondence by identifying information from a sensor group, it can also fuse information from a sensor group.

[0143] The self-positioning identification unit 12 performs vehicle 1 localization. The self-positioning identification unit 12 obtains global position data of vehicle 1 from the communication system 43 (e.g., a GNSS receiver). Furthermore, the self-positioning identification unit 12 can also obtain position information of objects extracted by the environment identification unit 11. Additionally, the self-positioning identification unit 12 obtains map information from the map database 44. The self-positioning identification unit 12 integrates this information to estimate the position of vehicle 1 on the map.

[0144] The internal identification unit 13 processes the detection data detected by each internal environment sensor 42 and identifies the vehicle status. The vehicle status may include the status of the physical quantities of motion of the vehicle 1 detected by the speed sensor, acceleration sensor, gyroscope sensor, etc. In addition, the vehicle status may include at least one of the user status, the user's operation status of the motion actuator 60, and the on / off status of the HMI device 70.

[0145] The planning unit 20 may also include a prediction unit 21, a driving planning unit 22, and a mode management unit 23, which serve as a processing unit for implementing further classified planning functions by executing computer programs through processors 51b and 53b.

[0146] The prediction unit 21 acquires information about the external environment identified by the environment recognition unit 11 and its own position recognition unit 12, and the vehicle status identified by the internal recognition unit 13. Based on the acquired information, the prediction unit 21 can also interpret the environment and estimate the current situation of the vehicle 1. This situation can be the operational situation, or it may include aspects of the operational situation.

[0147] The prediction unit 21 can also interpret the environment and predict the actions of other road users and other objects. These objects can be safety-relevant objects. The prediction of actions can include at least one of the following: prediction of the object's speed, prediction of the object's acceleration, and prediction of the object's trajectory. The prediction of actions can be performed based on reasonably foreseeable assumptions. Furthermore, the prediction unit 21 can also estimate the user's intention based on the predicted actions, predicted potential hazards, and the obtained vehicle status.

[0148] The driving planning unit 22 plans autonomous driving of the vehicle 1 based on at least one of the following: the estimated position information of the vehicle 1 on the map by the position recognition unit 12, the prediction information and user intention estimation information by the prediction unit 21, and the functional constraint information of the mode management unit 23.

[0149] The driving planning unit 22 implements route planning, behavior planning, and trajectory planning functions. The route planning function plans at least one of the following based on estimated information about the vehicle 1's position on the map: a route to the destination and lane planning at a medium distance. The route planning function may further include the function of determining at least one of a lane change request and a deceleration request based on lane planning at a medium distance. Here, the route planning function can be a task / route planning function within a strategic function, or it can be a function that outputs both task planning and route planning.

[0150] The behavior planning function is a function that plans the behavior of vehicle 1 based on at least one of the following: the route to the destination planned by the route planning function, lane planning for medium distances, lane change requests and deceleration requests, prediction information and user intent estimation information from the prediction unit 21, and functional constraint information from the mode management unit 23. The behavior planning function may include a function that generates conditions related to the state transitions of vehicle 1. Conditions related to the state transitions of vehicle 1 may also be equivalent to triggering conditions. Conditions related to state transitions may also include rollback conditions for performing DDT rollback.

[0151] The behavior planning function can also include the function of determining the state transition of the application implementing DDT based on this condition, and further determining the state transition of driving action. Thus, the driving planning unit 22 plans the execution of DDT rollback. When this does not involve a transfer of permissions, the driving planning unit 22 can further execute the Minimum Risk Maneuver (MRM) together with the motion control unit 31, causing the vehicle 1 to transition to the minimum risk state. The planning of MRM can be implemented through the behavior planning function or the trajectory planning function.

[0152] Additionally, the behavior planning function can also include the ability to determine the longitudinal constraints and lateral constraints related to the path of vehicle 1 based on this state transition information. The behavior planning function can be the tactical behavior planning within the DDT function, or it can be a function that outputs tactical behaviors.

[0153] The trajectory planning function plans the driving trajectory of vehicle 1 based on the judgment information of prediction unit 21, longitudinal constraints related to the path of vehicle 1, and lateral constraints related to the path of vehicle 1. The trajectory planning function may include the function of generating path plans. Path planning may include speed planning, which can also be generated as a plan independent of path planning. The trajectory planning function may include the function of generating multiple path plans and selecting the best path plan from multiple path plans, or the function of switching path plans. The trajectory planning function may further include the function of generating backup data for the generated path plans. The trajectory planning function can be the trajectory planning function within the DDT function, or it can be the function of outputting trajectory plans.

[0154] The mode management unit 23 monitors the driving system 2 and sets constraints on driving-related functions. The mode management unit 23 can also manage the autonomous driving mode, such as the state of the automation level. Automation level management can include switching between manual and autonomous driving, i.e., the transfer of authority between the user and the driving system 2, in other words, the management of driver takeover. The mode management unit 23 can also monitor the state of subsystems related to the driving system 2 and determine system malfunctions (e.g., errors, unstable operation, system obstacles, malfunctions). The mode management unit 23 can also determine the mode based on the user's intent estimation information generated by the internal recognition unit 13. The mode management unit 23 can also set constraints on driving-related functions based on at least one of the following: the system malfunction determination result, the mode determination result, the vehicle state further identified by the internal recognition unit 13, sensor anomaly (or sensor malfunction) signals output from the sensor 40, the state transition information of the driving planning unit 22's application, and trajectory planning.

[0155] In addition to constraints related to driving, the mode management unit 23 may also comprehensively determine longitudinal constraints and lateral constraints related to the path of vehicle 1. In this case, the driving planning unit 22 plans the behavior and trajectory according to the constraints determined by the mode management unit 23.

[0156] In addition, if the risk confirmation function is installed as part of the planning department 20, the risk confirmation function can also be installed as part of the functions implemented by the prediction department 21, the driving planning department 22, and the mode management department 23.

[0157] The risk assessment function obtains environmental models and sensor data from the detection unit 10, evaluates risks based on this information, and outputs the response based on the risk to the action unit 30 or the driving planning unit 22. This series of functions or processes can also be referred to as risk assessment or even risk monitoring.

[0158] More specifically, the risk assessment function outputs a status based on information obtained from the detection unit 10. This function determines whether the status is safe or dangerous. This assessment may include confirming the estimated collision risk between vehicle 1 and surrounding objects. In this assessment, uncertainties can be considered, and indicators such as collision probability can be used. The risk assessment function can also compare the permissible collision risk threshold with the estimated collision risk value to determine whether the situation is dangerous. The permissible collision risk threshold can also be preset based on the risk acceptance criteria / criterion, detailed later.

[0159] Based on the risk assessment result, the risk assessment function derives an appropriate response. This appropriate response is provided to the action unit 30 or the driving planning unit 22 and can only be initiated when the situation is determined to be dangerous. The appropriate response can be a restriction on the control commands of the motion actuator 60. Alternatively, the appropriate response can be a response designed to return the vehicle 1 to a safe condition.

[0160] Risk assessment is achieved through the installation of a safety model. This safety model can also be called a safety-related model. It can be a formal model. For example, an RSS model can be used, but other models such as the SFF model, more generalized models, or composite models combining multiple models can also be employed. SFF stands for Safety Force Field. The safety model monitors the risks of vehicle 1 based on driving strategies. In other words, risk monitoring can also be described as monitoring driving strategies.

[0161] In RSS models, longitudinal and lateral safety distances relative to other road users are used, for example, as indicators for confirming collision risk. It can be said that safety distance is an example of a geometric approach such as a safety envelope.

[0162] The action unit 30 may also include a motion control unit 31 and an HMI output unit 71, serving as a processing unit for implementing sub-functions that further categorize action functions by executing computer programs through the processor 51b. The motion control unit 31 controls the movement of the vehicle 1 based on trajectory planning (e.g., path planning and speed planning) obtained from the driving planning unit 22. Specifically, the motion control unit 31 generates acceleration request information, shift request information, braking request information, and steering request information corresponding to the trajectory planning, and outputs them to the motion actuator 60.

[0163] Here, the motion control unit 31 can directly obtain the vehicle state identified by the detection unit 10 (especially the internal recognition unit 13), such as at least one of the current speed, acceleration and yaw rate of the vehicle 1, and reflect it in the motion control of the vehicle 1.

[0164] The HMI output unit 71 outputs HMI-related information based on at least one of the following: prediction information and user intent estimation information from the prediction unit 21, application state transition information and trajectory planning from the driving planning unit 22, and function constraint information from the mode management unit 23. The HMI output unit 71 can also manage vehicle interaction. Based on the management status of vehicle interaction, the HMI output unit 71 can also generate notification requests to control the information prompt function in the HMI device 70. Furthermore, based on the management status of vehicle interaction, the HMI output unit 71 can also generate control requests for windshield wipers, sensor cleaning devices, headlights, and air conditioning devices, and control these devices.

[0165] <Testing in Highway Traffic>

[0166] For the driving system 2 described above, a verification and validation (V&V) process is required. The V&V referred to here can be a V&V for the intended functionality of the software used in driving system 2 or a V&V for SOTIF. Scenarios that vehicle 1 may encounter can be categorized into known hazardous scenarios, known non-hazardous scenarios, unknown hazardous scenarios, and unknown non-hazardous scenarios. The V&V process can be a process of mitigating the risks of known hazardous scenarios and unknown hazardous scenarios within these scenarios.

[0167] In the V&V of Driving System 2, there are verifications for meeting the safety requirements of each technology level and comprehensive verifications for the safety of each element. Verifications for meeting the safety requirements of each technology level may include evaluations of at least one, preferably all, of the following functions and capabilities. Additionally, verifications may include evaluations of other functions and capabilities.

[0168] For example, the evaluation objects of the detection unit 10 are the function of the sensor 40 or external data source (such as map data source), the function of the sensor algorithm that models the environment, and the reliability of the infrastructure and communication system 43.

[0169] For example, the evaluation object associated with the planning department 20 is the ability to determine the algorithm. The ability to determine the algorithm includes the ability to safely handle potential functional deficiencies and the ability to make appropriate decisions based on the environmental model, driving strategy, current destination, etc. Additionally, the evaluation object associated with the planning department 20 may include, for example, one of the following: the absence of unreasonable risks caused by behaviors that endanger the intended function; the functionality of the system for safely handling ODD use cases; the robustness of the execution of the driving strategy within the ODD as a whole; the suitability of DDT rollback; and the suitability of the minimum risk state.

[0170] Furthermore, for example, the evaluation object may include not only the nominal performance of the system or function, but also its robustness. The robustness of the system or function includes the system's robustness under harsh environmental conditions affected by various disturbances, the appropriateness of the system's actions in response to known triggering conditions, the sensitivity of the expected function, and the monitoring capability for various scenarios.

[0171] V&V can also be implemented with the goal of achieving a positive risk balance through autonomous driving performed by Driving System 2. It can be said that positive risk balance is the main measure of the ethically permissible level of risk.

[0172] More specifically, V&V can also be implemented with the goal of achieving a risk tolerance benchmark that can be set based on a positive risk balance. A quantitative benchmark for the risk tolerance benchmark is, for example, that the probability of a hazard occurring is below a threshold. The risk tolerance benchmark can also be set, for example, through a combination of statistical methods such as traffic incident statistics and scenario-based methods.

[0173] When verifying or testing software related to at least one of the behavior planning and trajectory planning performed by the driving planning unit 22, it is necessary to confirm that the driving system 2 performs at least safer actions than either a capable and careful driver or an experienced and attentive driver. When testing software in a virtual environment, a software-in-the-loop (SiL) verification method can be implemented, for example, using a benchmark dataset containing scenarios stored in the scenario database.

[0174] When verifying or testing software related to pattern management performed by Pattern Management Department 23, especially verification or testing related to the determination of ODD, working status and non-working status, it can be implemented through SiL, hardware-in-the-loop (HiL), or by both parties.

[0175] Furthermore, when verifying or testing HMI-related software such as the HMI output unit 71, verification or testing can also be carried out through HiL and driver-in-the-loop (DiL). Tests in DiL can also be conducted on vehicle users who lack prior experience or knowledge related to the driving system 2 and are not accustomed to autonomous driving.

[0176] Thus, in the verification and testing of the software in driving system 2, verification methods such as SiL, HiL, and DiL can be selected according to their objects and purposes. The loops used in SiL, HiL, and DiL can be either open loops or closed loops.

[0177] To manage unacceptable risks and improve driving system 2, a robust management system MS is preferred for driving system 2 of vehicles 1 that participate in road traffic after entering the market. For example, Figure 4 The management system MS shown executes software changes for individual vehicles via OTA (over-the-air) updates. The management system MS is structured as a vehicle population comprising multiple vehicles 1A, 1B, ... and a server 96. Vehicles 1A and 1B managed by the management system MS can have the same structure as vehicle 1 equipped with the aforementioned driving system 2; however, at a minimum, as long as there is compatibility in the software specifications, some hardware specifications such as vehicle category and model can differ.

[0178] During testing in the changeover process, data collected from the driving of each vehicle (1A, 1B) within the vehicle group on public roads can be used. Additionally, safety-related indicators can also be used in the testing during the changeover process. The results of the appropriateness verification using safety-related indicators can be used not only to determine the applicability of the testing software. For example, the results can also be used for setting the ODD itself and setting parameter limitations based on the ODD. These settings can include specification changes made through software updates.

[0179] Additionally, the following may sometimes be distinguished in the description, such as referring to software used in temporary testing as test software and software used in formal (permanent) testing as formal software.

[0180] Furthermore, testing based on this management system MS is not limited to software updates; it can also be conducted to improve the environment in which the driving system 2 is used. That is, test results can be fed back into urban planning. For example, as a result of the test, the shape of roads at intersections, merging points, etc., can be changed to a safer shape. Additionally, as a result of the test, the control of traffic lights at intersections, etc., can be changed to control the frequency of congestion. In this case, the test may not be conducted with the application of test software.

[0181] Safety-related metrics can be values ​​derived from the risk indicators confirmed by the risk assessment unit 26. These risk indicator values ​​may include, for example, the aforementioned risk value, safety envelope, safety distance, and violation metric (violation degree). The violation metric (violation degree) is a value that evaluates the degree to which vehicle 1 (the test vehicle) violates the rules stored in the rule set.

[0182] Safety-related metrics can be safety metrics for autonomous driving systems. Safety metrics can refer to quantifiable scales based on collision rates. Examples of safety metrics include collision severity and frequency, citable offense severity and frequency, longitudinal and lateral distances, longitudinal and lateral accelerations, longitudinal and lateral jerk movements, and OEDR reaction time. Collision severity can be evaluated using the AIS (Abbreviated Injury Scale) on a six-level scale. Longitudinal and lateral distances are metrics related to maintaining the safety envelope and even the safe distance.

[0183] Furthermore, evaluations of safety-related indicators can also be human evaluations. Here, "human" can refer to the vehicle user of the test vehicle. It can also refer to a vehicle user unfamiliar with autonomous driving. In DiL testing, evaluation indicators can be input by the driver as the vehicle user through the test vehicle's operation input device 70a (e.g., feedback device 70a1), or by input through the driver's mobile terminal 91. Additionally, "human" here can also refer to other VRUs, such as pedestrians, who encounter the test vehicle. In this case, the evaluation can also be sent by other VRUs that perceive a danger to the test vehicle using their own mobile terminals 91, etc., to the driving systems 2A and 2B of vehicles 1A and 1B, or even the server 96.

[0184] The management system MS can also use individual vehicle groups to perform tests on software that can be used in driving systems 2A and 2B. The management system MS can also apply software whose suitability has been verified through testing to each driving system 2A and 2B within the individual vehicle group. In this way, the management system MS functions as a test system for testing the suitability of driving systems 2A and 2B and the environment in which they are used.

[0185] like Figure 5 As shown, the management system MS can also be configured with a test implementation unit MS1, a test stop unit MS2, and a software management unit MS3, which serve as processing units implemented through the cooperation of the server 96 and driving systems 2A and 2B.

[0186] The test implementation unit MS1 sets up the test plan in server 96 and conducts the test using the driving systems 2A and 2B of some or all of the vehicles 1A and 1B in the vehicle group. The processing related to the overall test planning is mainly performed by the test management unit 97a of server 96. The processing related to the test implementation is mainly performed by the test management unit 97a, software release unit 97b, data collection unit 97c, and test software evaluation unit 97d of server 96, as well as the participation status management unit 81, software application unit 82, and motion measurement unit 83 of driving systems 2A and 2B.

[0187] The test stop unit MS2, in either the test preparation or execution state, has both an individual stop function SFi (for stopping the test individually by a participating unit) and an overall stop function SFw (for stopping the test as a whole), and executes processing related to both functions. A participating unit in the test can be, for example, a vehicle unit. The term "stop" here refers to the cessation of the test execution state, encompassing both temporary and permanent stops. A locational stop can also be referred to as an interruption, and a permanent stop can be referred to as a termination. Processing related to the individual stop function SFi can be performed primarily by the participation state management unit 81 of driving systems 2A and 2B, with the test management unit 97a of server 96 also potentially participating. Processing related to the overall stop function SFw can be performed primarily by the test management unit 97a of server 96.

[0188] The Software Management Department (MS3) distributes the test software to the test vehicles and manages the use of the software. Based on the activation status of the Individual Stop Function (SFi) and the Overall Stop Function (SFw), the MS3 performs software rewriting and switching. Software management-related processing can be primarily performed by the Software Distribution Department (97b) of Server 96 and the Software Application Department (82) of Driving Systems 2A and 2B.

[0189] <Example of server structure in a management system>

[0190] like Figure 1 , 4 As shown, server 96 is located in the external environment relative to vehicles 1A and 1B. Server 96 can communicate with each vehicle 1A and 1B, for example, via V2X communication through a communication infrastructure. Server 96 can also connect to an operating terminal operated by a human operator, and together with the operating terminal, form a remote management center for managing a group of individual vehicles.

[0191] like Figure 1As shown, the server 96 can also be primarily configured as a dedicated computer having at least one memory 96a and one processor 96b. The memory 96a can be at least one non-transitory physical storage medium, such as semiconductor memory, magnetic media, and optical media, that non-transitorily stores computer programs and data that can be read by the processor 96b. Furthermore, the memory 96a can also be a rewritable, volatile storage medium such as RAM (Random Access Memory). The processor 96b includes, for example, at least one of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and RISC (Reduced Instruction Set Computer) CPU as its core.

[0192] A dedicated computer can also be a System on a Chip (SoC) that integrates memory 96a, processor 96b, and interface into a single chip, or it can have at least one SoC as a component of a dedicated computer.

[0193] Server 96 may further have a management database (hereinafter, management DB) 96c. Management DB 96c may also store information used to determine vehicles 1A and 1B as managed objects of the management system MS. Management DB 96c may also store information related to the specifications of each vehicle 1A and 1B. Management DB 96c may also store various information collected from each vehicle 1A and 1B. Among this various information may be information used for evaluation in the tests described later. Additionally, among this various information may be information indicating whether the test-participating vehicles are undergoing testing or have stopped.

[0194] Server 96 is equipped with improved features based on software deemed suitable for the V&V process in driving systems. For example... Figure 4 As shown, the server 96 may also include a test management unit 97a, a software distribution unit 97b, a data collection unit 97c, and a test software evaluation unit 97d, which serve as a processing unit for executing computer programs through the processor 96b to achieve its functions.

[0195] Test Management Department 97a manages the testing of individual vehicle groups. Alternatively, a test software can be prepared as an improved version of the software used in the individual vehicle group application (hereinafter, prior software) for testing. When two test software programs are prepared and compared, this test is called A / B testing. Multiple test software programs can also be three or more software programs with the same functionality that can be compared with each other.

[0196] The following explanation focuses on one typical example of A / B testing: assigning multiple test software programs to a single test vehicle. However, various methods can be employed as testing approaches; for instance, A / B testing can also be implemented by assigning multiple test software programs to a single test vehicle.

[0197] The test software may be provided, for example, by the administrator of server 96 (hereinafter, the test administrator), who is a human. In this case, the test is conducted with the aim of selecting the best software from the release candidates. On the other hand, if the test is conducted to optimize the parameters used in the computer program, the test software may also be automatically generated by the test management department 97a. For example, the test software may also be automatically generated by changing parameters such as the judgment threshold, upper limit, and lower limit of an existing program.

[0198] The Test Management Department 97a manages the scale and duration of tests. The scale and duration can be set by the test administrator inputting values ​​into the server 96, or they can be automatically set by the Test Management Department 97a. Based on the scale and duration, the Test Management Department 97a determines the suitable test vehicles from vehicles 1A and 1B belonging to the vehicle group. If the number of vehicles registered in the management system MS is less than the optimal scale for test implementation, all vehicles 1A and 1B belonging to the vehicle group can be designated as test vehicles.

[0199] Test Management Department 97a assigns one of multiple test software programs to each test vehicle within a group of vehicles 1A and 1B. In AB testing comparing test software A and test software B, for example, half of the test vehicles are tested with test software A, and the remaining half with test software B. Test Management Department 97a may also use pseudo-random numbers to randomly assign test software to each vehicle 1A and 1B. Alternatively, Test Management Department 97a may refer to vehicle information stored in Management DB96c to reduce the deviation of conditions between groups testing each test software program when assigning test software.

[0200] The Test Management Department 97a can also set restrictions on the content of the tests. For example, it can limit the testing area to a specific country or region. It can also limit the testing time to, for example, daytime only.

[0201] Furthermore, the test management department 97a sets at least one evaluation metric for evaluating the test software. The evaluation metric can also be set based on the functionality and characteristics of the test software or the intent of the test, and can be determined by input operations from the test manager to the server 96. Alternatively, the evaluation metric can also be set by the test management department 97a based on the functionality and characteristics of the test software. The evaluation metric may include security-related metrics.

[0202] Furthermore, the Test Management Department 97a can also manage at least one of the methods for obtaining consent and the content of consent from vehicle users related to the application of test software in each test vehicle. The Test Management Department 97a can also delegate the management of at least one of the methods for obtaining consent and the content of consent to the driving systems 2A and 2B of each test vehicle.

[0203] The consent may include information regarding whether or not participants are permitted to disengage from the test (i.e., cancellation policy). For example, if the test content and its evaluation metrics require continuous testing, the consent may include prohibiting disengagement due to vehicle user reasons throughout the entire test period or for a certain period within the test period. Tests requiring continuous testing include, for example, tests validating the new user interface until the driver becomes accustomed to it, or tests validating the new vehicle controls until the occupants become accustomed to them.

[0204] Where vehicle users can choose whether to prohibit departure from the test due to their own reasons during the test period, incentives can be provided to vehicle users who agree to the prohibition of departure. Incentives could include, for example, upgrading vehicle 1's features, issuing discount coupons for optional features of vehicle 1, granting electronic currency, or awarding points. Upgrading vehicle 1's features could be, for example, unlocking the seat heater function, which is equipped on vehicle 1 but whose functionality is currently frozen, during the test period. Then, at the end of the test, vehicle users could use discount coupons to purchase the seat heater function at a discounted price and use it permanently.

[0205] In the test preparation state, Test Management Department 97a determines whether the test should begin entirely. It can be said that preserving the start of the overall test constitutes the activation of the overall stop function SFw. Furthermore, in the test execution state, Test Management Department 97a determines whether the test should be stopped entirely. It can be said that stopping the overall test constitutes the activation of the overall stop function SFw. Test Management Department 97a can also, if the overall stop function SFw is activated, decide whether it is a permanent or temporary stop. Test Management Department 97a can also, if the overall stop function SFw is activated, and the activated stop is temporary, determine whether to invalidate the overall stop function SFw and restart the test execution.

[0206] In addition to managing the overall stop function SFw, the test management department 97a can also manage the individual stop function SFi. The test management department 97a can also obtain information related to the activation and deactivation of the individual stop function SFi from the driving systems 2A and 2B of each test vehicle, and collect this information. The test management department 97a can also determine whether testing of each test vehicle should be stopped individually, and request control from the driving systems 2A and 2B based on that determination.

[0207] Based on the test software allocation set by the test management department 97a, the software distribution unit 97b distributes the test software to the driving system 2 of the test target vehicle. Along with the distribution of the test software, information related to test planning may also be distributed. This information includes details about the test duration and evaluation metrics, and may also include information about the scale of the test. Thus, the distributed test software is temporarily applied in each driving system 2A and 2B, and the test begins.

[0208] The data collection unit 97c collects information as probe data from each driving system 2A and 2B where the test software has been temporarily applied, including information related to the actions and results of the test software. The information collected may include at least one of safety-related indicators themselves and information used to derive safety-related indicators. The data collection unit 97c accumulates the data collected sequentially from each vehicle 1A and 1B in the management DB96c.

[0209] The Test Software Evaluation Department 97d compares multiple test software programs. The Test Software Evaluation Department 97d compares the test software programs against the evaluation indicators set by the Test Management Department 97a. When multiple evaluation indicators exist, the Test Software Evaluation Department 97d compares each evaluation indicator separately among the test software programs.

[0210] Specifically, the test software evaluation unit 97d performs statistical processing on the data accumulated in the management DB96c from each vehicle 1A and 1B. For example, when the evaluation indicators can be expressed proportionally by the occurrence rate, the test software evaluation unit 97d calculates the occurrence rate for each vehicle and / or per unit time based on the occurrence frequency in the data from each vehicle 1A and 1B.

[0211] Furthermore, when using safety metrics as evaluation indicators, severity potential is preferably considered. When evaluating the severity and frequency of collisions to test software, even if the test software is temporarily applied to multiple vehicles 1A and 1B, it is difficult to make a statistical evaluation if actual collisions are infrequent. Therefore, the test software evaluation unit 97d can, for example, estimate the probability of a severe collision with high severity based on the occurrence rate of appropriate warning signs such as near collisions and low-severity collisions.

[0212] Furthermore, the distribution and sensitivity of collision types may differ between autonomous driving systems and human drivers. Therefore, in the statistical evaluation of safety metrics for test software, the evaluation can also be conducted by classifying vehicle 1 (using test software) as a Level 3 or higher autonomous driving vehicle and vehicle 1 (driven by a human user).

[0213] Furthermore, the test software evaluation department 97d can also select the best test software from multiple test software options. The best test software may be the one with the highest security. The test management department 97a can also decide on the test software selected by the test software evaluation department 97d as the officially adopted software.

[0214] When multiple test software programs are improved versions of previous software currently in use in vehicles 1A and 1B, the test software evaluation unit 97d can also perform a relative evaluation of the selected test software compared to the previous software. The test software evaluation unit 97d can also determine that the selected test software does not have higher performance than the previous software. In this case, the test management unit 97a can also exclude all of the multiple test software programs from the officially adopted software.

[0215] On the other hand, the test software evaluation department 97d may not have the function of selecting the best test software. In this case, the test management department 97a, through the HMI of the server 96, prompts the test manager with the comparison results of the evaluation indicators and accepts the input operation of the test manager to select the official software to be adopted. According to the input operation, the test management department 97a decides on the official software.

[0216] The software distribution unit 97b can also distribute the official software to each vehicle 1A and 1B belonging to the vehicle group, based on the decision of the official software. The vehicles at the distribution destination can include vehicles belonging to the vehicle group that are not test subjects. Furthermore, the software distribution unit 97b can also request the adoption of the selected software or recommend the selected software to other management systems.

[0217] <Example of a driving system structure in a management system>

[0218] Each vehicle 1A and 1B belonging to the vehicle group is equipped with a separate driving system 2A and 2B. In addition to recognition, planning, and action functions, driving systems 2A and 2B are also equipped with software management functions. Driving systems 2A and 2B may also have a participation status management unit 81, a software application unit 82, a motion measurement unit 83, and an HMI cooperation unit 84, which serve as processing units for executing computer programs through the processor 57b of the software management unit 57 to realize functions.

[0219] The Participation Status Management Department 81 manages the test participation status related to vehicles 1A and 1B equipped with various driving systems 2A and 2B. The test participation status can be either currently participating in testing or not participating in testing. Furthermore, the current testing status can be categorized into three states: test in progress, the overall test stop function SFw being activated, or the individual test stop function SFi being activated.

[0220] If the participation status management unit 81 fails to obtain consent from the vehicle user for participation in the test (hereinafter, test consent), it sets the test participation status to a non-participation status. Under this setting, the participation status management unit 81 prohibits vehicles 1A and 1B mounted on it from participating in the test.

[0221] Once the participation status management unit 81 obtains test consent from the vehicle user, it sets the test participation status to "participating in test". Then, the participation status management unit 81 determines whether the overall stop function SFw is activated by obtaining information related to the test from the server 96 via the communication system 43. Additionally, the participation status management unit 81 controls the activation and deactivation of the individual stop functions SFi of its vehicles 1A and 1B, reflecting this in the test participation status.

[0222] The Software Application Department 82 manages the application of software installed on driving systems 2A and 2B. Software application here can include software download and installation, i.e., preparation for making the software usable. Software application management may include defense against external attacks exploiting security vulnerabilities, software privacy management, and software update management. Update management may include software version management.

[0223] Upon receiving update information from server 96 and distributing the official software from server 96, the software application unit 82 permanently applies the official software to driving systems 2A and 2B. "Permanent application" here means application until the next update of the official software.

[0224] Furthermore, the application of management software can also include the management of the application of test software. When vehicles 1A and 1B are selected as test vehicles, the software application unit 82 temporarily applies the test software downloaded from server 96 to driving systems 2A and 2B to begin the verification of the test software.

[0225] The following examples illustrate two methods for applying test software when testing a modified version of prior software already applied to the hardware of driving systems 2A and 2B. Here, the hardware may be, for example, the memory 51a of computer 51, the memory 53a of risk assessment unit 532, etc. When performing tests related to rule sets, scenario structures, etc., the hardware may be rules DB58, scenario DB59, etc.

[0226] The first method involves rewriting the existing software into test software as part of the testing process. This method is suitable when hardware resources for driving systems 2A and 2B are limited. Furthermore, this method allows the test software to be tested under conditions almost identical to actual application.

[0227] In the first method, with the test software already applied, and at least one of the overall stop function SFw and the individual stop function SFi being activated, the software application unit 82 downloads the previous software from the server 96 and restores the test software to the previous software.

[0228] The second method is to retain the existing software and download the new test software, allowing them to coexist. In this case, the hardware storing the test software can be the same as the existing software or other hardware. This other hardware could be the memory 57a of the software management unit 57, or hardware specifically configured for testing (e.g., hardware dedicated to shadow-mode). This method is suitable when the hardware resources of the driving systems 2A and 2B are relatively abundant.

[0229] In the second method, a preferred configuration is that, in the coexistence of conventional software and test software, a software switching function enables one software and disables the other. For example, when both the overall stop function SFw and the individual stop function SFi are disabled, the software application unit 82 disables the conventional software and enables the test software. Alternatively, when at least one of the overall stop function SFw and the individual stop function SFi is enabled, the software application unit 82 enables the conventional software and disables the test software. This software switching function can be easily implemented, for example, by incorporating conditional branch commands such as IF statements into the computer program of the control software.

[0230] Here, driving systems 2A and 2B may also completely refrain from executing the calculations of the invalidated software. On the other hand, driving systems 2A and 2B may execute the calculations of the invalidated software themselves, but prevent the calculation results from being reflected in other functions such as DDT that affect the behavior of vehicles 1A and 1B.

[0231] Testing using Shadow Mode can also be performed by fixing the existing software in an active state and the test software in an inactive state. In this case, the calculation results of the inactive test software may not be reflected in other functions such as DDT that affect the behavior of vehicles 1A and 1B, but are instead stored specifically in the storage medium 55c of the recording device 55 or the management DB96c for the purpose of verifying the test software. The following explanation will continue based on the premise of using the second method.

[0232] The motion measurement unit 83 measures the motion results of the temporarily applied test software. The measurement object is at least one of the evaluation index specified by the server 96 and the data required for calculating the evaluation index. As described above, when the calculation is performed in the invalidated test software, the measurement object may also include the calculation results of the invalidated test software. Depending on the characteristics of the evaluation index, the measurement of motion results may be performed continuously or only under specified conditions based on a pre-set trigger.

[0233] Then, the motion measurement unit 83 sends the measured test results to the server 96. The test results may be sent sequentially during the test or summarized and sent at the end of the test period. The test results include at least one of the following: the participation status set by the participation status management unit 81, the test software application information of the driving systems 2A and 2B by the software application unit 82, and the measurement information measured by the motion measurement unit 83. The test software application information may include the date, time, and period of applying the test software to the driving systems 2A and 2B, the method of applying the test software to the driving systems 2A and 2B, etc. The measurement information is information related to the aforementioned measurement objects.

[0234] Alternatively, the motion measurement unit 83 can also record the test results in the storage medium 55c. The test results can also be recorded in the storage medium 55c along with the logs sent to the server 96.

[0235] Thus, based on the evaluation index information received from the server 96, the motion measurement unit 83 selects the index and data to be measured, and the source for obtaining the index and data, i.e., the measurement object. Then, the motion measurement unit 83 generates the measured index and data as measurement information in a pre-set format. This format can also be specified by the server 96 in the evaluation index information. The generated measurement information, as described above, is sent to the server 96 as probe data, and can also be recorded in the storage medium 55c.

[0236] HMI Collaboration Unit 84 collaborates with HMI Device 70 to perform the aforementioned functions of obtaining consent, providing information prompts related to software management, and receiving feedback from vehicle users.

[0237] The HMI Collaboration Unit 84 can also attempt to obtain test consent from the vehicle user (e.g., the driver) upon request from the Participation Status Management Unit 81. Specifically, the HMI Collaboration Unit 84 uses the information prompting device 70b to notify the vehicle user (e.g., the driver) of the request for test consent. The HMI Collaboration Unit 84 accepts the driver's input to the operation input device 70a corresponding to the notification and confirms the driver's intention to consent. The HMI Collaboration Unit 84 provides the information that consent has been obtained from the driver to the software application unit 82. In addition, the HMI Collaboration Unit 84 performs necessary notifications to the vehicle user at the start, during, and end of the test, accompanying the test. These notifications are easily identifiable to the vehicle user if they are implemented using a display such as CID 70b2 in the HMI device 70.

[0238] In addition, the HMI Collaboration Unit 84 accepts feedback from vehicle users during and at the end of the test. Feedback can be received through input operations on CID70b2 or through voice input from the vehicle user in the feedback device 70a1.

[0239] The following describes several use cases of the overall stop function SFw and the individual stop function SFi based on the management system MS.

[0240] <Examples of how to handle user errors>

[0241] like Figure 6 As shown in the flowchart, the management system MS uses the separate stop function SFi of the test stop unit MS2 to appropriately handle erroneous operations by vehicle users who have expressed their intention to participate in the test. Here, an erroneous operation means mistakenly agreeing to the actions of a vehicle user who has expressed their intention to participate in the test. An action by a vehicle user who has agreed to participate in the test is an example of an action that expresses their intention to participate in the test.

[0242] exist Figure 6In a series of processes within the method, for example, the processor 96b of server 96 executes a computer program stored in memory 96a. Correspondingly, on the driving system 2A, 2B side, the processor 53b of software management unit 57 executes a computer program stored in memory 53a. A trigger for initiating this process can also be provided by inputting an operation to the user interface in the server 96 of the test administrator.

[0243] In the initial S101, server 96 (e.g., test management unit 97a) sets the planned test to a ready state. Based on this, processing for implementing the test begins in server 96 and each driving system 2A, 2B. After processing in S101, proceed to S102. The following will describe the specific driving system 2.

[0244] In S102, the driving system 2 (e.g., involving the status management unit 81 and the HMI collaboration unit 84) attempts to obtain the vehicle user's test consent and determines whether test consent has been obtained. If yes, proceed to S103. If no, proceed to S103.

[0245] In S103, the driving system 2 (e.g., the participation status management unit 81) sets the participation status of vehicle 1 to "participating in the test". After processing in S103, proceed to S104.

[0246] In S104, the driving system 2 (e.g., the participation state management unit 81) activates the individual stop function SFi for the corresponding vehicle 1. Thus, until the individual stop function SFi is deactivated, the vehicle 1 remains in a state where testing has not been initiated. After processing in S104, the process proceeds to S105.

[0247] In S105, the driving system 2 (e.g., the participation status management unit 81) determines whether the preset grace period has elapsed. The grace period is a preset time starting from the time when the vehicle user performs the operation to agree to the test, for example, set to 1 minute. If not, proceed to S106. If yes, proceed to S108.

[0248] In S106, the driving system 2 (e.g., participating in the status management unit 81 and the HMI cooperation unit 84) accepts the vehicle user's consent to cancel the operation. After processing in S106, the process proceeds to S107. In S107, the driving system 2 (e.g., participating in the status management unit 81) determines whether a consented cancellation operation exists. If yes, the process proceeds to S110. If no, the process returns to S105.

[0249] In S108, the driving system 2 (e.g., the participation status management unit 81) disables the individual stop function SFi of the corresponding vehicle 1. After processing in S108, proceed to S109.

[0250] In S109, server 96 (e.g., test management unit 97a) or driving system 2 (e.g., participation status management unit 81) sets vehicle 1, whose individual stop function is disabled, to the test implementation state. That is, in vehicle 1, the test begins. The series of processes ends in S109.

[0251] On the other hand, in S110, if test consent is not obtained in S102, or if a cancellation operation is determined to have consent in S107, the driving system 2 (e.g., the participation status management unit 81) sets the participation status of vehicle 1 to "non-participation in test status". That is, it is decided not to conduct the test in vehicle 1. The series of processes ends in S110.

[0252] <Example of how to handle a vehicle user changing their mind>

[0253] like Figure 7 As shown in the flowchart, in order to appropriately respond to a vehicle user changing their mind after the test has started, the management system MS uses the separate stop function SFi of the test stop unit MS2. Figure 7 In a series of processes within the method, for example, on the driving system 2 side, the processor 53b of the software management unit 57 executes a computer program stored in memory 53a. Correspondingly, the processor 96b of the server 96 sometimes executes a computer program stored in memory 96a. This process begins during a test implementation, where the individual stop function SFi for the specific driving system 2 being processed is disabled. The trigger for initiating this process can also be given through input operation by the vehicle user on the operation input device 70a.

[0254] In the initial S201, the driving system 2 (e.g., the participation status management unit 81) determines, based on the aforementioned input operation, whether the vehicle user wishes to disengage from the test. If yes, proceed to S202. If no, the series of processes ends.

[0255] In S202, the driving system 2 (e.g., the participation status management unit 81) refers to the agreed content. If the agreed content is managed by the test management unit 97a, the agreed content is obtained from the server 96 via the communication system 43. After processing in S202, proceed to S203.

[0256] In S203, the driving system 2 (e.g., the participation status management unit 81) determines whether the agreed content allows for disengagement based on the vehicle user's reasons. If yes, proceed to S204. If no, proceed to S206.

[0257] In S204, the driving system 2 (e.g., the participation status management unit 81) activates the individual stop function SFi. After processing in S204, proceed to S205.

[0258] In S205, the driving system 2 (e.g., software application unit 82) responds to the activation of the individual stop function SFi by deactivating the test software and activating the previous software. The series of processes ends in S205.

[0259] In S206, the driving system 2 (e.g., HMI collaboration unit 84) notifies the vehicle user, via information prompting device 70b, that disengaging from the test is not permitted. This notification may also include the actual consent information provided to the vehicle user. The series of processes concludes in S206.

[0260] <Examples of how to handle previous functions or performance issues caused by testing software problems>

[0261] like Figure 8 As shown in the flowchart, in order to enable the vehicle 1 to perform its previous functions or performance, the management system MS uses the separate stop function SFi of the test stop unit MS2 when there are previous functions or performance issues that cannot be performed due to the test software.

[0262] For example, suppose conventional software addresses the road traffic laws of multiple countries spanning national borders, while the test software only addresses the road traffic laws of one country. Test Management Department 97a limits the test implementation area itself. In this case, when vehicle 1 moves outside the implementation area, the functionality of the conventional software is required.

[0263] exist Figure 8 In a series of processes within the method, for example, on the driving system 2 side, the processor 53b of the software management unit 57 executes a computer program stored in memory 53a. Correspondingly, the processor 96b of the server 96 sometimes executes a computer program stored in memory 96a. Assuming that the process begins during a test implementation, the individual stop function SFi for the specific driving system 2 being processed is disabled. In other words, the test software is in a state of being enabled, whereas previously the software was disabled. Furthermore, assuming that the test management unit 97a, functioning as the test implementation unit MS1, restricts the test implementation area to a specific country. The specific country is the country where the driver of vehicle 1 resides.

[0264] In the initial S301, the driving system 2 (e.g., the participation status management unit 81, which functions as the test stop unit MS2) obtains information related to the test implementation area set by the test management unit 97a. After processing in S301, the process proceeds to S302.

[0265] In S302, the driving system 2 (e.g., the participation status management unit 81) determines whether the vehicle 1 has moved outside the implementation area. If yes, proceed to S303. If no, the process in S302 is executed again after a predetermined time.

[0266] In S303, the driving system 2 (e.g., the participation status management unit 81) activates the individual stop function SFi. After processing in S303, proceed to S304.

[0267] In S304, the driving system 2 (e.g., software application unit 82) responds to the activation of the individual stop function SFi by deactivating the test software and activating the previous software. After processing in S304, proceed to S305.

[0268] In S305, the driving system 2 (e.g., the participation status management unit 81) determines whether vehicle 1 has returned to the implementation area. If yes, proceed to S306. If no, the process in S305 is executed again after a predetermined time.

[0269] In S306, the driving system 2 (e.g., the participation status management unit 81) disables the individual stop function SFi. After processing in S306, proceed to S307.

[0270] In S307, the driving system 2 (e.g., software application unit 82) responds to the activation of the individual stop function SFi by deactivating the previous software and activating the test software. The series of processes ends with S307. Furthermore, during the test, the change processing of the individual stop function SFi corresponding to the position of vehicle 1 is repeatedly performed. That is, after the processing in S307, the processing in S302 can be performed again after a specified time.

[0271] Furthermore, the testing area can be set based on legal constraints or other circumstances related to the market testing. The testing area can also be limited to special economic zones where market testing is recognized. Highly confidential areas such as military facilities can also be excluded. If the testing software is unsuitable for testing in high-traffic urban areas, then only low-traffic suburbs can be used as the testing area.

[0272] <Examples of how to handle unresolved issues in test software>

[0273] To appropriately address unresolved issues with the test software, the management system MS selectively utilizes the overall stop function SFw and the individual stop function SFi of the test stop unit MS2. In the management system MS, while the test software is being tested on a highway, the test stop unit MS2 monitors one or more metrics used to determine the stop function. These metrics may be the same as or different from the evaluation metrics evaluated in the test software evaluation unit 97d. Since these metrics determine whether the test can continue safely, the aforementioned safety-related metrics are preferred.

[0274] Then, the test stop unit MS2 presets one or more thresholds for the monitored metrics. The thresholds define the permissible range of the metrics. The thresholds can also be set based on, for example, the boundary between a safe and unsafe state. Based on the thresholds, the test stop unit MS2 determines whether the metrics have fallen outside the permissible range, and selectively activates the overall stop function SFw and the individual stop function SFi accordingly. A metric falling outside the permissible range based on a threshold is an example of a stop condition.

[0275] The stopping conditions can also be separated into individual stopping conditions and overall stopping conditions, and applied separately. To selectively activate the overall stopping function SFw and the individual stopping function SFi, for example, in each vehicle 1 serving as a test vehicle, the metric used to determine the individual stopping condition can be monitored based on the individual stopping condition that activates the individual stopping function SFi for each vehicle unit. The determination of the activation of the individual stopping function SFi can be made by the participation status management unit 81 of each vehicle 1 through self-diagnosis, or by the test management unit 97a of the server 96 that receives the monitoring results. The metric and threshold used in the self-diagnosis can be common among the test vehicles participating in the same test. On the other hand, in the server 96, the metric used to determine the overall stopping condition can also be monitored based on the overall stopping condition that activates the overall stopping function SFw.

[0276] Here, as an example of how to deal with unresolved issues, the following is an example of how to address them. Figure 9 , 10 The method shown in the flowchart will be explained. Figure 9 In a series of processes, in each vehicle 1, the processor 53b of the software management unit 57 of the driving system 2 functions as the test stop unit MS2 and executes the computer program stored in the memory 53a. Figure 9 A series of processes, such as coordinating the timing of the start of each vehicle's individual test. In Figure 10In a series of processes, the processor 96b of the server 96 functions as the test stop unit MS2, executing the computer program stored in the memory 96a. Figure 10 A series of processes, such as coordinating with server 96 to determine when the overall test should begin on a scheduled basis.

[0277] exist Figure 9 In the initial S401, the driving system 2 (e.g., the participation status management unit 81) sets one or more metrics and their thresholds to be used under the condition of a standstill. This setting can also be performed by obtaining the metric and its threshold settings determined by the test management unit 97a of the server 96. Alternatively, this setting can be obtained from the settings determined by the test management unit 97a, and partially modified based on the individual specifications of the vehicle 1 and the individual driving environment. After processing in S401, proceed to S402.

[0278] In addition, one or more metrics can include the number of software errors. Software errors can include software malfunctions and software defects. A software error can be the number of times the software itself outputs diagnostic codes, or the number of times the driving system 2 detects anomalies in the software. The number of software errors can be only the number of errors occurring in the test software, or it can also include the number of errors occurring in software directly related to the test software, or it can be the total number of errors occurring in all software of the driving system 2. Software directly related to the test software is, for example, software that directly refers to the output of the test software to perform processing. Alternatively, instead of the number of errors, a function that weights the occurrence of errors based on one or both of the importance and urgency of the error can be used as the metric.

[0279] One or more metrics may include an error representing a decrease in performance. The performance referred to here can be general performance. If the test software is software that electronically controls the damper included in the motion actuator 60, the error may be a value representing the occurrence of abnormal swaying of the vehicle 1 when cornering. If the test software is software that implements the image recognition function of a camera acting as an external environment sensor 41, the error may be a value representing a decrease in the recognition rate of objects affected by interference such as severe weather.

[0280] One or more metrics can be the number of violations of safety judgments. Violations of safety judgments can be violations of the safety envelope or violations of safety distances. When a violation of a safety judgment is judged as including a collision risk exceeding a risk threshold, a function weighted by one or both of the severity and urgency of the violation can be used as the metric instead of the number of violations. Similarly, when a violation of a safety judgment is judged as including vehicle 1 violating rules stored in the rule set, a function weighted by one or both of the severity and urgency of the violation can be used as the metric instead of the number of violations. That is, the above-mentioned violation metrics can also be used as metrics in the case of a standalone stop. Furthermore, one or more metrics can include the number of times the behavior of vehicle 1, which has been monitored by driving system 2 according to the driving strategy, has failed.

[0281] In S402, the driving system 2 (e.g., the participation status management unit 81) continuously monitors the metrics set in S401. After processing in S402, the process proceeds to S403.

[0282] In S403, the driving system 2 (e.g., the participation status management unit 81) determines whether the monitored metrics exceed the thresholds set in S401. That is, it determines whether the condition for a standalone stop is met. If multiple metrics are set, S401 pre-sets whether all metrics need to exceed the corresponding thresholds, or only one or a portion of the metrics need to exceed the corresponding thresholds, and performs the judgment according to this condition setting. If yes, proceed to S404. If no, return to S402, and the loop processing of S402 to S403 is repeated until the end of the test period.

[0283] In S404, the driving system 2 (e.g., the participation status management unit 81) activates the individual stop function SFi. After processing in S404, proceed to S405.

[0284] In S405, the driving system 2 (e.g., software application unit 82) responds to the activation of the individual stop function SFi by deactivating the test software and activating the previous software. After processing in S405, proceed to S406.

[0285] In S406, the driving system 2 (e.g., the participation status management unit 81) sends a message to the test management unit 97a of the server 96 stating that the test in the vehicle 1 equipped with it has been stopped independently. At the same time, the reason for the independent test stoppage and monitoring data of the measurement indicators may also be sent. The series of processes ends in S406.

[0286] Next, based on Figure 10 , to Figure 9 The processing in vehicle 1 will be explained in relation to the processing in server 96. In the initial S501, server 96 (e.g., test management unit 97a) sets one or more metrics and their thresholds to be used under the condition of overall stoppage. The metric could be, for example, the number of vehicles 1 that stopped testing individually through self-diagnosis during the test implementation. The metric could also be used instead of the number of vehicles, using the ratio of individually stopped vehicles 1 to all test vehicles, i.e., the individual stoppage rate. Hereinafter, the explanation will continue with the premise that the metric is the number of vehicles. After the processing in S501, proceed to S502.

[0287] In S502, server 96 (e.g., test management unit 97a) counts the number of vehicles that have individually stopped testing based on the information received from each vehicle 1 through the processing in S406. After processing in S502, proceed to S503.

[0288] In S503, server 96 (e.g., test management department 97a) determines whether the number of vehicles counted in S502 exceeds a threshold. That is, it determines whether the condition for overall stopping is met. If yes, it proceeds to S504. If no, it returns to S502, and the loop processing from S502 to S503 is repeated until the end of the test period.

[0289] In S504, server 96 (e.g., test management department 97a) activates the overall shutdown function SFw. After processing in S504, proceed to S505.

[0290] In S505, server 96 (e.g., test management unit 97a) sends a message to all test vehicles to stop the test. Upon receiving the message in S505, each vehicle 1 activates the overall stop function SFw in the participation status management unit 81.

[0291] Additionally, the destination can include vehicle 1, which has already enabled the individual stop function. This is because, if the individual stop function SFi is disabled again based on a certain trigger, and the participation status management unit 81 of vehicle 1 does not recognize the activation of the overall stop function SFw, the test in vehicle 1 may be restarted. The series of processes ends at S505.

[0292] Summary

[0293] According to the first embodiment described above, it is possible to distinguish between the individual stop function SFi used by participating units and the overall stop function SFw. Therefore, it is possible to stop the test in an appropriate manner, taking into account the suitability and usability of the test application.

[0294] Furthermore, in the first embodiment, particularly according to the first method described above, in the test preparation state, the conventional software corresponding to the participating unit and applied to the hardware of the driving system 2, which is a vehicle system, is rewritten as test software downloaded from the server 96 to the driving system 2. Then, if a specific participating unit stops the test individually using the individual stop function SFi, the conventional software is downloaded from the server 96, and the test software is restored to the conventional software. By rewriting and restoring in accordance with the start and stop of the test, the test can be performed and stopped with fewer hardware resources.

[0295] Furthermore, in the first embodiment, particularly according to the second method described above, in the preparation state, in addition to the conventional software, test software is downloaded from server 96 to the hardware of the driving system 2 corresponding to the participating unit. Then, with one or both of the conventional software and the test software valid, both are stored in the hardware. Furthermore, if the participating unit stops testing, with both stored in the hardware, the conventional software is valid, and the test software is invalidated. In this way, when vehicle 1 stops testing, the test software remains in the hardware of the driving system 2, thus facilitating the restart of the test. Additionally, since the state of the software during the test is saved, it is easy to verify the test afterward. Therefore, the test can be stopped in an appropriate manner.

[0296] Furthermore, according to the first embodiment, in the standby function SFi during the preparation state, if a participating unit performs a participation intention expression operation for the test via the HMI device 70, the participating unit's test is not started and is stopped until the grace period for canceling the participation intention expression operation has elapsed. Therefore, even if a participating unit mistakenly performs a participation intention expression operation, it is possible to suppress any impact on the test, such as the behavior of vehicle 1, caused by violating the actual intention.

[0297] Furthermore, according to the first embodiment, consent is obtained from specific participating entities that they cannot cancel their participation in the test within a specified period. During this period, the individual stop function SFi for the specific participating entity is fixed in an invalid state. By avoiding the easy invalidation of the individual stop function SFi, testing can be used efficiently, which helps to quickly improve the driving system 2.

[0298] Furthermore, according to the first embodiment, a test implementation area is set. Then, during the test implementation state, if a specific vehicle among the multiple vehicles moves outside the implementation area, a separate stop function SFi is used to individually stop the test for the participating unit corresponding to that specific vehicle. That is, it is possible to suppress the movement of only a portion of the vehicles outside the implementation area, and the entire test is stopped. Therefore, the test can be used efficiently, which helps to quickly improve the driving system 2.

[0299] Furthermore, according to the first embodiment, when a specific vehicle among multiple vehicles meets the conditions for individual stopping, the individual stopping function SFi is used to individually stop the test for the participating unit corresponding to that specific vehicle. Then, when the conditions for overall stopping are met, the overall stopping function is used to stop the test as a whole. Since the individual stopping function SFi and the overall stopping function can be used differently depending on the activation conditions, the test can be stopped in an appropriate manner.

[0300] Furthermore, according to the first embodiment, the condition for overall stopping is that, among the multiple vehicles 1, the number or proportion of vehicles that have individually stopped testing is above a predetermined threshold. By linking the condition for overall stopping with the individual stopping function, testing can be stopped in stages, thereby improving the appropriateness of the testing process.

[0301] Furthermore, according to the first embodiment, within a single vehicle unit, a metric is used to determine whether to stop the test individually; and in a whole, the decision to stop the test as a whole is based on the condition of multiple vehicles 1. By determining the stop function in this phased manner, the test can be stopped in an appropriate way.

[0302] (Second Implementation)

[0303] like Figure 11 , 12 As shown, the second embodiment is a variation of the first embodiment. The second embodiment will be described focusing on its differences from the first embodiment.

[0304] In the second embodiment, the HMI collaboration unit 84 collaborates with the participation status management unit 81 to implement notifications to vehicle users at appropriate timings. Therefore, the software management unit 57 functions as a notification control device for controlling the implementation of test-related notifications.

[0305] exist Figure 11 In the series of processes shown in the flowchart, the processor 53b of the software management unit 57 of the driving system 2 functions as the test stop unit MS2, executing the computer program stored in the memory 53a. Figure 11 A series of processes are repeatedly executed at specified intervals during the test implementation.

[0306] In the initial S601, the driving system 2 (e.g., the participation state management unit 81) obtains the conditions for a separate stop of the test. The conditions for a separate stop here include the area where the test is conducted, meaning the test is conducted only on a highway. The conditions for a separate stop can also be linked to the ODD (Operational Development Code). For example, if the conditions for performing Level 3 autonomous driving are limited to a highway, the area where the test is conducted is substantially the same as the ODD. After processing in S601, proceed to S602.

[0307] In S602, the driving system 2 (e.g., the participation status management unit 81) determines whether the conditions for a separate stop are about to be met. For example, if vehicle 1 is traveling on a highway, it determines whether there is a planned departure from the highway. "About" could also mean within a preset time, such as 5 minutes later. That is, it determines whether the probability of the conditions for a separate stop being met within the preset time is above a predetermined threshold. If yes, proceed to S603. If no, end the series of processes.

[0308] In S603, the driving system 2 (e.g., HMI collaboration unit 84) uses information notification device 70b to issue a test interruption notification to the vehicle user (e.g., the driver). The test interruption notification may include the fact that the test is about to be interrupted and the reason for the interruption.

[0309] For example, such as Figure 12 As shown, along with the route guidance image, the CID70b2 screen may also display text such as "The ongoing test will be interrupted in 3 minutes due to leaving the highway." After processing in S603, proceed to S604.

[0310] In S604, the driving system 2 (e.g., the participation status management unit 81) activates the individual stop function SFi. After processing in S604, proceed to S605.

[0311] In S605, the driving system 2 (e.g., software application unit 82) responds to the activation of the individual stop function SFi by deactivating the test software and activating the previous software. After processing in S605, proceed to S606.

[0312] In S606, the driving system 2 (e.g., driving planning unit 22 and motion control unit 31) performs vehicle control to establish the conditions for a separate stop. For example, vehicle 1 is guided autonomously to a highway exit and then proceeds onto a regular road. The series of processes concludes in S606.

[0313] The test interruption notification is preferably issued before the actual test is interrupted, after the vehicle control that causes the stop condition is executed. Therefore, in S602, if it is determined that there is a possibility that the vehicle control in S606 will be executed before the notification in S603, the participation state management unit 81 can also output a retention request or delay request for the vehicle control to the driving planning unit 22 for adjustment, so that the test interruption notification is implemented before the vehicle control. For example, the driving planning unit 22 can also control the speed of vehicle 1 to be lower, delaying the timing of leaving the highway.

[0314] According to the second embodiment described above, adjustments are made before controlling vehicle 1, which becomes the reason for the separate stop, so that a notification related to the test being stopped is delivered to the vehicle user. This adjustment allows the vehicle user to recognize the test's cessation before the test becomes unavoidable, thus improving the appropriateness of the testing process.

[0315] (Third implementation method)

[0316] like Figure 13 , 14 As shown, the third embodiment is a variation of the second embodiment. The third embodiment will be described focusing on its differences from the second embodiment.

[0317] In the third embodiment, the HMI collaboration unit 84 implements a test interruption notification at a time that avoids the vehicle 1 being in motion. When the test is being conducted while the driver is manually driving, avoiding the in-motion phase reduces the negative impact of the notification on driving. The test during manual driving could, for example, be a test of an HMI providing route guidance to the driver's destination.

[0318] exist Figure 13 In the series of processes shown in the flowchart, the processor 53b of the software management unit 57 of the driving system 2 functions as the test stop unit MS2, executing the computer program stored in the memory 53a. Figure 13 A series of processes are repeatedly executed at specified intervals during the test implementation.

[0319] In the initial S701, the driving system 2 (e.g., the participation status management unit 81) obtains the conditions for a test-based independent stop. In S702, following the processing of S701, the driving system 2 (e.g., the participation status management unit 81) determines whether the conditions for an independent stop are met. If yes, proceed to S703. If no, proceed to S704.

[0320] In S703, the driving system 2 (e.g., the participation status management unit 81) activates the individual stop function SFi. After processing in S703, proceed to S704.

[0321] In S704, the driving system 2 (e.g., the participation status management unit 81) obtains the status of the overall stop function SFw from the test management unit 97a of the server 96 and determines whether the overall stop function SFw has been activated. If yes, proceed to S705. If no, end the series of processes.

[0322] In S705, the driving system 2 (e.g., software application unit 82) responds to the activation of the individual stop function SFi or the overall stop function SFw, deactivating the test software and activating the previous software. After processing in S705, proceed to S706.

[0323] In S706, the driving system 2 (e.g., HMI coordination unit 84) determines whether the vehicle 1 is in motion. If yes, the determination in S706 is repeated after a predetermined time has elapsed. If no, the vehicle proceeds to S707.

[0324] In S707, the driving system 2 (e.g., HMI collaboration unit 84) uses information notification device 70b to notify the vehicle user (e.g., the driver) of a test interruption. The test interruption notification may include a message that the test has been interrupted and the reason for the interruption.

[0325] exist Figure 14 The example shown illustrates a notification when the overall stop function SFw is activated and the test is stopped. Alternatively, the CID70b2 screen may display text such as "Route guidance test during manual driving has stopped." and "(Reason) Due to an increase in cases where drivers misinterpret the route guidance display and make forced lane changes." The series of processes concludes with the S707 process.

[0326] Furthermore, the HMI coordination unit 84 can adjust the timing of the notification in more detail. For example, even if the HMI coordination unit 84 allows notification at the time of temporary stop of vehicle 1, it can avoid the time of temporary stop at the stop line and instead issue the notification at the time of temporary stop due to the stop signal. This is because at the time of temporary stop at the stop line, it can be assumed that the driver is focused on monitoring the surroundings in order to restart, and the restart time is difficult to predict. In addition, the HMI coordination unit 84 can also retain the notification after vehicle 1 stops until the time of restarting, issue the notification at the time when the driver gets into the vehicle to restart, or at the time when the power switch (ignition switch) of vehicle 1 is turned on.

[0327] According to the third embodiment described above, when a participating unit stops testing, a notification related to the test being stopped is executed at a scheduled time, avoiding the driving of vehicle 1. Since the notification is executed at a time when vehicle users are likely to be concentrated, it is easier for vehicle users to recognize the notification.

[0328] (Fourth Implementation)

[0329] like Figure 15 , 16 As shown, the fourth embodiment is a variation of the second embodiment. The fourth embodiment will be described focusing on its differences from the second embodiment.

[0330] In the fourth embodiment, it is assumed that the agreed-upon test content includes a strategy that allows the vehicle user to freely and temporarily withdraw from the test. Therefore, the participation status management unit 81 and the HMI cooperation unit 84, which function as the test stop unit MS2, provide a function that allows the vehicle user to set a method for temporarily interrupting the test. The temporary interruption mentioned here is equivalent to a method of stopping the standby function SFi.

[0331] For example, such as Figure 15 As shown, the HMI Collaboration Unit 84, for example using CID70b2, provides a user interface UI1 for vehicle users to set temporary interruption methods for testing. Temporary interruption methods include period-based and status-based methods. In the period-based method, vehicle users can directly set the period for temporary interruption testing. For example, vehicle users can set the start and end dates of the period.

[0332] In a condition-based approach, vehicle users can individually configure whether to temporarily suspend the test if a specified condition has occurred. Therefore, it is preferable that multiple conditions can be set independently. For example, in... Figure 15 In the context of the system, as a configurable condition, the interface provides conditions such as "when there are fellow passengers", "commuting", and "night".

[0333] For example, if the vehicle user has set up a temporary interruption "when there are passengers", the participation status management unit 81 can also obtain sensor data such as seating sensors to determine whether there are passengers. In addition, for example, if the vehicle user has set up a temporary interruption "during commuting", the participation status management unit 81 can also obtain the departure and destination information in the route planning to determine whether the departure or destination is the work location pre-registered by the vehicle user.

[0334] The Status Management Department 81 and HMI Collaboration Department 84 can also implement reminder notifications to remind vehicle users of the existence of the test if the temporary interruption lasts for more than a specified period. For example, in Figure 16In the series of processes shown in the flowchart, the processor 53b of the software management unit 57 of the driving system 2 functions as the test stop unit MS2, executing the computer program stored in the memory 53a. Figure 16 A series of processes are repeatedly executed under the temporary interruption state set by the vehicle user.

[0335] In the initial S801, the driving system 2 (e.g., the participation status management unit 81) obtains the duration of the activation state of the individual stop function SFi. In S802, following the processing of S801, the driving system 2 (e.g., the participation status management unit 81) determines whether the duration obtained in S801 exceeds a preset threshold (e.g., one week). If yes, proceed to S803. If no, end the series of processes.

[0336] In S803, the driving system 2 (e.g., HMI collaboration unit 84) implements a reminder notification. The reminder notification may include a prompt for the vehicle user to decide whether to continue or cancel participation in the test. The reminder notification may also include a reiteration of the test content.

[0337] According to the fourth embodiment described above, the provided user interface UI1 allows vehicle users to set a temporary stop method for testing within the participating unit. Then, based on the settings in the user interface UI1, the validity of the individual stop function SFi is determined. Therefore, usability during testing can be improved.

[0338] Furthermore, according to the fourth embodiment, if the duration of the temporary stop set in the user interface UI1 exceeds a preset period, a reminder notification is implemented to remind the vehicle user of the existence of the test. Since this increases the likelihood of the user returning to the test after remembering its existence, the test can be used more efficiently, contributing to the rapid improvement of the driving system 2.

[0339] (Fifth implementation method)

[0340] like Figure 17 As shown, the fifth embodiment is a variation of the first embodiment. The fifth embodiment will be described focusing on its differences from the first embodiment.

[0341] In the fifth embodiment, vehicle 1 is envisioned as a vehicle used in MaaS, such as a robot taxi, an autonomous bus, or a carpooling vehicle. Furthermore, in the fifth embodiment, the participating unit in the test is not a vehicle unit, but a passenger unit. The passenger unit can be a single passenger or a group of passengers.

[0342] The tests conducted in the passenger unit can be related to the HMI used by passengers during their journey to vehicle 1, or they can be related to new payment services. The functions that are subject to testing can include those provided in collaboration with mobile terminals 91 such as smartphones owned by passengers.

[0343] In the fifth embodiment, the participation status management unit 81 of the driving system 2 confirms each passenger's intention to refuse the test, and based on the confirmation of intention, performs the test for each passenger. For example, in Figure 17 In the series of processes shown in the flowchart, the processor 53b of the software management unit 57 of the driving system 2 functions as the test stop unit MS2, executing the computer program stored in the memory 53a. Figure 17 A series of processes are repeatedly executed at specified intervals during the test implementation.

[0344] In the initial S901, the driving system 2 (e.g., the participation status management unit 81) determines whether a new passenger has boarded vehicle 1. If yes, proceed to S902. If no, the series of processes ends.

[0345] In S902, the driving system 2 (e.g., the participation status management unit 81) determines whether the new passenger has refused the test. This determination can also be made by obtaining the passenger's pre-set intention expression in the application used in the MaaS. Alternatively, this determination can be based on information that the new passenger confirmed their intention using the onboard HMI when boarding vehicle 1. If yes, proceed to S903. If no, proceed to S905.

[0346] In S903, the driving system 2 (e.g., the participation status management unit 81) activates the individual stop function SFi for the passenger who was the subject of judgment in S902. In S904, following the processing in S903, the driving system 2 (e.g., the software application unit 82) is set to not provide the test function for the passenger who was the subject of judgment in S902, and only provide the previous function. The series of processes ends in S904.

[0347] In S905, the driving system 2 (e.g., the participation status management unit 81) disables the individual stop function SFi for the passenger who was the subject of judgment in S902. In S906, following the processing of S905, the driving system 2 (e.g., the software application unit 82) is configured to provide a test function for the passenger who was the subject of judgment in S902. The series of processes ends in S904.

[0348] According to the fifth embodiment described above, the participating unit is set as a passenger unit based on the passenger. Then, for passenger units that confirm their intention to refuse testing, the individual stop function SFi is enabled, while for other passenger units, the testing function is provided. Since the testing in vehicle 1 can be carried out with respect to the individual wishes of passengers, usability can be improved.

[0349] (Other implementation methods)

[0350] The above describes several implementation methods, but this disclosure should not be limited to these implementation methods. It can be applied to various implementation methods and combinations without departing from the spirit of this disclosure.

[0351] As another implementation method, when using a POV (Point of Vehicle) 1 capable of participating in road traffic to conduct the test, the participating unit in the test can also be an individual unit. The term "individual" here may include passengers in addition to the driver.

[0352] For example, in vehicle 1, which has displays facing the driver and front passenger and displays facing the rear seats, tests related to the display layout are conducted. Furthermore, if an occupant in the rear seats refuses to participate in the test, the test can be conducted only on the displays facing the driver and front passenger. Additionally, for example, in tests related to the temperature control of the seat heaters, the test can be conducted only in the seat occupied by the occupant who wishes to participate.

[0353] As another embodiment related to the fifth embodiment, it can also be configured such that the server 96 is not required in the test system, and the test system is completed within a vehicle used in MaaS. In this case, the driving system 2 can have all the functions of the test implementation unit MS1, the test stop unit MS2, and the software management unit MS3. Then, the test stop unit MS2 of the driving system 2 can also control both the individual stop function SFi and the overall stop function.

[0354] As another implementation, the structure of the processing system 50 may also be as follows: Figure 18 , 19 The structure shown. For example, in Figure 18 The system employs a structure with multiple domain controllers 451 to 454. Each domain controller 451 to 454 can have the same hardware structure as the processing system or ECU in the first embodiment.

[0355] The ADAS domain controller 451 integrates functions related to ADAS (Advanced Driver-Assistance Systems). The ADAS domain controller 451 may also implement a portion of the recognition function, a portion of the judgment function, and a portion of the control function in a combined manner. For example, the recognition function implemented by the ADAS domain controller 451 may be a function equivalent to the fusion of information detected by multiple sensors 40 in the detection unit 10 of the first embodiment, or a simplified version of that function. The judgment function implemented by the ADAS domain controller 451 may be, for example, a function equivalent to the prediction unit 21 and driving planning unit 22 of the first embodiment, or a simplified version of that function. The control function implemented by the ADAS domain controller 451 may be, for example, a function equivalent to the motion control unit 31 of the first embodiment, which generates request information to the motion actuator 60.

[0356] The powertrain domain controller 452 integrates functions related to powertrain control. The powertrain domain controller 452 may also implement at least a portion of the identification function and at least a portion of the control function in combination. For example, a portion of the identification function implemented by the powertrain domain controller 452 may be the function of identifying the driver's operating state on the motion actuator 60, which corresponds to the function of the internal identification unit 13 in the first embodiment. A portion of the control function implemented by the powertrain domain controller 452 may be the function of controlling the motion actuator 60, which corresponds to the function of the motion control unit 31 in the first embodiment.

[0357] The cockpit domain controller 453 integrates cockpit-related functions. The cockpit domain controller 453 may also implement at least a portion of the identification function and at least a portion of the control function in combination. For example, a portion of the identification function implemented by the cockpit domain controller 453 may be the function of identifying the on / off state of the HMI device 70 in the internal identification unit 13 of the first embodiment. A portion of the control function implemented by the cockpit domain controller 453 may be, for example, the function equivalent to the HMI output unit 71 of the first embodiment.

[0358] The connectivity domain controller 454 aggregates connectivity-related functions. The connectivity domain controller 454 may also implement at least a portion of the identification functions in combination. A portion of the identification functions implemented by the connectivity domain controller 454 may be the function of organizing and converting global location data, V2X information, etc. of the vehicle obtained from the communication system 43 into a form that can be used by, for example, the ADAS domain controller 451 and the cockpit domain controller 453.

[0359] In this embodiment, when the management of the software is performed by each domain controller 451 to 454 that performs its own function, the multiple domain controllers 451 to 454 may sometimes be equivalent to "software management units" that constitute the management system MS or even the test system.

[0360] In addition, Figure 19 The system employs a structure comprising a unified ECU 551 and multiple regional ECUs 551a to d. In this structure, the multiple regional ECUs 551a to d perform control over devices, modules, units, and apparatuses located in designated areas within the vehicle 1.

[0361] For example, external environment sensors 41 such as cameras located at the front of vehicle 1, and information display devices 70b such as CID 70b2 located in the cabin, are controlled by area ECUs 551a or 551b located at the front of vehicle 1. External environment sensors 41 such as millimeter-wave radar located at the rear of vehicle 1 are controlled by area ECUs 551c or 551d located at the rear of vehicle 1.

[0362] The integrated ECU 551 collects detection information from each regional ECU 551a to 551d and performs comprehensive control of the driving system 2 on each regional ECU 551a to 551d. For example, the integrated ECU 551 can also perform almost all of the planning and software management functions.

[0363] As another implementation, vehicles 1, 1A, and 1B equipped with driving systems 2, 2A, and 2B can be right-hand drive vehicles or left-hand drive vehicles. Furthermore, the traffic environment in which vehicles 1, 1A, and 1B operate can be either a left-hand drive or right-hand drive traffic environment. The driving systems 2, 2A, and 2B based on this disclosure can also be appropriately optimized to take into account the road traffic laws and customs of various countries and regions, as well as the methods of police investigation, prosecution, criminal proceedings, and even civil proceedings related to traffic accidents.

[0364] The control unit and method described in this disclosure can also be implemented by a dedicated computer comprising a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and method described in this disclosure can also be implemented by dedicated hardware logic circuitry. Alternatively, the apparatus and method described in this disclosure can also be implemented by one or more dedicated computers comprising a processor executing a computer program and a combination of one or more hardware logic circuits. Furthermore, the computer program can also be stored as instructions executed by a computer in a computer-readable, non-temporary physical recording medium.

[0365] (The disclosure of technical ideas)

[0366] This specification discloses several technical ideas described in the following list of items. Some items are sometimes described in a multiple dependent form, whereby a preceding item is selectively referenced in a subsequent item. These items described in a multiple dependent form define several technical ideas.

[0367] <Technical Idea 1>

[0368] A testing system is used to test vehicle systems (2, 2A, 2B) mounted on one or more vehicles (1, 1A, 1B) capable of participating in road traffic, wherein the system comprises:

[0369] Test Implementation Department (MS1) sets up and implements tests; and

[0370] The test stop unit (MS2) has both an individual stop function (SFi) that stops the test individually by participating units and an overall stop function (SFw) that stops the test as a whole, whether the test is in the preparation or execution state.

[0371] <Technological Ideas 2>

[0372] According to the test system described in technical concept 1, wherein,

[0373] The test was conducted in conjunction with changes to the software used in the vehicle system.

[0374] The testing system also includes a software management unit (MS3) for managing the application of the software to the vehicle.

[0375] The software management department performs the following processing:

[0376] In the prepared state, the existing software used in the hardware (51a, 53a, 58, 59) of the vehicle system corresponding to the participating unit is rewritten as test software downloaded from the server (96) to the vehicle system.

[0377] In the event that a specific participating unit stops the test individually via the individual stop function, the previous software is downloaded from the server, and the test software is restored to the previous software.

[0378] <Technological Idea 3>

[0379] According to the test system described in technical concept 1, wherein,

[0380] The test was conducted in conjunction with changes to the software used in the vehicle system.

[0381] The testing system also includes a software management unit (MS3) for managing the application of the software to the vehicle.

[0382] The software management department performs the following processing:

[0383] In the prepared state, in addition to the usual software, test software is downloaded from the server (96) to the vehicle system, which is equipped with hardware (51a, 53a, 58, 59) corresponding to the participating unit.

[0384] In a state that enables one or both of the prior software and the test software to be effective, the software of both parties shall be retained in the hardware.

[0385] If the participating unit stops the test, while keeping the software of both parties on the hardware, the previous software is made valid and the test software is made invalid.

[0386] <Technological Ideas 4>

[0387] The test system according to any one of technical concepts 1 to 3, wherein,

[0388] In the separate stop function of the preparation state, if the participating unit performs a participation intention expression operation for the test through the HMI device, the test will not start for the participating unit until the grace period for canceling the participation intention expression operation has elapsed.

[0389] <Technological Idea 5>

[0390] The test system according to any one of technical concepts 1 to 4, wherein,

[0391] The test stop unit obtains consent from a specific participating unit that participation in the test cannot be cancelled within a specified period, and during the specified period, the individual stop function for the specific participating unit is fixed to an invalid state.

[0392] <Technological Ideas 6>

[0393] The test system according to any one of technical concepts 1 to 5, wherein,

[0394] The term "more than one vehicle" refers to multiple vehicles.

[0395] The test implementation unit sets the implementation area for the test.

[0396] In the implementation state, if a specific vehicle among the multiple vehicles moves outside the implementation area, the test stop unit, as the individual stop function, individually stops the test of the participating unit corresponding to the specific vehicle.

[0397] <Technological Ideas 7>

[0398] The test system according to any one of technical concepts 1 to 6, wherein,

[0399] The term "more than one vehicle" refers to multiple vehicles.

[0400] The test stop section performs the following processing:

[0401] Define the conditions for stopping each individual test and the conditions for stopping the entire test.

[0402] If a specific vehicle among the multiple vehicles meets the conditions for individual stopping, then, as part of the individual stopping function, the test of the participating unit corresponding to that specific vehicle is stopped individually.

[0403] If the conditions for overall stop are met, the test is stopped entirely as a function of overall stop.

[0404] <Technological Ideas 8>

[0405] According to the test system described in technical concept 7, wherein,

[0406] The condition for the overall stop is that, among the multiple vehicles, the number or proportion of vehicles that individually stopped the test is above a specified threshold.

[0407] <Technological Ideas 9>

[0408] The test system according to any one of technical concepts 1 to 8, wherein,

[0409] The test stop section performs the following processing:

[0410] Set the individual stopping conditions for the test.

[0411] The method is adjusted to provide notifications to vehicle users regarding the cessation of the test before controlling the vehicle for reasons that would cause the individual stop to occur.

[0412] <Technological Ideas 10>

[0413] The test system according to any one of technical concepts 1 to 8, wherein,

[0414] When the participating unit stops the test, the test stop unit will issue a notification related to the test being stopped, avoiding the time when the vehicle is in motion.

[0415] <Technological Ideas 11>

[0416] The test system according to any one of technical concepts 1 to 10, wherein,

[0417] The test stop section performs the following processing:

[0418] A user interface (UI1) is provided that allows vehicle users to configure how the test is temporarily stopped in the participating units.

[0419] Based on the settings in the user interface, determine the validity of the separate stop function.

[0420] <Technological Ideas 12>

[0421] According to the test system described in technical concept 11, wherein,

[0422] If the duration of the temporary stop set in the user interface exceeds a preset period, the test stop unit will implement a reminder notification to remind the vehicle user of the existence of the test.

[0423] <Technological Ideas 13>

[0424] The test system according to any one of technical concepts 1 to 12, wherein,

[0425] The vehicle in question is used for passenger service.

[0426] The test is a test that uses the participating units as passenger units based on the passengers.

[0427] The test stop section performs the following processing:

[0428] For passenger units that are confirmed to have the intention to refuse the test, the individual stop function is enabled; for other passenger units, the test function is provided.

[0429] <Technological Ideas 14>

[0430] A testing system is used to test vehicle systems (2, 2A, 2B) mounted on each of multiple vehicles (1, 1A, 1B) capable of participating in road traffic, wherein the system includes:

[0431] Each of the aforementioned vehicle systems; and

[0432] The server (96) is communicatively connected to each of the vehicle systems.

[0433] Each of the vehicle systems uses one or more metrics within its respective vehicle to determine the effectiveness of the individual stop function, which is tested on a vehicle-by-vehicle basis.

[0434] Based on the effectiveness of the individual stop functions in the multiple vehicles, the server decides to disable the overall stop function of the test.

[0435] Based on this technical approach, metrics are used in each vehicle system to determine whether to stop testing individually, while the server determines whether to stop testing as a whole based on the status of multiple vehicles. By making this phased decision to stop the function, testing can be stopped in an appropriate manner.

[0436] <Technological Ideas 15>

[0437] A vehicle system configured to be mounted on a vehicle (1, 1A, 1B) capable of participating in road traffic, comprising at least one processor (57b) for performing tests associated with said vehicle, wherein...

[0438] The at least one processor performs the following processing:

[0439] Determine one or more metrics and the threshold values ​​for those metrics;

[0440] By monitoring one or more metrics and, based on the relationship between the one or more metrics and thresholds, determining the effectiveness of the individual stop function for stopping tests on a vehicle-by-vehicle basis; and

[0441] The information indicating that the individual stop function is enabled is sent to the server (96) that manages the test as a whole.

[0442] Based on this technical concept, metrics are used in each vehicle system to determine whether testing should be stopped individually. Therefore, even in cases with highly urgent reasons for stopping, testing can be stopped quickly on a vehicle-by-vehicle basis. Then, in the case of an individual test stoppage, this information is sent to the server, thus enabling objective management of the testing process on the server.

[0443] <Technological Ideas 16>

[0444] A server is provided for testing vehicle systems (2, 2A, 2B) mounted on each of multiple vehicles (1, 1A, 1B) that can participate in road traffic, and is communicatively connected to each of the vehicle systems. The server has at least one processor (96b).

[0445] The at least one processor performs the following processing:

[0446] Receive information from the vehicle indicating that the individual stop function has been activated by self-diagnosis on a vehicle-by-vehicle basis; and

[0447] Based on the effectiveness of the individual stop function in each of the vehicles, it is determined whether the overall stop function of the test is to be stopped.

[0448] Based on this technical concept, each vehicle uses self-diagnostic metrics to determine whether to stop testing individually, while the server determines whether to stop testing as a whole based on the status of multiple vehicles. By making this phased decision to stop the function, testing can be stopped in an appropriate manner.

[0449] <Technological Ideas 17>

[0450] A notification control device is configured to be mounted on a vehicle (1, 1A, 1B) capable of participating in road traffic, and includes at least one processor (57b) for controlling the implementation of test-related notifications associated with the vehicle.

[0451] The at least one processor performs the following processing:

[0452] Refer to the conditions for individually stopping the test by referring to the vehicle;

[0453] Determine whether the condition for the individual stop is about to be met; and

[0454] If it is determined that the conditions for the individual stop are about to be met, the method is adjusted to provide a notification to the vehicle user related to the test being stopped before executing the control of the vehicle that is the cause of the individual stop.

[0455] Based on this technical concept, an adjustment is made to notify vehicle users of the test being stopped before controlling the vehicle that would otherwise be subject to conditions for a separate stop. This adjustment allows vehicle users to recognize the test's cessation before the situation necessitates stopping, thus improving the appropriateness of the testing process.

[0456] <Technological Ideas 18>

[0457] A notification control device is configured to be mounted on a vehicle (1, 1A, 1B) capable of participating in road traffic, and includes at least one processor (57b) for controlling the implementation of test-related notifications associated with the vehicle.

[0458] When the at least one processor stops the test being conducted while the vehicle is manually driven, it performs the following processing:

[0459] Determine whether the vehicle is in motion; and

[0460] Based on the determination, and excluding the timing of the test being conducted while the vehicle is in motion, a notification related to the test being stopped is issued.

[0461] Based on this technical concept, when a vehicle is stopped for testing, a notification related to the test being stopped is delivered at a scheduled time, avoiding the vehicle being in motion. Because the notification is delivered at a time when vehicle users are likely to be present, it is easier for vehicle users to recognize the notification.

Claims

1. A testing system for testing vehicle systems (2, 2A, 2B) mounted on one or more vehicles (1, 1A, 1B) capable of participating in road traffic, wherein, have: Test Implementation Department (MS1) sets up and implements tests; and The test stop unit (MS2) has both an individual stop function (SFi) that stops the test individually by participating units and an overall stop function (SFw) that stops the test as a whole, whether the test is in the preparation or execution state.

2. The testing system according to claim 1, wherein, The test was conducted in conjunction with changes to the software used in the vehicle system. The testing system also includes a software management unit (MS3) for managing the application of the software to the vehicle. The software management department performs the following processing: In the prepared state, the existing software used in the hardware (51a, 53a, 58, 59) of the vehicle system corresponding to the participating unit is rewritten as test software downloaded from the server (96) to the vehicle system. In the event that a specific participating unit stops the test individually via the individual stop function, the previous software is downloaded from the server, and the test software is restored to the previous software.

3. The testing system according to claim 1, wherein, The test was conducted in conjunction with changes to the software used in the vehicle system. The testing system also includes a software management unit (MS3) for managing the application of the software to the vehicle. The software management department performs the following processing: In the prepared state, in addition to the usual software, test software is downloaded from the server (96) to the vehicle system, which is equipped with hardware (51a, 53a, 58, 59) corresponding to the participating unit. In a state that enables one or both of the prior software and the test software to be effective, the software of both parties shall be retained in the hardware. If the participating unit stops the test, while keeping the software of both parties on the hardware, the previous software is made valid and the test software is made invalid.

4. The testing system according to claim 1, wherein, In the separate stop function of the preparation state, if the participating unit performs a participation intention expression operation for the test through the HMI device, the test will not start for the participating unit until the grace period for canceling the participation intention expression operation has elapsed.

5. The testing system according to claim 1, wherein, The test stop unit obtains consent from a specific participating unit that participation in the test cannot be cancelled within a specified period, and during the specified period, the individual stop function for the specific participating unit is fixed to an invalid state.

6. The testing system according to claim 1, wherein, The term "more than one vehicle" refers to multiple vehicles. The test implementation unit sets the implementation area for the test. In the implementation state, if a specific vehicle among the multiple vehicles moves outside the implementation area, the test stop unit, as the individual stop function, individually stops the test of the participating unit corresponding to the specific vehicle.

7. The testing system according to claim 1, wherein, The term "more than one vehicle" refers to multiple vehicles. The test stop section performs the following processing: Define the conditions for stopping each individual test and the conditions for stopping the entire test. If a specific vehicle among the multiple vehicles meets the conditions for individual stopping, then, as part of the individual stopping function, the test of the participating unit corresponding to that specific vehicle is stopped individually. If the conditions for overall stop are met, the test is stopped entirely as a function of overall stop.

8. The testing system according to claim 7, wherein, The condition for the overall stop is that, among the multiple vehicles, the number or proportion of vehicles that individually stopped the test is above a specified threshold.

9. The testing system according to claim 1, wherein, The test stop section performs the following processing: Set the individual stopping conditions for the test. The method is adjusted to provide notifications to vehicle users regarding the cessation of the test before controlling the vehicle for reasons that would cause the individual stop to occur.

10. The testing system according to claim 1, wherein, When the participating unit stops the test, the test stop unit will issue a notification related to the test being stopped, avoiding the time when the vehicle is in motion.

11. The testing system according to claim 1, wherein, The test stop section performs the following processing: A user interface (UI1) is provided that allows vehicle users to configure how the test is temporarily stopped in the participating units. Based on the settings in the user interface, determine the validity of the separate stop function.

12. The testing system according to claim 11, wherein, If the duration of the temporary stop set in the user interface exceeds a preset period, the test stop unit will implement a reminder notification to remind the vehicle user of the existence of the test.

13. The testing system according to claim 1, wherein, The vehicle in question is used for passenger service. The test is a test that uses the participating units as passenger units based on the passengers. The test stop section performs the following processing: For passenger units that are confirmed to have the intention to refuse the test, the individual stop function is enabled; for other passenger units, the test function is provided.

14. A testing method for testing a vehicle system (2, 2A, 2B) mounted on multiple vehicles (1, 1A, 1B) capable of participating in road traffic, wherein... Includes the following steps: In each of the aforementioned vehicles, one or more metrics are used to determine the effectiveness of the individual stop function, which is tested individually on a vehicle-by-vehicle basis; and Based on the effectiveness of the individual stop functions in the multiple vehicles, it is decided to disable the overall stop function of the test.

15. A software management device comprising at least one processor, for managing software used in a vehicle system (2, 2A, 2B) installed in one or more vehicles capable of participating in road traffic, wherein... The at least one processor performs the following processing: In the test preparation state, in addition to the usual software, test software is downloaded from the server (96) to the vehicle system from the hardware (51a, 53a, 58, 59) provided by the vehicle system. The software of both the conventional software and the test software shall be retained in the hardware in a state that enables one or both of them to be effective; as well as When the vehicle equipped with the vehicle system stops the test, while keeping the software of both parties in the hardware, the previous software is made valid and the test software is made invalid.

Images