System and Program

Abstract

To allow a plurality of automatic operation systems whose technical levels are different from each other or a manager therefor to take an action corresponding to each of the technical levels, according to technical development of the automatic operation systems.SOLUTION: A system (1) using a plurality of automatic operation systems (2) each for controlling traveling of an automatic operation vehicle (3), in different modes from each other, comprises a test part (136) for simulating traveling of the automatic operation vehicle along an operation scenario in which a traveling condition of the automatic operation vehicle and an event generated during traveling are written, and an output part (137) for outputting the simulated result for each of the automatic operation systems.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a system and a program. [Background technology]

[0002] Various techniques for evaluating autonomous driving technologies have been proposed. For example, Patent Document 1 describes a data processing system that performs the following steps: for each of a plurality of driving scenarios, in response to sensor data from one or more sensors of the vehicle and the driving scenario, collecting driving statistics data and environmental data of the vehicle when the vehicle is driven by a driver according to the driving scenario, requesting the driver to select a label for the completed driving scenario and storing the label selected in accordance with the driver's selection, and extracting features including the driving statistics data and environmental data collected at different times during the driving scenario from the driving statistics data and environmental data based on predetermined criteria, and using the extracted features to evaluate the behavior of a plurality of autonomous vehicles. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-185783 Summary of the Invention [Problem to be solved by the invention]

[0004] There are multiple automated driving systems in the world today, and each system has a different level of technology. For this reason, for example, a problem detected during automated driving using a highly technical automated driving system may not be solvable by an automated driving system with a lower level of technology. In other words, to quickly popularize automated driving technology, it is necessary to respond to the system's technical level. [Means for solving the problem]

[0005] A system according to one aspect of the present invention uses a plurality of autonomous driving systems that each control the driving of an autonomous vehicle in a different manner, and includes a test unit that simulates the driving of the autonomous vehicle according to a driving scenario that describes the driving conditions of the autonomous vehicle and events that occur during driving, and an output unit that outputs the results of the simulation for each autonomous driving system.

[0006] Each aspect of the present invention may be realized by a computer, in which case the program that realizes the system on a computer by causing the computer to operate as each part (software element) of the system, as well as the computer-readable recording medium on which it is recorded, also fall within the scope of the present invention. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing an example of the relationship between a system according to an embodiment of the present invention, an autonomous driving system, and a manually driven vehicle. FIG. [Figure 2] FIG. 10 is a block diagram showing another example of the relationship between the system, an automated driving system, and a self-driving vehicle. [Figure 3] FIG. 2 is a block diagram showing a functional configuration of the system according to the embodiment. [Figure 4] This is a diagram showing the process of improving the technical level of autonomous driving systems using this system. [Figure 5] FIG. 10 is a diagram showing the flow of processing performed by the system up to the accumulation of driving scenarios. [Figure 6] 10 is a table showing an example of a manner in which problem events are managed by the system. [Figure 7] 10 is a table showing an example of a management form of an operation scenario by the system. [Figure 8] 10 is a table showing an example of a management form of simulation results by the system. [Figure 9]10 is a table showing an example of a management form of technical level by the system. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of one aspect of the present invention will be described.

[0009] [Autonomous Driving System] Prior to describing the system 1 according to this embodiment, an overview of the multiple autonomous driving systems 2 that are the targets of evaluation by the system 1 will be described.

[0010] The multiple autonomous driving systems 2 each control a corresponding autonomous vehicle 3. Each autonomous driving system 2 according to this embodiment takes the form of, for example, one or more boards on which one or more semiconductor chips are mounted. As shown in FIG. 1, each autonomous driving system 2 is mounted on the autonomous vehicle 3 to be controlled. The autonomous driving system 2 acquires driving data from the autonomous vehicle 3 to be controlled. Details of the driving data will be described later. The autonomous driving system 2 according to this embodiment also acquires driving data from various sensors on the driving route of the autonomous vehicle 3. The sensors on the driving route include at least one of road cameras installed along the driving route and sensors (such as on-board cameras) installed on vehicles other than the autonomous vehicle 3 that are on the driving route. The autonomous driving system 2 also transmits the acquired driving data to system 1 as necessary. Note that the autonomous driving system 2 may be configured as a device independent of the autonomous vehicle 3, as shown in FIG. 2, for example. In this case, each autonomous driving system 2 remotely controls the autonomous vehicle 3 by wirelessly communicating with the autonomous vehicle 3.

[0011] The multiple automated driving systems 2 each control the driving of the automated driving vehicle 3 in a different manner based on the acquired driving data. The manner includes at least one of a control algorithm and a threshold for recognition and judgment. Therefore, each of the multiple automated driving systems 2 may have a different level of automated driving technology. The level of automated driving technology can also be said to be the likelihood of problems occurring while the automated driving vehicle 3 is driving. Problems while driving include, for example, traffic accidents, deviation from traffic rules, sudden stops, etc.

[0012] [Autonomous vehicles] Next, an overview of the automatically driven vehicle 3 controlled by the above-mentioned multiple automatically driven systems will be described.

[0013] The autonomous vehicle 3 collects various driving data and transmits it to the autonomous driving system 2. The driving data includes at least one of sensor data, voice data, and telemetry data. The sensor data is data obtained by detecting events occurring inside and / or outside the autonomous vehicle 3 using various sensors equipped in the autonomous vehicle 3. The sensor data includes image data generated by a camera (image sensor). The voice data is data obtained by collecting voices uttered by passengers in the autonomous vehicle 3, sounds emitted by the autonomous vehicle 3, and sounds around the autonomous vehicle 3 using a microphone equipped in the autonomous vehicle 3. The voice data is data obtained by digitizing the voices uttered by passengers in the autonomous vehicle 3, sounds emitted by the autonomous vehicle 3, and sounds around the autonomous vehicle 3. The passengers in the autonomous vehicle 3 include the driver and passengers. The telemetry data is data of measured values measured and output by measuring equipment equipped in the autonomous vehicle 3. The measured values include driving speed, acceleration, battery and fuel levels, etc.

[0014] Furthermore, autonomous vehicles 3 are used for a variety of operation types. Operation types include shuttle service, delivery, mobile sales, etc. Shuttle service includes on-demand service and regular route service. Furthermore, autonomous vehicles 3 can be configured with vehicles of various types (shapes and sizes). Autonomous vehicle 3 types include, for example, passenger cars, taxis, buses, trucks, and various commercial vehicles. Depending on the type of autonomous vehicle 3, there are operation types that are suitable for some and not others.

[0015] [system] Next, the system 1 according to this embodiment will be described in detail.

[0016] {System configuration} The system 1 is for evaluating the multiple autonomous driving systems 2. The system 1 may include the autonomous driving system 2 of each vehicle, may be incorporated into the autonomous driving system 2 or the autonomously driven vehicle 3, or may be configured as a device independent of these. As described above, the autonomous driving system 2 according to this embodiment is in the form of a circuit board. Therefore, the system 1 incorporated into the autonomous driving system 2 refers to the circuit board. If the autonomous driving system 2 is independent of the autonomously driven vehicle 3, the system 1 including the autonomous driving system 2 or the system 1 incorporated into the autonomous driving system 2 becomes a device (computer) that functions as both the system 1 and the autonomous driving system 2. Also, as described above, the autonomous driving system 2 according to this embodiment is mounted on the autonomously driven vehicle 3. Therefore, the system 1 incorporated into the autonomously driven vehicle 3 refers to a state in which the autonomous driving system 2, an ECU (Engine Control Unit), and other hardware are connected to each other and are all enclosed within a single piece of hardware called the autonomously driven vehicle 3. For convenience, the following description will be given taking as an example a case in which the system 1 is configured as a computer independent of the autonomous driving system 2 and the autonomously driven vehicle 3.

[0017] 3, the system 1 according to this embodiment includes a communication unit 11, a storage unit 12, and a control unit 13. The units 11 to 13 of the system 1 may be integrated into one device (the present invention is also referred to as a system in this case), or may be distributed across multiple devices.

[0018] [Communications Department] The communication unit 11 communicates with the autonomous driving system 2. The communication unit 11 according to this embodiment is configured as a communication module. The communication unit 11 may be configured to be able to communicate with an administrator of the autonomous driving system. The administrator of the autonomous driving system includes communication devices owned by the administrator, a device that trains (updates) the trained model used by the autonomous driving system 2, and the like. The communication unit 11 may also be configured to be able to communicate directly with the autonomous driving vehicle 3.

[0019] [Storage section] The storage unit 12 stores a program 121 that describes the operation of the control unit 13 of the system 1. The storage unit 12 also stores multiple types of databases (DBs). The multiple types of DBs include a record DB 122, a problem event DB 123, a scenario DB 124, a simulation result DB 125, and a technology level definition DB 126. Details of each DB will be described later. The storage unit 12 according to this embodiment is configured with a semiconductor memory, a hard disk, etc. The system 1 may also be provided with multiple storage units. In this case, each DB may be stored in a distributed manner in two or more storage units. The storage unit 12 may also be an independent storage device.

[0020] [Control Unit] The control unit 13 includes an acquisition unit 131, a reception unit 132, an identification unit 133, a discrimination unit 134, a generation unit 135, a test unit 136, an output unit 137, a second output unit 138, a determination unit 139, and a decision unit 140. The control unit 13 according to this embodiment is configured with a processor. When this processor executes the program 121 stored in the storage unit 12, the program 121 causes the control unit 13 to function as each of the control blocks 131 to 140. The system 1 may include multiple control units. In this case, the control units may be integrated into one device or distributed across multiple devices. In this case, the control blocks 131 to 140 may be distributed across two or more control units.

[0021] (Acquisition Department) As shown in Fig. 4, the acquisition unit 131 acquires driving data from the autonomous driving system 2 via the communication unit 11 (S1). The driving data acquired by the acquisition unit 131 according to this embodiment includes data collected by the autonomous driving vehicle 3 itself while it is driving in a predetermined demonstration environment, as well as information detected by sensors on the driving route while the autonomous driving vehicle 3 is driving. The demonstration environment refers to a combination of the autonomous driving system 2 to be used, the autonomous driving vehicle 3 to be used, the road type of the road on which the autonomous driving vehicle 3 is to be driven, and the operation type of the autonomous driving vehicle 3. The acquisition unit 131 according to this embodiment stores the acquired driving data in the recording DB 122 (S2).

[0022] (Reception Department) The reception unit 132 receives input of information provided by at least one of a person in the autonomous vehicle 3 and a supervisor of the autonomous vehicle 3 (S3). The reception unit 132 according to this embodiment receives input of information in the form of a message from the driver to the administrator of the autonomous driving system, a daily report (problem report), a passenger questionnaire, or the like, via character input means (e.g., a keyboard, scanner, etc.) not shown. Note that the reception unit 132 may be configured to receive, as information, speech uttered by a person in the autonomous vehicle 3 or a supervisor of the autonomous vehicle 3. Furthermore, if the identification unit 133 (described later) is configured to identify a problem event based only on driving data, the control unit 13 does not need to include the reception unit 132.

[0023] (Specific part) The identification unit 133 identifies problematic events that occurred when the autonomously driven vehicle 3 traveled in the past, based on the travel data (S4). Specifically, by executing various processes S41 to S47 as shown in Fig. 5, when the identification unit 133 detects at least one of the following events from the travel data, it identifies the event as a problematic event. - Telemetry data exceeds a predetermined threshold (for example, the speed exceeds the speed limit, or the battery / fuel level falls below a predetermined level) (S41). The degree of change over time in the telemetry data exceeds a predetermined threshold (for example, a sudden decrease in speed or an increase in acceleration due to hard braking, a sudden change in direction due to abrupt steering, or a sudden decrease in the battery or fuel level) (S42). The distance between the object detected based on the sensor data and the autonomous vehicle 3 falls below a predetermined threshold (for example, the autonomous vehicle 3 and the object get too close) (S43). The relative relationship between the object detected based on the sensor data and the autonomous vehicle 3 differs from the predetermined relative relationship (for example, the autonomous vehicle 3 is traveling in the wrong direction on the road, traveling in an area that is not a road, or crossing lanes) (S44). The volume of the audio based on the audio data exceeds a predetermined threshold (S45). The sound based on the audio data was a dangerous sound (for example, sudden braking, collision, or a person's scream) (S46).

[0024] When the identification unit 133 identifies the problem event as the distance between an object and the autonomously driven vehicle 3 falling below a threshold, the identification unit 133 may further have a function of detecting an object around the autonomously driven vehicle 3 based on image data and calculating the distance between the object and the autonomously driven vehicle 3 (S47). The object detection process can be performed, for example, by applying the image data to a trained model constructed by machine learning the relationship between the image data and the object depicted in the image. The detection result may be stored in the storage unit 12 (S48). When the identification unit 133 identifies the problem event as the classification of a sound as a dangerous sound, the identification unit 133 may further have a function of classifying the type of sound. The sound can be classified, for example, by applying the sound data to a trained model constructed by machine learning the relationship between the sound data and dangerous sounds. The identification unit 133 may also be configured to predict an event that will occur after the driving data is obtained, based on the driving data, and, if the predicted event corresponds to one of the events described above, identify the predicted event as a problem event (S49).

[0025] Furthermore, when the receiving unit 132 receives input of information, the identifying unit 133 according to this embodiment can identify a problematic event based on the information, as shown in FIGS. 4 and 5. Specifically, the identifying unit 133 analyzes the text information or audio received by the receiving unit 132 (searches for specific words or phrases from the text information or audio). If a specific word or phrase is included in the text information or audio, the identifying unit 133 identifies the content of the received information as a problematic event. This makes it possible to identify, as a problematic event, an event that could not be detected by a sensor or the like of the autonomously driven vehicle 3, an event that the extraction unit was unable to fully extract, or an event that the discrimination unit 134 has determined to be less severe than the actual event, and that a person finds strange.

[0026] Furthermore, the identification unit 133 according to this embodiment associates the identified problematic event with at least one of the identification information of the autonomous driving system 2 that controlled the autonomous driving vehicle 3, the vehicle model of the autonomous driving vehicle 3, and the operation type. The identification unit 133 according to this embodiment further associates the identified problematic event with the presence or absence of an accident and the event type. This allows the testing unit 136 to perform a simulation using only driving scenarios with settings that differ from the information associated with the problematic event (this avoids the waste of simulating the same content as in a demonstration experiment using a scenario based on driving data).

[0027] (Discrimination part) The determination unit 134 determines the level (rank) of the identified problem event. Specifically, when the content of the problem event corresponds to the following, the determination unit 134 determines the level of the problem event as follows. Traffic accidents (contact between autonomous vehicle 3 and other traffic participants or buildings, etc.) were judged to be the most severe (Rank A). -Violation of traffic laws by the autonomous vehicle itself (for example, speeding, driving in the wrong direction, etc.) is judged to be the second highest level (rank B or C) after traffic accidents. - The fall of a person inside an autonomous vehicle 3 due to sudden braking is judged to be the second highest level of seriousness (rank D) after violating traffic laws. - Sudden braking due to violation of traffic laws by other traffic participants (for example, speeding, driving in the wrong direction, etc.) is judged to be the second most serious (rank E) after traffic accidents. - Sudden braking caused by a change in the vehicle's position was judged to be the second most serious problem (rank F) after a person falling over.

[0028] The determination unit 134 then stores the various pieces of information associated by the identification unit 133 with the driving data for which the degree of the problem event has been determined in the problem event DB 123 of the storage unit 12 (S5). At this time, the determination unit 134 assigns an event number and a storage date and time to each piece of driving data. Therefore, the data in the problem event DB 123 is organized as a list as shown in FIG. 6. This makes it possible to extract driving data corresponding to a combination of the degree of any problem event, the presence or absence of an accident, the road type, the event type, the identification information of the autonomous driving system 2, the model of the autonomously driven vehicle 3, and the operation type from among multiple pieces of driving data stored in the past. The determination unit 134 may be configured to automatically determine the degree of the problem event identified based on the information received by the reception unit 132 to be a predetermined level (e.g., rank A) or higher. Furthermore, if the generation unit 135 (described later) is configured to generate a driving scenario regardless of the degree of the problem event, the control unit 13 does not need to include the determination unit 134.

[0029] (Generation part) As shown in FIG. 4, the generation unit 135 generates a driving scenario based on the driving data (S6). The driving scenario is data that defines the operation of a simulation and describes the driving conditions of the autonomous vehicle 3 and events that occur during driving. The driving conditions include at least one of the driving route, location, road type, weather, vehicle type, and operation type. Events that occur during driving include a traffic participant other than the autonomous vehicle 3 jumping onto the road, the autonomous vehicle 3 or a traffic participant driving in the wrong direction, and the autonomous vehicle 3 exceeding the speed limit. The generation unit 135 according to this embodiment generates a driving scenario using widely available scenario creation software, such as ASAM. This makes it possible to easily generate driving scenarios that can be incorporated into each autonomous driving system 2 operated by various companies. The generation unit 135 may also be configured to generate driving scenarios using a scenario creation program dedicated to the system 1.

[0030] As described above, the driving data used to generate a driving scenario (acquired by the acquisition unit 131) includes not only data collected by the autonomously driven vehicle 3 itself, but also information detected by sensors on the driving route while the autonomously driven vehicle 3 is traveling. This allows the generation unit 135 to generate a driving scenario whose content is closer to the actual driving environment.

[0031] The generation unit 135 according to this embodiment generates a driving scenario based on driving data from which problematic events of a predetermined level or higher have been extracted. As a result, scenarios are generated by narrowing down the driving data to those in which the level of problematic events is a predetermined level or higher, which further reduces the number of driving scenarios to be generated and the number of simulations. As a result, the time and cost required to improve the autonomous driving system 2 can be further reduced. The generation unit 135 may be configured to adjust the level of problematic events that serves as the criterion for whether or not to generate a driving scenario, depending on user operations, the number of problematic events stored in the database, etc.

[0032] Furthermore, as shown in FIG. 5, the generation unit 135 according to this embodiment identifies the occurrence timing of a problem event based on the driving data before generating a driving scenario. The generation unit 135 according to this embodiment first refers to the problem event DB 123 to determine whether the problem event was caused solely by the subject vehicle (S61). If it is determined that the problem event was caused solely by the subject vehicle (YES in S61), the generation unit 135 identifies the timing at which a change in the subject vehicle was detected as the occurrence timing of the problem event (S62). On the other hand, if it is determined that the problem event was not caused solely by the subject vehicle (other traffic participants are involved) (NO in S61), the generation unit 135 identifies the timing at which another traffic participant was detected as the occurrence timing of the problem event (S63). Then, the generation unit 135 calculates the relationship between the subject vehicle, the traffic participants, and the shape of the road (S64). Thereafter, a driving scenario is generated based on the calculated relationship (S65).

[0033] The generation unit 135 also classifies the generated driving scenarios according to at least one of location, road type, and weather. Road types include general roads and expressways. General roads include national highways, prefectural roads, and municipal roads. Weather includes weather and temperature. The generation unit 135 according to this embodiment classifies driving scenarios by location, road type, and weather. This makes it easy to select a driving scenario that corresponds to the desired simulation. The generation unit 135 stores the generated and classified driving scenarios in the scenario DB 124 (S7). At that time, the generation unit 135 assigns a scenario number and an event number to each driving scenario. Therefore, the data in the scenario DB 124 is a list as shown in FIG. 7. This makes it possible to extract a scenario that corresponds to any combination of location, road type, weather, and temperature from multiple scenarios stored in the past.

[0034] (Testing Department) As shown in FIG. 4 , the test unit 136 uses multiple autonomous driving systems 2 to simulate the driving of the autonomous vehicle 3 according to driving scenarios (S8). The test unit 136 according to this embodiment simulates the driving of the autonomous vehicle 3 according to driving scenarios based on driving data obtained when the autonomous vehicle 3 was previously controlled by the autonomous driving system 2 and drove in real space. This uses scenarios based on driving data obtained when the autonomous vehicle 3 actually drove under the control of the target autonomous driving system 2, making it possible to reduce the number of driving scenarios to be generated and the number of simulations. As a result, it is possible to reduce the time and cost required to improve the autonomous driving system 2. The simulation is performed by setting a demonstration environment different from the demonstration environment used when the driving data was previously obtained (by changing at least one of the autonomous driving system 2, the autonomous vehicle 3, the road type, and the operation type).

[0035] When a simulation is performed using an autonomous driving system 2 with a high level of technology to control an autonomously driven vehicle 3 suitable for the autonomous driving system 2, few problems occur during the simulation, and the problem is often of low severity. On the other hand, when a simulation is performed using an autonomous driving system 2 with a low level of technology, many problems occur during the simulation, and the problem is often of high severity. Also, when a simulation is performed using an autonomous driving system 2 with a high level of technology but an autonomously driven vehicle 3 that is not fully compatible with it, many problems occur during the simulation, and the problem is often of high severity. If a problem of a predetermined level or higher occurs, the test unit 136 according to this embodiment marks the result of the simulation as "X", indicating that the result is a bad result. On the other hand, if no problem of a predetermined level or higher occurs, the test unit 136 marks the result of the simulation as "O", indicating that the result is a good result.

[0036] The test unit 136 according to this embodiment classifies the results of the simulation according to the identification information of the autonomous driving system used and the vehicle type of the autonomously driven vehicle 3. Furthermore, the test unit 136 according to this embodiment stores the results of the simulation in the simulation result DB 125 (S9). At that time, the test unit 136 assigns a scenario number and the test date and time to each simulation result. Therefore, the data in the simulation result DB 125 is listed as shown in FIG. 8. This makes it possible to extract a result according to a combination of identification information of an arbitrary autonomous driving system 2 and an autonomously driven vehicle 3 from among multiple simulation results stored in the past.

[0037] (output section) As shown in Fig. 4, the output unit 137 outputs the results of the simulation for each autonomous driving system 2 (S10). The results of the simulation include a notice that a problem occurred in the simulation, the details of the problem that occurred, a request to fix the problem, the details of the fix, etc. The output includes displaying the simulation results on a display device (not shown), controlling the communication unit 11 or a connection terminal to send the simulation result data to an external device, storing the simulation result data in a medium, etc. The external device includes the autonomous driving system 2, an administrator of the autonomous driving system, etc.

[0038] (Second output unit) The second output unit 138 outputs information for displaying how the test unit 136 is performing the simulation. The information for displaying how the test unit 136 is performing the simulation is, for example, recorded data of the simulation. The output includes the same form as that of the output unit 137. The form of output by the second output unit 138 may be the same as or different from that of the output unit 137. When the output is displayed on a display device, the state of the simulation is reproduced on the display device. By providing such a second output unit 138, it is possible to check not only the simulation results but also the process of the performed simulation. As a result, it is possible to easily obtain information useful for improving the autonomous driving system 2.

[0039] (Judgment Department) The determination unit 139 determines the technical level of each of the multiple autonomous driving systems 2 based on the results of the simulation (S11). The determination unit 139 according to this embodiment determines the technical level of each combination of the multiple autonomous driving systems 2 and the controlled autonomously driven vehicle 3. Specifically, when the simulation results meet the following criteria, the determination unit 139 determines the technical level of the combination of the autonomous driving system 2 and the autonomously driven vehicle 3 as follows: The combination that caused no problems in the simulation was judged to have the highest technical level (Level A). Combinations that caused minor problems in the simulation were judged to be at the second highest technical level (Level B). The combination in which minor problems frequently occurred in the simulation was judged to be the third highest technical level (Level C). - Combinations that caused serious problems in the simulation were judged to be at the lowest technical level (Level D).

[0040] The determination unit 139 stores the determination results of the technical level in the technical level definition DB 126. At that time, the determination unit 139 according to this embodiment sets an offering price for the combination of the autonomous driving system 2 and the autonomously driven vehicle 3 for which the technical level has been determined. The determination unit 139 may set a predetermined amount according to the technical level, or may calculate the offering price based on various factors (for example, development costs, expected sales volume, etc.). For this reason, the data in the technical level definition DB 126 is a list such as that shown in FIG. 9. This makes it possible to extract a combination corresponding to an arbitrary combination of technical level and offering price from among previously stored combinations of autonomous driving systems 2 and autonomously driven vehicles 3.

[0041] (Decision section) The determination unit 140 determines a combination of the vehicle type of the autonomous driving vehicle 3 and the autonomous driving system 2 that corresponds to the desired driving conditions, based on the technical levels of each of the multiple autonomous driving systems 2 (S12). Driving conditions include road type, operation type, etc. The determination unit 140 according to this embodiment determines the combination of the vehicle type of the autonomous driving vehicle 3 and the autonomous driving system 2 based on the technical level and offering price of each combination of the autonomous driving system 2 and the autonomous driving vehicle 3. Specifically, the determination unit 140 references the technical level definition DB 126 and selects the combination with the highest technical level among the combinations of the autonomous driving system 2 and the autonomous driving vehicle 3 whose offering price is within the budget. This sets up a new demonstration environment using the selected combination for the new road type and operation type.

[0042] The appropriate combination of the vehicle type of autonomous vehicle 3 and the autonomous driving system 2 varies depending on the driving conditions, but until now, there has been a mismatch in the combination. However, by adopting the determined combination, the combination of vehicle type and autonomous driving system 2 becomes optimal (the one least likely to cause problems at this stage). Furthermore, the driving data obtained from the optimally combined autonomous driving vehicle 3 may contain problematic events that have not previously occurred. By generating driving scenarios based on this driving data and performing simulations, the technical level of the autonomous driving system 2 can be further improved.

[0043] {System effects} According to the system 1 described above, simulation results for each of multiple automated driving systems 2 with different control modes (technical levels) are output to each automated driving system 2. This allows multiple automated driving systems 2 with different technical levels or their managers to take appropriate measures in the technological development of the automated driving systems according to their respective technical levels. Specifically, an automated driving system 2 with a certain technical level or its manager or designer can efficiently recognize problems in the automated driving system 2 that correspond to its own technical level rather than a different technical level, and can efficiently invest in improvements. As a result, the technical level of an automated driving system 2 with a relatively low technical level is efficiently raised, reducing the variation in the technical levels of the automated driving systems of the many automated driving vehicles on the road. Therefore, for example, it can be expected that all automated driving vehicles equipped with any automated driving system will have a certain level of safety, leading to the realization of a society in which automated driving is more widespread.

[0044] Furthermore, such effects will also contribute to achieving the Sustainable Development Goals (SDGs) advocated by the United Nations, such as Goal 3 "Good health and well-being for all," Goal 9 "Industry, innovation and infrastructure," and Goal 11 "Sustainable cities and communities."

[0045] [Variations] It should be noted that some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0046] [summary] The system according to aspect 1 of the present invention is configured to use a plurality of autonomous driving systems that each control the driving of an autonomous vehicle in a different manner, and to include a test unit that simulates the driving of the autonomous vehicle in accordance with a driving scenario that describes the driving conditions of the autonomous vehicle and events that occur during driving, and an output unit that outputs the results of the simulation for each autonomous driving system.

[0047] A system according to aspect 2 of the present invention may be configured in the above-mentioned aspect 1 such that the test unit simulates the driving of the autonomous vehicle in accordance with a driving scenario based on driving data obtained when the autonomous vehicle was previously controlled by the autonomous driving system and drove in real space.

[0048] The system according to aspect 3 of the present invention, in the above-mentioned aspect 1 or 2, may further include a generation unit that generates the driving scenario based on the driving data, and the generation unit may be configured to classify the generated driving scenario according to at least one of location, road type, and weather.

[0049] The system according to aspect 4 of the present invention may be configured in accordance with aspect 3 above, further comprising an identification unit that identifies problematic events that occurred when the autonomous vehicle was traveling in the past based on the traveling data, and a discrimination unit that discriminates the severity of the identified problematic events, and the generation unit may be configured to generate the driving scenario based on the traveling data from which problematic events of a predetermined severity or higher have been extracted.

[0050] A system according to aspect 5 of the present invention may be configured such that, in aspect 4 above, the identification unit associates the identified problem event with at least one of identification information of the autonomous driving system that controlled the autonomous vehicle, the vehicle model of the autonomous vehicle, and the operation type.

[0051] The system according to aspect 6 of the present invention may be configured in any of aspects 1 to 5 above, further comprising a reception unit that receives input of information provided by at least one of a person riding in the autonomous vehicle and a monitor of the autonomous vehicle, and the identification unit is capable of identifying the problem event based on the information.

[0052] A system according to aspect 7 of the present invention may be configured in any one of aspects 2 to 6 above, wherein the driving data includes information detected by a sensor on the driving route while the autonomous vehicle is traveling.

[0053] The system according to aspect 8 of the present invention may be configured in any of aspects 1 to 7 above, further comprising a judgment unit that judges the technical level of each of the plurality of autonomous driving systems based on the results of the simulation, and a determination unit that determines the vehicle type of the autonomous driving vehicle in accordance with any driving conditions and the combination with the autonomous driving system based on the technical level of each of the plurality of autonomous driving systems.

[0054] A system according to a ninth aspect of the present invention may be configured in any one of the first to eighth aspects above, further comprising a second output unit that outputs information for displaying how the test unit is performing a simulation. [Explanation of symbols]

[0055] 1 System 11 Communications Department 12 Storage section 121 Programs 121 Record Database 122 Problem Event Database 123 Scenario Database 124 test result database 125 Simulation Results Database 126 Technology Level Definition Database 13 Control Unit 131 Acquisition Department 132 Reception Department 133 Specific part 134 Discrimination part 135 Generation part 136 Testing Department 137 Output section 138 Second output section 139 Judgment section 140 Decision Section 2. Autonomous Driving System 3. Self-driving vehicles

Claims

1. a test unit that uses a plurality of autonomous driving systems that each control the driving of an autonomous vehicle in a different manner, and simulates the driving of the autonomous vehicle according to a driving scenario that describes the driving conditions of the autonomous vehicle and events that occur during driving; an output unit that outputs the results of the simulation for each of the autonomous driving systems; a reception unit that receives input of information provided by at least one of a person riding in the autonomously driven vehicle and a monitor of the autonomously driven vehicle; an identification unit that identifies a problem event that occurred while the autonomously driven vehicle was traveling based on the information; and a generation unit that generates a new driving scenario based on driving data obtained when the autonomously driven vehicle is traveling, the driving data from which the problem event is extracted; and Equipped with The information includes a message or daily report from the driver of the autonomous vehicle to the administrator of the autonomous driving system, a questionnaire of passengers of the autonomous vehicle, or a voice uttered by at least one of a person riding in the autonomous vehicle and a monitor of the autonomous vehicle. system.

2. the test unit simulates driving of the autonomously driven vehicle in accordance with a driving scenario based on driving data obtained when the autonomously driven vehicle was previously controlled by the autonomous driving system and drove in a real space; The system of claim 1 .

3. The generation unit classifies the generated driving scenarios according to at least one of location, road type, and weather. The system of claim 2 .

4. Further comprising a determination unit that determines the severity of the identified problem phenomenon; the generation unit generates the driving scenario based on the traveling data from which the problem event having a predetermined degree or higher has been extracted. The system of claim 3 .

5. The identification unit associates the identified problem event with at least one of identification information of the autonomous driving system that controlled the autonomous driving vehicle, a vehicle model of the autonomous driving vehicle, and an operation type. The system of claim 4.

6. The driving data includes information detected by sensors on a driving route while the autonomous vehicle is driving. The system of claim 2 .

7. a determination unit that determines the technical level of each of the plurality of automated driving systems based on the results of the simulation; a determination unit that determines a combination of the vehicle type of the autonomous driving vehicle and the autonomous driving system according to any driving conditions based on the technical levels of the autonomous driving systems; Further comprising: The system of claim 1 .

8. a second output unit that outputs information for displaying how the test unit performs a simulation; The system of claim 1 .

9. A test unit that uses a plurality of autonomous driving systems that each control the driving of an autonomous vehicle in a different manner, and simulates the driving of the autonomous vehicle according to a driving scenario that describes the driving conditions of the autonomous vehicle and events that occur during driving; an output unit that outputs the results of the simulation for each of the autonomous driving systems; a determination unit that determines the technical level of each of the plurality of automated driving systems based on the results of the simulation; a determination unit that determines a combination of the vehicle type of the autonomous driving vehicle and the autonomous driving system according to any driving conditions based on the technical levels of the autonomous driving systems; Equipped with system.

10. 2. A program for causing a computer to function as the system according to claim 1, the program causing a computer to function as the testing unit, the output unit, the receiving unit, the identifying unit, and the generating unit.

Images