Design support method, design support system, and program

Abstract

Provided are a design support method, a design support system, and a program with which it is possible to design an ODD that suppresses an increase in the development cost of an automatic driving service and improves the important performance evaluation index of an enterprise owner. A design support system for acquiring customer requirements and environmental data from an environmental measurement device for measuring the environment of a service provision area managed by a customer, and outputting an improvement value of an important performance evaluation index of the customer, the design support system being characterized by being provided with: an automatic driving control function database for providing an automatic driving control function for the customer; the server stores a program for performing automatic driving control on a vehicle traveling in a service providing area. A regional environment data generation unit provided with a regional digital map generation unit for generating a digital map of the service provision region using the environment data; an automatic driving ODD automatic definition unit that generates and outputs an operation design domain, which is an environmental restriction condition for operating an automatic driving service, on the basis of information on the digital map; an ODD adjustment unit that evaluates the sufficiency of the automatic driving control function specification stored in pairs with the automatic driving control program with respect to the operation design domain output from the automatic driving ODD automatic definition unit, and changes the operation design domain on the basis of the evaluation result; an automatic driving control system evaluation unit that evaluates the operation design domain on the basis of customer requirements; and a KPI evaluation unit which, on the basis of customer requirements, generates a customer requirement evaluation formula for evaluating the customer's important performance evaluation index, calculates the customer requirement evaluation formula on the basis of the value output from the automatic driving control system evaluation unit, and outputs the operation design domain and the customer's important performance evaluation index. The ODD adjustment unit changes the operation design field on the basis of the value of the customer demand evaluation formula calculated and output by the KPI evaluation unit so that the value of the customer demand evaluation formula is higher than the value of a previous customer demand evaluation formula, and the KPI evaluation unit outputs an improved value of the customer's important performance evaluation index on the basis of the changed operation design field.

Description

Technical Field

[0001] The present invention relates to a design support method, a design support system and a program for an operational design domain (ODD) of a control system for automatically driving a transport robot, a car or other working vehicles. Background Art

[0002] In recent years, control systems have been developed to realize the automatic driving of automobiles and various industrial vehicles such as handling robots and construction machinery. Through the automation of industrial vehicles, it is expected that the cost will be reduced by unmanned work and the work efficiency will be improved by making the work efficiency uniform. In addition, the automatic driving of automobiles is expected to reduce traffic congestion and accidents.

[0003] The configuration and specifications of the control system that realizes automation vary depending on the target vehicle and the operating environment. Since its design requires professional skills such as how to balance the vehicle's control characteristics with its suitability for the environment, it takes a lot of man-hours.

[0004] In particular, in the process of adjusting the vehicle's control characteristics and its suitability for the environment, it is necessary to define the environmental conditions that guarantee the operation of the autonomous driving control system, namely the ODD. Development engineers must investigate the environment in which the autonomous driving vehicle will be introduced and confirm its compatibility with the specifications of the autonomous driving control system, which also requires huge development costs.

[0005] As a method for defining and updating ODD, a design support system described in Patent Document 1 is known. Patent Document 1 describes a technology for generating a data set based on an environmental map and defining an ODD for a conditional space based on a safety evaluation of an autonomous driving control system. In addition, the defined ODD can be updated when additional data is received. With such a design support system, ODD can be automatically defined based on an environmental map without relying on personal expertise, thereby reducing the number of people required for development and reducing development costs.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: WO2021147301A1 Summary of the invention

[0009] According to the design support system described in Patent Document 1, which defines ODD based on reliability evaluation (such as operation rate) based on environmental data sets, there is a situation where the required autonomous driving control system becomes highly functional for the ODD that matches the environment in which the autonomous driving service is provided, but does not satisfy important performance evaluation indicators presented by the business owner based on the realization of the autonomous driving service, such as an increase in development costs and a decrease in operation rate.

[0010] Therefore, the present invention aims to provide a design support method, a design support system and a program that can design an ODD that can suppress the increase in development costs of autonomous driving services and improve important performance evaluation indicators of business owners.

[0011] Based on the above, the present invention provides "a design support system that obtains customer requirements and environmental data from an environmental measuring device that measures the environment of a service provision area managed by the customer, and outputs improvement values ​​of important performance evaluation indicators of the customer, and the above-mentioned design support system is characterized in that it has: an automatic driving control function database, which stores a program for automatic driving control of a vehicle traveling in a service provision area; a regional environmental data generating unit, which has a regional digital map generating unit that uses environmental data to generate a digital map of the service provision area; an automatic driving ODD automatic definition unit, which generates and outputs an operation design domain based on information on the digital map, and the operation design domain is an environmental constraint condition for operating an automatic driving service; an ODD adjustment unit, which adjusts the operation design domain output from the automatic driving ODD definition unit , evaluates the adequacy of the automatic driving control function specifications stored in pairs with the automatic driving control program, and changes the operation design domain based on the evaluation results; an automatic driving control system evaluation unit, which evaluates the operation design domain based on customer requirements; and a KPI evaluation unit, which generates a customer requirement evaluation formula for evaluating the customer's important performance evaluation indicators based on customer requirements, calculates the customer requirement evaluation formula based on the value output from the automatic driving control system evaluation unit, and outputs the operation design domain and the customer's important performance evaluation indicators, the ODD adjustment unit applies changes to the above-mentioned operation design domain based on the value of the customer requirement evaluation formula calculated and output by the KPI evaluation unit in a manner that improves the value of the customer requirement evaluation formula compared to the previous value of the customer requirement evaluation formula, and the KPI evaluation unit outputs the improvement value of the customer's important performance evaluation indicator based on the changed operation design domain. "

[0012] In addition, the present invention provides "a design support method, which obtains customer requirements and environmental data from an environmental measuring device that measures the environment of a service provision area managed by the customer, and uses a computer device to output improvement values ​​of important performance evaluation indicators of the customer. The above-mentioned design support method is characterized in that it has an automatic driving control function database, which stores an automatic driving control program for automatic driving control of a vehicle traveling in the service provision area, and the computing unit of the computer device uses the environmental data to generate a digital map of the service provision area, and generates an operation design domain that becomes an environmental constraint condition for operating the automatic driving service based on the information on the digital map, for An operation design domain is provided, evaluating the adequacy of the automatic driving control function specification stored in a pair with the automatic driving control program, changing the operation design domain based on the evaluation result, evaluating the operation design domain based on the customer's requirements, and generating a customer requirement evaluation formula for evaluating the customer's important performance evaluation index based on the customer's requirements, calculating the customer requirement evaluation formula based on the value output from the automatic driving control system evaluation unit, obtaining the operation design domain and the customer's important performance evaluation index, and applying a change to the operation design domain based on the value of the customer requirement evaluation formula so that the value of the customer requirement evaluation formula is improved compared to the previous value of the customer requirement evaluation formula, and obtaining an improvement value of the customer's important performance evaluation index based on the changed operation design domain.

[0013] In addition, the present invention provides "a design support program for obtaining customer requirements and environmental data from an environmental measuring device that measures the environment of a service provision area managed by the customer, and outputting improvement values ​​of important performance evaluation indicators of the customer, wherein the above-mentioned design support program is characterized in that it has the following programs: an automatic driving control function program, which is an automatic driving control program for automatic driving control of a vehicle traveling in a service provision area; a regional environmental data generation program, which has a regional digital map generation unit that uses environmental data to generate a digital map of the service provision area; an automatic driving ODD automatic definition program, which generates and outputs an operation design domain based on information on the digital map, and the operation design domain is an environmental constraint condition for operating an automatic driving service; an ODD adjustment program, which evaluates and automatically adjusts the operation design domain. The driving control program evaluates the adequacy of the automatic driving control function specifications stored in pairs, and changes the operation design domain based on the evaluation results; an automatic driving control system evaluation program that evaluates the operation design domain based on customer requirements; and a KPI evaluation program that generates a customer requirement evaluation formula for evaluating the customer's important performance evaluation index based on customer requirements, calculates the customer requirement evaluation formula based on the value output from the automatic driving control system evaluation program, and outputs the operation design domain and the customer's important performance evaluation index, the ODD adjustment program applies changes to the operation design domain based on the value of the customer requirement evaluation formula calculated and output by the KPI evaluation program in such a way that the value of the customer requirement evaluation formula is improved compared to the previous value of the customer requirement evaluation formula, and the KPI evaluation program outputs the improvement value of the customer's important performance evaluation index based on the changed operation design domain. "

[0014] Effects of the Invention

[0015] According to the present invention, compared with the conventional method of providing autonomous driving only under the ODD based on the environment in which the autonomous driving service is provided, a design support system can be provided that can automatically design the ODD that can improve the important performance evaluation index of the business owner. Thus, the autonomous driving service that improves the important performance evaluation index of the business owner can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a diagram showing a configuration example of a design support system according to the first embodiment of the present invention.

[0017] Figure 2 This is a diagram showing an example of an automatic transport area that provides cargo transport services.

[0018] Figure 3 This is a diagram showing an example of a shape digital map.

[0019] Figure 4 This is a diagram showing an example of the data structure of the automatic driving control function database.

[0020] Figure 5 This is a diagram showing an example of a traceability table showing the relationship between autonomous driving control function data, ODD items, and KPI values.

[0021] Figure 6 It is a diagram showing the flow of the design support method according to the first embodiment of the present invention.

[0022] Figure 7 This is a diagram showing an example of utilization of the output of the design support system 1 .

[0023] Figure 8 This is a diagram showing an example of a work site where cargo transportation services are provided.

[0024] Fig. 9 This is a diagram showing an example of a changed environment at a work site.

[0025] Fig.10 FIG. 1 is a diagram showing an example of a general public road environment in which a cargo transportation service is provided.

[0026] Fig.11 This is a diagram showing an example of a traceability table. DETAILED DESCRIPTION

[0027] Embodiments of the present invention will be described below using the drawings. The design support method, design support system, and program of the embodiments of the present invention will be described using an example of designing an automatic transport service for goods in a logistics warehouse using an automatic transport vehicle.

[0028] Furthermore, the present invention can be applied to the design of the entire service related to a machine that can travel automatically in a specific environment, and the application of the present invention is not limited to the autonomous transport vehicle service.

[0029] Hereinafter, the basic structure of the design support method and the design support system of the embodiment of the present invention will be described in Example 1, the case where the service provision area is a work site will be described in Example 2, the case where the service provision area is a general public road will be described in Example 3, and the procedure of the design support system will be described in Example 4.

[0030] Example 1

[0031] Figure 1 1 is a diagram showing a configuration example of a design support system 1 according to an embodiment of the present invention. In the first embodiment, a design support system 1 applied to an automated transport service construction of goods in a logistics warehouse (area) 10 using automated transport vehicles is described.

[0032] Figure 1The design support system 1 includes a regional environment data generation unit 3, an autonomous driving control function database DB, an autonomous driving control system evaluation unit 5, a customer information input unit 6, an autonomous driving ODD automatic definition unit 7, an ODD adjustment unit 8, and a KPI evaluation unit 9, and takes in the output of the environmental measuring device 2 and customer requirements from an external logistics warehouse (area) 10 to provide design support.

[0033] The design support system 1 formed of a computer device stores a design support program for automatically designing an ODD of an autonomous driving vehicle in an internal ROM, and implements a design support method by executing processing in accordance with the program in a CPU. The program is described in Embodiment 5.

[0034] The following details Figure 1 The structure shown here includes the present invention portion shown in bold lines and the prior art portion shown in thin lines, so the prior art portion shown in thin lines will be described first.

[0035] The design support system 1 is connected to the environment measuring device 2 via a signal line, and the design support system 1 acquires the environment information D1 of the area 10 where the service is provided. Figure 2 The automatic transport area of ​​the cargo transport service shown in the figure. In the automatic transport area 10, trucks 18 enter and exit from the outside, and the forklift 17 moves the cargo in and out between the trucks 18 and the cargo storage area 16. In addition, the cargo in the cargo storage area 16 is moved by the automatic transport vehicle 11 on a path avoiding obstacles such as the rack 12 in the automatic transport service providing area 15, so as to transfer the cargo to the indicated appropriate storage place.

[0036] The automated transport vehicle 11 is equipped with a camera and / or LiDAR as an environment measuring device 2B, and a camera and / or LiDAR is arranged as a fixed environment measuring device 2A at an appropriate position in the area 10, and the environment in the area is grasped by these devices. The environmental information D1 measured by these environment measuring devices 2A and 2B is transmitted to the design support system 1.

[0037] In this way, the environment measuring device 2 can obtain the environment information of the automatic transport area 10 by driving the automatic transport vehicle 11 on a path specified by the area data collection unit 3a described later, or the environment measuring device 2 can be installed at any location in the automatic transport area 10 to collect the environment information of the automatic transport area 10.

[0038] Here, the environmental information D1 refers to the road surface on which the automated transport vehicle 11 is traveling, the location and shape of the structure, the lighting brightness in the driving area, whether infrastructure such as WiFi is available, etc. Therefore, the environmental measurement device 2 is not limited to cameras and LiDAR, but also includes a measurement device for radio waves and lighting brightness, etc. In addition, the environmental measurement device 2 can be set on either the vehicle side or the equipment side, and the signal line between the environmental measurement device 2 and the design support system 1 can be either a wired connection or a wireless connection.

[0039] The environmental information D1 measured by the environmental measurement device 2 is input to the area environmental data generating unit 3 .

[0040] The regional environment data generating unit 3 is composed of a regional data collecting unit 3a, a regional digital map generating unit 3b and a service providing area determining unit 3c. The service providing area determining unit 3c which is partially related to the present invention will be described later.

[0041] In the regional data collection unit 3a, for example, Figure 2 The environment measuring device 2B mounted on the automatic transport vehicle 11 traveling in the automatic transport area 10 providing cargo transport services as shown collects the environment information D1 of the automatic transport area 10. The environment measuring device 2B can obtain the environment information D1 of the automatic transport area 10 by driving the automatic transport vehicle 11 on the path specified by the area data collection unit 3a, or the environment measuring device 2B can be installed at any place in the automatic transport area 10 to collect the environment information D1 of the automatic transport area 10. In addition, Figure 2 The field reference coordinates 19 used to define the location within the area are also indicated.

[0042] The area digital map generator 3 b collects the environmental information D1 collected by the area data collector 3 a and constructs a shape digital map that expresses obstacles such as the racks 12 provided in the automated transport area 10 as a collection of three-dimensional points.

[0043] exist Figure 3 As an example of a shape digital map, the shape data of the shelf 12 generated based on an image acquired by an image sensor mounted on an automated transport vehicle 11 is shown. The data is obtained by extracting the outer shape line 13 based on the image of the shelf 12, and the data of the outer dimensions of the shelf 12, the data 20 of the aisle width, etc. are processed as a shape digital map.

[0044] In addition, the environmental information related to radio waves such as WiFi in the automatic transport area 10 can be stored in the digital map obtained by the environmental measurement device 2 for management, such as management by area based on whether wireless infrastructure can be used. Figure 3A WiFi antenna 21 is installed in the shooting area, and environmental information related to the radio waves in the place can also be managed.

[0045] exist Figure 1 The automatic driving control function database DB stores automatic driving control function data D2. For an example of the automatic driving control function data D2, Figure 4 As shown, the ID (D21) of the automatic driving control function, the function name D22, the function specification D23, the automatic driving control program D24 of the function, the sensor D25 required to realize the function, the KPI (D26, D27), etc. are stored in the form of a relational database. The automatic driving control function is, for example, a function of obtaining the position of the vehicle in the automatic transport area 10, and a control of the automatic transport vehicle 11 for obtaining goods from the shelf 12.

[0046] Figure 1 The automatic driving control system evaluation unit 5 includes an operation evaluation unit 5a having a function of evaluating the automatic driving performance in the automatic transport area 10 when some functions of the automatic driving control functions as automatic driving control programs recorded in the automatic driving control function database DB are mounted on the automatic transport vehicle 11. The operation evaluation unit 5a may be composed of a function such as a vehicle motion simulator that can be simulated on a computer, or may be a computing unit that performs evaluation based on the functional specifications of the automatic driving control functions recorded in the automatic driving control function database DB.

[0047] The structure shown by thin lines above is partly related to the prior art. Based on this premise, the present invention is described in detail below. First, the customer information input unit 6 sets the key performance evaluation index (hereinafter referred to as KPI) as a variable based on the requirements input from the customer providing the service.

[0048] KPIs include, for example, service introduction cost, maintenance cost, safety, operating rate, number of shipments per unit time, etc. In addition, the customer information input unit 6 also defines a calculation formula for calculating KPIs.

[0049] In addition, the customer information input unit 6 determines the service provision area based on the input service request, and outputs the information of the provision area to the service provision area determination unit 3c. For example, the customer information input unit 6 may include a system for determining the area using a graphical user interface (GUI) or the like, such as by specifying a rectangle on a map. In this case, Figure 2 The area 10 is determined as the service provision area.

[0050] In the Figure 2In the case where the automatic transport area 10 is set as the service provision area, the customer can also specify the automatic transport service provision area 15 and the cargo storage area 16 using the GUI. In this case, the forklift 17 is specified to enter only the cargo storage area 16 in the automatic transport service provision area 15. In addition, the customer can specify that the forklift 17 is manually driven and that the forklift 17 transports the cargo from the cargo storage area 16 to the truck 18.

[0051] From this Figure 1 The service providing area determination unit 3c receives information about the providing area 10 and determines the area. For example, the area is determined based on the location on the map specified by the GUI provided by the customer information input unit 6, by using the location specified based on the latitude and longitude of the area, the fixed area specified based on the size, or the running path obtained by specifying the latitude and longitude of multiple points. Figure 2 As an example, the on-site reference coordinates 19 are set in the automatic transport service providing area 15 , and the coordinates of the four corners of the automatic transport service providing area 15 are specified by the on-site reference coordinates 19 .

[0052] The determined area information is provided to the area data collection unit 3a, which collects environmental information within the area. In addition, the area information is also provided to the area digital map generation unit 3b to specify the range of the digital map to be generated by the area digital map generation unit 3b.

[0053] Thus, the regional environment data generation unit 3 organizes the service area 10, such as the current regional structure (regional digital map, obstacles, etc.) and the environmental information within the region, as well as what (where) and how to change the operating environment based on customer requirements based on this state, and delivers them to the autonomous driving ODD automatic definition unit 7 for the next level of processing.

[0054] Figure 1 The automatic definition unit 7 of the autonomous driving ODD includes an operating environment characteristic extraction unit 7a and an operating environment change unit 7b. The operating environment characteristic extraction unit 7a defines the ODD based on the digital map generated by the regional digital map generation unit 3b and the environmental information D1. The items of the ODD are, for example, Figure 3 the minimum value of the channel width 20 through which the automated transport vehicle 11 passes, the presence or absence of infrastructure such as a WiFi antenna 21 required for the automated transport vehicle 11 to use WiFi, Figure 2 The automatic transport service providing area 15 includes whether there are operators, the maximum speed of the vehicle defined in accordance with the safety regulations of the automatic transport area 10, etc.

[0055] The operating environment changing unit 7b has a function of outputting the change to the operating environment characteristic extracting unit 7a when a change occurs in the ODD item in the automatic transport area 10. For example, when the layout of the shelves 12 in the automatic transport area 10 is changed, Figure 3 When the minimum value of the channel width 20 is changed, the minimum value is output. Figure 2 Even in the case where a fixed monitoring camera 2A is installed in the automated transport area 10, the change is extracted. The installation of the fixed monitoring camera 2A means that the infrastructure sensor-based support of the ODD project, that is, the automated driving of the automated transport vehicle 11 can be supported by the infrastructure sensor, and this change point is defined as the presence or absence of the infrastructure sensor-based support of the ODD project by the operating environment characteristic extraction unit 7a.

[0056] Furthermore, the operating environment change unit 7b has a function of calculating the installation candidates of sensors as infrastructure, such as the surveillance camera 2A, which performs infrastructure sensor support in the automatic transport area 10 based on the digital map. For example, when there is an area in the automatic transport service provision area 15 where multiple automatic transport vehicles 11 are difficult to detect each other due to a blind spot caused by the racks 12 based on the digital map, the type of sensor to be deployed, such as the surveillance camera 2A, and its installation position are output as candidates in a manner to eliminate the blind spot. The types of sensors that can be used in infrastructure sensor support and their use are stored in the automatic driving control function database DB.

[0057] exist Figure 1 The ODD adjustment unit 8 has a function of adjusting and determining the ODD item D3 of the automatic transport vehicle 11 in the provided service and the automatic transport control function data D2 of the automatic transport function mounted on the automatic transport vehicle 11 based on the ODD item D3 from the operating environment characteristic extraction unit 7a and the automatic transport control function data D2 from the automatic transport control function database DB. The adjustment is performed in a manner to improve the customer KPI evaluation value from the KPI evaluation unit 9 described later.

[0058] Regarding adjustment methods, for example, based on Figure 5 The processing shown in the traceability tables 23a and 23b showing the relationship between the automatic driving control function data D2, the ODD item D3 and the KPI values ​​D26 and D27 is performed.

[0059] The traceability table 23a lists multiple autonomous driving control functions (ID 1, 2, 3) referenced from the autonomous driving control function database DB on the vertical axis, and shows the ID (D21) of the corresponding autonomous driving control function data D2, the function name D22, and the KPI values ​​D26 and D27 on the horizontal axis. The KPI values ​​in each autonomous driving control function and D27 are KPI values ​​that may be generated by installing the function. For example, KPI1 of D26 is the impact on the installation cost (k￥) of the function, and KPI2 of D27 is the impact on power consumption and service operation time (h).

[0060] In the example shown in the figure, when the ID (D21) of the automatic driving control function data D2 is any one of 1, 2, and 3, the impact KPI2 on power consumption and service operation time (h) is 0, and the impact KPI on the function installation cost (k￥) is -4, -3, and -8.

[0061] In addition, the traceability table 23a shows a list of ODD items D3 of each of the plurality of autonomous driving control functions (IDs are 1, 2, and 3) on the horizontal axis (ODD items 1, 2, and 3), and shows the correspondence between the presence or absence of the ODD items and the influence of the numerical values ​​on the autonomous driving control functions on the vertical axis. In this example, an example is shown where ODD items 1 and 3 are applied to autonomous driving control functions 1 and 2, and ODD item 2 is applied to autonomous driving control function 3.

[0062] In addition, the traceability table 23a has KPI values ​​(customer KPI evaluation value 13) that may be generated with the change of ODD item D3. In this example, the KPI of the impact on the cost of installing the function (k￥) after the change of ODD item D3 is 0 in any of ODD items 1, 2, and 3, indicating that there will be no cost increase due to the change. In contrast, the KPI2 of the impact on the power consumption and service operation time (h) after the change of ODD item D3 shows that it is +12, +4, and +12 in ODD items 1, 2, and 3, respectively.

[0063] according to Figure 5 The KPI value after the ODD item is applied in the traceability table 23a (customer KPI evaluation value 13), the improvement effect based on ODD item 2 is not that great, but the improvement effect based on ODD items 1 and 3 is significant. In addition, the evaluation result is finally accepted and the application of ODD item 2 to the automatic driving control function 2 is removed in the traceability table 23b. The new application completed KPI value at this time is shown as the customer KPI evaluation value 17.

[0064] The KPI for each item of the traceability table 23 a is set by transferring the KPI set by the customer information input unit 6 to the ODD adjustment unit 8 via the KPI evaluation unit 9 and the automatic driving control function database DB.

[0065] exist Figure 6 Flowchart 24 shows an example of a process performed by the ODD adjustment unit 8, the automatic driving control system evaluation unit 5, and the KPI evaluation unit 9 to determine a combination of ODD items and automatic driving control functions so as to improve the customer KPI evaluation value.

[0066] The flowchart 24 includes steps S100 to S107. Steps S100, S101, S102, S105, S106, and S107 are processed by the ODD adjustment unit 8. Step S103 is processed in the automatic driving control system evaluation unit 5, and step S104 is processed in the KPI evaluation unit 9.

[0067] In this series of processing, first, starting from step S100, in step S101, the temporary storage variable Ep of the KPI evaluation value is initialized to 0. In step S102, the combination of the autonomous driving control function and the ODD item on the vertical axis and the horizontal axis of the traceability table 23 is changed. In step S103, the selected ODD item D3 and the autonomous driving control function D2 are evaluated by the autonomous driving control system evaluation unit 5. In step S104, the customer KPI evaluation value is calculated by the KPI evaluation unit 9 based on the evaluation result of the autonomous driving control system evaluation unit 5, and is substituted into the variable E. The formula for calculating the customer KPI evaluation value E is as follows.

[0068] [Formula 1]

[0069] E = Σwi·Ki(i∈1…L) (1)

[0070] Here, Ki is each KPI value, and wi is the weight of each KPI calculated by the customer information input unit 6 based on the information input by the customer into the customer information input unit 6, for example, it is set based on the importance and priority of the customer. In addition, the weight wi can also be used as a parameter for normalizing KPI values ​​with different units. L is the number of KPIs.

[0071] In step S105, it is determined whether the customer KPI evaluation value has been improved and whether it has fully converged. If the condition is true, it is determined that the customer KPI has been improved to the maximum extent, and the combination of the ODD item D3 and the automatic driving control function D2 at this time is output, and the process ends in step S108. If the conditional expression of step S105 is false, the process proceeds to step S107, the calculated customer KPI evaluation value E is substituted into the temporary storage variable Ep, and the process returns to step S102 to search for a combination that improves the customer KPI evaluation value D2 again.

[0072] In the above description, although the combination of the ODD item D2 and the automatic driving control function D3 is adjusted, the customer KPI evaluation value E may be improved by changing only the ODD item D2.

[0073] Back to Figure 1 , further specifying Figure 6 process. Figure 1 The automatic driving control system evaluation unit 5 shown includes an operation scenario generation unit 5 b that automatically generates a test scenario for evaluation by the operation evaluation unit 5 a based on the ODD item D2 received from the ODD adjustment unit 8 , and transmits the test scenario to the operation evaluation unit 5 a .

[0074] For example, the number of vehicles in operation in the test scenario is set according to the number of vehicles in operation of the ODD. In addition, in the case of an ODD defined as a situation where an operator is present in the automatic transport service providing area 15, whether the operator can safely control the automatic transport vehicle 11 when walking in various places is recorded in the test scenario and evaluated.

[0075] The KPI evaluation unit 9 collects the results of the operation evaluation unit 5a and calculates each KPI value, for example, using the formula (1) to calculate the customer KPI evaluation value E. In addition, when it is determined that the customer KPI evaluation value E is sufficiently improved by the ODD adjustment unit 8, the KPI evaluation unit 9 outputs the ODD, the automatic driving control function, and the customer KPI evaluation value as the calculation result of the design support system 1.

[0076] The final ODD, autonomous driving control function and customer KPI evaluation values ​​are Figure 5 Traceability Table 23b will remove the applicability of ODD item 2 to the automatic driving control function 2 and present other situations as the final solution.

[0077] The effects of the present invention using the above series of treatments are described. As a specific example of the effects, it is assumed here that Figure 2The illustrated work site newly introduces an automatic transport vehicle 11 and provides an automatic cargo transport service in the automatic transport area 10. In this hypothetical example, since it is not clear whether the automatic transport area 10 is an environmental condition that fully satisfies the driving performance of the automatic transport vehicle 11, there is a problem that the introduction requires man-hours and increases costs. In addition, even if the automatic cargo transport service is introduced at such an introduction cost, it is still unclear whether the performance that satisfies the KPI value expected by the customer will be shown before the introduction and operation, and adjustments for improving the KPI value will also incur costs.

[0078] However, in this embodiment, by using Figure 1 The design support system 1 shown can determine whether the environmental conditions in the automatic transport area 10 fully meet the driving performance of the automatic transport vehicle 11 before the automatic transport vehicle 11 is introduced into the automatic transport area 10, and can also estimate the performance of the automatic cargo handling service that maximizes the KPI value required by the customer, thereby reducing the time and cost caused by adjustments after introduction.

[0079] In the design support system 1, first, the customer inputs KPI and the service to be introduced into the customer information input unit 6, and the regional environment data generation unit 3 collects and analyzes the environment information in advance. Based on the environment information, the ODD automatic definition unit 7 can automatically define the ODD. The ODD adjustment unit 8 can evaluate the performance sufficiency relationship between the defined ODD and the functions that can be installed in the automatic transport vehicle 11 stored in the automatic driving control function database DB, and based on the customer KPI evaluation value estimated by the automatic driving control system evaluation unit 5 and the KPI evaluation unit 9, automatically design a combination of the ODD and the automatic driving control function that maximizes the customer KPI evaluation value.

[0080] Show use Figure 5 An example of a combination of traceability tables 23a and 23b to calculate improved customer KPI evaluation values. The vertical axis of traceability table 23a records three types (ID: 1, 2, 3) of autonomous driving control functions. For each autonomous driving control function, a KPI value relative to the KPI presented by the customer, namely, the installation cost (KPI1) and the service provision time (KPI2), is set. The horizontal axis is set with ODD items 1 to 3 and the KPI value determined based on whether or not each ODD item is considered. The corresponding relationship is shown in the orthogonal table of the vertical and horizontal axes. In formula (1), if the weights in the traceability table 23a are all set to 1 to calculate the customer KPI evaluation value, the value is 13.

[0081] If based on the customer KPI evaluation value and in accordance with Figure 6By improving the customer KPI evaluation value according to flowchart 24, a combination such as traceability table 23b can be calculated. In traceability table 23b, as an example, ODD item 2 is taken into consideration. If the KPI2 value related to ODD item 2 is compared with the KPI1 value of the automatic driving control function CCC required to cope with ODD item 2, it can be seen that the value of improving the customer KPI evaluation value is a negative value.

[0082] Therefore, follow Figure 6 Flowchart 24 is set as follows Figure 5 As in the traceability table 23b of FIG. 2 , the automatic cargo handling service of ODD item 2 is not considered, and the automatic driving control function CCC is not installed on the automatic transport vehicle 11. Therefore, although the service provision time of KPI2 is reduced, if the weights in formula (1) are all set to 1 to calculate the customer KPI evaluation value of the traceability table 23b, it is 17, and a design solution that is an improvement over the traceability table 23a can be obtained.

[0083] In this way, by having the customer start the automatic cargo handling service based on the combination of the ODD and the automatic driving control function as the output of the design support system 1, the adjustment after introduction that has been occurring until now can be suppressed to a minimum, and a service that satisfies the customer's KPI to the maximum extent can be quickly started. For example, if the KPI is cost, a design solution can be obtained before actual operation, that is, a design solution that minimizes the development of additional functions and performance improvements for the automatic driving control function and uses rules to exclude pedestrians in the automatic handling area 10 from the area to relax the ODD.

[0084] As Figure 1 The processing result of the design support system 1 shown in the figure shows various improvement information related to the improvement of the regional environment from the design support system 1. Among them, the improvement information provided by the KPI evaluation unit 9 is the improvement information D4 related to the automatic driving control function, the improvement information D5 related to the ODD, and the customer KPI (D6), and the improvement information provided by the automatic driving control system evaluation unit 5 is the information D7 of the infrastructure sensor related to the sensor configuration and sensor type.

[0085] Figure 7 This is a diagram showing an example of utilization of the output of the design support system 1 . Figure 7 The upper-level improvement information D4 related to the autonomous driving control function is provided directly to the autonomous driving vehicle 11, or is provided indirectly to the autonomous driving vehicle 11 via the control system 30.

[0086] The autonomous driving control function D4 includes control software and sensor specifications required therefor, and the autonomous driving control function D4 from the design support system 1 is newly added to the autonomous driving vehicle 11. The control software is installed in the computing device 11a (controller, etc.) of the autonomous driving vehicle 11, and the sensor 11b is installed on the autonomous driving vehicle 11 in accordance with the sensor specifications.

[0087] Correspondingly, the automatic driving vehicle 11 calculates the control instruction value for automatic driving control of the vehicle by the computing device 11a, sends the control instruction value to the control device 11a, and sends the control signal for driving the actuator of the vehicle (not shown, the tire speed and steering angle) to the actuator.

[0088] In the case of indirect use via the control system 30, the computing device 31 and the control device 11a may be provided in different devices. For example, the computing device 31 may be provided in the control system 30 which is a device provided in a place other than the autonomous driving vehicle 11. The control system 30 and the autonomous driving vehicle 11 can transmit and receive signals through wireless communication or the like.

[0089] In this case, the control software of the automatic driving control function D4 is installed in the computing device 31 of the control system 30, and the sensor is mounted on the automatic driving vehicle 11 based on the sensor specification. The control command value for performing the automatic driving control of the vehicle calculated by the computing device 31 of the control system 30 is sent to the automatic driving vehicle 11 via wireless communication, and the control device 11a of the automatic driving vehicle 11 sends a control signal for driving the actuator of the vehicle to the actuator based on the control command value. The sensor signal is sent to the computing device 31 of the control system 30 via wireless communication and is used for the calculation of the automatic driving control.

[0090] exist Figure 7 In the second layer from the top, examples of utilization of improvement information D5 related to ODD are shown, in which case it is directly provided to the autonomous driving vehicle 11 or indirectly provided to the autonomous driving vehicle 11 via the control system 30.

[0091] In the case of indirect use, the improvement information D5 related to ODD is used in the control system 30. The ODD departure judgment unit 32 of the control system 30 judges whether the environment (service provision area) in which the autonomous driving vehicle 11 is traveling satisfies the drivable environment conditions defined by the improvement information D5 related to ODD based on the improvement information D5 related to ODD and various environmental information.

[0092] The control system 30 can send and receive signals with the control device 11a of the autonomous driving vehicle 11 via wireless communication. The ODD separation judgment unit 32 does not send any signal to the control device when the driving conditions are met. On the other hand, when the driving conditions are not met (when the ODD is separated), an ODD separation signal is sent to the control device 11a of the autonomous driving vehicle 11.

[0093] When the control device 11a of the autonomous driving vehicle 11 receives the ODD disengagement signal, it notifies the driver who arrives to cancel the autonomous driving state that the vehicle is switched to manual driving, or changes from the autonomous driving state to the emergency stop state, automatically generates a control signal to stop the vehicle safely, and sends it to the actuator. In addition, the ODD disengagement judgment unit can also be installed in the autonomous driving car. In addition, the various environmental information includes weather, time, GNSS radio wave conditions, road shape and state, and the presence or absence of unintentional pedestrians.

[0094] exist Figure 7 The third layer from the top shows an example of using the customer KPI (D6). The customer KPI evaluation D6 (evaluation result numerical value, etc.) is used to notify the customer of the customer KPI evaluation result via the display device 40. When the customer accepts the evaluation result notified by the display device, for example, the customer instructs to start the post-process of the automatic excavation control function, ODD, and infrastructure sensor. When the customer does not accept the evaluation result, the customer changes the Figure 1 According to the customer's request, the design support system 1 is executed again.

[0095] exist Figure 7 The third layer from the top shows an example of using the infrastructure sensor information D7. Based on the infrastructure sensor information D7, the designer or the like performs a detailed design 50 (design of the support column for installing the sensor, etc.) for installing the infrastructure sensor in the service provision area as a post-process, and generates a detailed drawing 51. Then, based on the detailed drawing 51, the construction personnel install 52 the infrastructure sensor in the service provision area.

[0096] In addition, in Example 1, although it is explained that the ODD condition setting is performed in accordance with the customer's intention in principle, it can also be a proposal for changing the service conditions. In addition, if the ODD is exceeded during installation, the vehicle can be stopped or the automatic driving can be stopped and the driving authority can be handed over to the driver.

[0097] According to the present invention, a design support system can be provided that can automatically design an operational design domain (ODD) adjusted to maximize the important performance evaluation index of the business owner and ensure reliability at the same time, thereby providing an autonomous driving service that improves the important performance evaluation index of the business owner.

[0098] Example 2

[0099] In Example 2, it is assumed that the service provider Figure 8 . In Figure 8 In the work site 25 shown, the excavator 27 excavates soil 26 and loads and transports it to the dump truck 28 every day, so changes in environmental information frequently occur. When the excavator 27 and the dump truck 28 are made autonomous, the design support system 1 supports the design before introduction as a service related to autonomous driving, but it is also desired to redefine the ODD in accordance with environmental changes after the start of operation and maintain or improve the customer KPI evaluation value.

[0100] In order to solve the above problems, use Figure 1 , 8 9 illustrate aspects and structures different from those of Example 1. First, in Example 2, Figure 1 The regional data collection unit 3a is newly equipped with an environment change detection unit 3d. The environment change detection unit 3d has the following function: when the environment of the service area changes, the regional data collection unit 3a collects data again, and the regional digital map generation unit 3b generates a digital map again, and the series of processing of the design support system 1 is executed again.

[0101] Therefore, if Figure 8 In the work site 25, the excavator 27 operating by automatic driving is digging sand 26 forward. Fig. 9 In area 29, the sand 26 disappears and the road surface has a region where mud remains. Therefore, the road surface of area 29 is more uneven than other areas. By redefining the state where such an uneven road surface exists in the autonomous driving service provision area as ODD, it is necessary to study whether the autonomous driving dump truck 28 can normally travel in area 29, and quickly verify whether the function of the dump truck 28 needs to be improved or whether the area 29 should be leveled to the same level as other areas to relax the ODD.

[0102] Therefore, for example, when the environment change detection unit 3d of the design support system 1 detects that a change has occurred in the area 29 at the work site 25 based on the environment information D2 from the environment measuring device 2 mounted on the dump truck 28, the area environment data generation unit 3, the ODD automatic definition unit 7, the ODD adjustment unit 8, the automatic driving control system evaluation unit 5 and the KPI evaluation unit 9 are executed again to redesign and output the ODD and automatic driving control functions that maintain or improve the customer's KPI evaluation value.

[0103] As a result, a design policy such as leveling the area 29 to the same level as other areas to relax the ODD can be obtained in response to the ever-changing environmental changes, and a level that satisfies customer KPIs such as cost can be maintained.

[0104] Example 3

[0105] In Example 3, it is assumed that the service provider Fig.10 .exist Fig.10 In the general public road environment 30 shown in the figure, the autonomous driving car 31 needs to drive in accordance with various traffic rules including traffic lights 32. Traffic rules are defined as ODD like the driving environment, and the autonomous driving car 31 needs to be equipped with an autonomous driving control function that satisfies the ODD of the general public road environment 30.

[0106] In order to make the autonomous driving car 31 travel in such an ordinary public road environment 30. It is necessary to deal with not only a variety of traffic regulations, but also a large number of ODD items such as weather and pedestrians, so the autonomous driving control function installed is increased. When studying the autonomous driving control function installed in the autonomous driving car 31, it is necessary to conduct design research on whether to develop a new one or adopt a developed autonomous driving control function. However, in the design support system 1 of Example 1, since research can only be conducted based on the functions entered in the autonomous driving control function database DB, when providing an autonomous driving car 31 in an ordinary public road environment 30 with no design experience, there is a problem that the design support system 1 does not output an appropriate design and cannot obtain a design solution that meets the customer's KPI.

[0107] In order to solve the above problems, use Figure 1 The following describes aspects and configurations that are different from those of Example 1. Figure 1 middle, Figure 1 The KPI evaluation unit 9 has a function of adding an automatic driving control program with changed functional specifications and an automatic driving control program with a newly added function to the automatic driving control function database DB.

[0108] Fig.10 The ordinary public road environment 30 has a traffic light 32, so it is defined as an environment with a traffic light by ODD. Therefore, the self-driving car 31 is required to have a function of recognizing the sign of the traffic light 32.

[0109] When the traffic light 32 sign is recognized by the onboard camera of the self-driving car 31, it may become a function that costs money to install, depending on the weather and lighting conditions, which are other ODD items. Therefore, a support environment for recognition based on the infrastructure 33 is constructed, and an ODD that can be supported by the infrastructure is set up.

[0110] In the Fig.11 When the ODD item 4 shown in the traceability table 34 is set to indicate an ODD supported by the infrastructure capable of transmitting information to the traffic light 32, the automatic driving control function DDD35 corresponding to the ODD item 4 needs to be installed in the automatic driving car 31. Figure 6 The result of improving the customer KPI evaluation value is shown in flowchart 24. Fig.11 The traceability table 34 is the design solution. In the case where the automatic driving control function DDD35 is a newly added function, Figure 1 The KPI evaluation unit 9 enters the autonomous driving control function DDD35 into the autonomous driving control function database DB. Thus, when the design support system 1 is used to provide other autonomous driving services in an environment similar to the general public road environment 30, the autonomous driving control function DDD35 can be selected, so that a design solution that satisfies the customer's KPI can be quickly obtained from the design support system 1 that provides the autonomous driving car 31 without the cost of developing new functions.

[0111] In the above embodiments, the case where the present invention is applicable to an automatic transport vehicle, an engineering vehicle, and an automatic driving car is cited as an example for explanation, but the present invention can also be applied to other automatic driving vehicles such as agricultural vehicles and autonomous working robots.

[0112] Example 4

[0113] In the fourth embodiment, a program of the design support system is described. The program of the design support system is stored in the ROM of the computer device, and a part of the program is executed. Figure 6 The ODD adjustment unit 8, the automatic driving control system evaluation unit 5, and the KPI evaluation unit 9 are shown in the figure. Although other programs are not shown in the figure, they can be Figure 1 The program is composed of the calculation functions of each part.

[0114] For example, a program may be formed for the regional environment data generating unit 3 as a unit, and the functions of the regional data collecting unit 3a, the regional digital map generating unit 3b, and the service providing area determining unit 3c may be realized as subroutines therein. This consideration may also be applied to the calculation functions of other units.

[0115] Specifically, for example, the following programs may be provided: an autonomous driving control function program, which is a program for performing autonomous driving control of a vehicle traveling in a service provision area; a regional environmental data generation program, which has a regional digital map generation unit that uses environmental data to generate a digital map of the service provision area; an autonomous driving ODD automatic definition program, which generates and outputs an operation design domain that becomes an environmental constraint condition for operating an autonomous driving service based on information on the digital map; an ODD adjustment program, which evaluates the adequacy of the autonomous driving control function specifications stored in pairs with the autonomous driving control program for the operation design domain, and changes the operation design domain based on the evaluation results; an autonomous driving control system evaluation program, which evaluates the operation design domain based on customer requirements; and a KPI evaluation program, which generates a customer requirement evaluation formula for evaluating important performance evaluation indicators of customers based on customer requirements, calculates the customer requirement evaluation formula based on the values ​​output from the autonomous driving control system evaluation program, and outputs the operation design domain and the important performance evaluation indicators of customers.

[0116] Description of Reference Numerals

[0117] 1: Design support system

[0118] 2: Environmental measurement device

[0119] 3: Regional Environmental Data Generation Department

[0120] 3a: Regional Data Collection Department

[0121] 3b: Regional digital map generation department

[0122] 3c: Service provision area determination department

[0123] 3d: Environmental change detection department

[0124] 4: Autonomous driving control function database

[0125] 5: Autonomous driving control system evaluation department

[0126] 5a: Operation Evaluation Department

[0127] 5b: Run the script generation section

[0128] 6: Customer information input department

[0129] 7: ODD automatic definition unit

[0130] 8: ODD adjustment department

[0131] 9: KPI Evaluation Department

[0132] 10: Automatic handling area

[0133] 11: Automated transport vehicles

[0134] 12: Shelves

[0135] 13: Outline

[0136] 15: Automatic transport service provision area

[0137] 16: Cargo storage area

[0138] 17: Forklift

[0139] 18: Truck

[0140] 19: On-site reference coordinates

[0141] 20: Channel width

[0142] 21: WiFi Antenna

[0143] 22: Surveillance Camera

[0144] 23a, 23b: Traceability table

[0145] 24: Flowchart

[0146] 25: Operation site

[0147] 26: Sandy soil

[0148] 27: Excavator

[0149] 28: Dump Truck

[0150] 29: Region

[0151] 30: Ordinary public road environment

[0152] 31: Self-driving cars

[0153] 32: Traffic lights

[0154] 33: Infrastructure

[0155] 34: Traceability table

[0156] 35: Automatic driving control function DDD

[0157] S100~S108: steps.

Claims

1. A design support system that obtains customer requirements and environmental data from an environmental measurement device that measures the environment of a service provision area managed by the customer and outputs an improvement value of an important performance evaluation index of the customer, the design support system comprising: an automatic driving control function database storing an automatic driving control program for performing automatic driving control on a vehicle traveling in the service provision area; an area environment data generating unit including an area digital map generating unit for generating a digital map of the service providing area using the environment data; an automatic definition unit for autonomous driving ODD, which generates and outputs an operation design domain based on the information of the digital map, wherein the operation design domain is an environmental constraint condition for operating an autonomous driving service; an ODD adjustment unit that evaluates the adequacy of the automatic driving control function specification stored in pair with the automatic driving control program for the operation design domain output from the automatic driving ODD automatic definition unit, and changes the operation design domain based on the evaluation result; an automatic driving control system evaluation unit that evaluates the operation design domain based on the customer requirements; and a KPI evaluation unit that generates a customer requirement evaluation formula for evaluating the customer's important performance evaluation index based on the customer requirement, calculates the customer requirement evaluation formula based on the value output from the automatic driving control system evaluation unit, and outputs the operation design domain and the customer's important performance evaluation index, The ODD adjustment unit changes the operation design domain based on the value of the customer requirement evaluation formula output by the KPI evaluation unit so that the value of the customer requirement evaluation formula is improved compared with the previous value of the customer requirement evaluation formula. The KPI evaluation unit outputs an improvement value of an important performance evaluation indicator of a customer based on the changed operation design domain.

2. The design support system according to claim 1, wherein: The autonomous driving ODD automatic definition unit generates and outputs the autonomous driving control function of the vehicle, the autonomous driving control system evaluation unit evaluates the autonomous driving control function of the vehicle based on the customer's requirements, and the KPI evaluation unit outputs important performance evaluation indicators of the autonomous driving control function and the customer.

3. The design support system according to claim 1, wherein: The automatic driving control system evaluation unit prompts improvement of the environment in the service provision area.

4. The design support system according to claim 1, wherein: The ODD adjustment unit changes the operation design domain and the automatic driving control program based on the value of the customer requirement evaluation formula output by the KPI evaluation unit so that the value of the customer requirement evaluation formula is improved compared with the previous value of the customer requirement evaluation formula. The KPI evaluation unit outputs an improvement value of an important performance evaluation indicator of a customer based on the changed operation design domain and the automatic driving control program.

5. The design support system according to claim 1, wherein: The automatic driving control system evaluation unit utilizes the automatic driving control program stored in the automatic driving control function database and the generated digital map to perform automatic driving in a hypothetical environment based on the operation design domain, the automatic driving control program and the digital map to evaluate the automatic driving control system and the operation design domain.

6. The design support system according to claim 1, wherein: The automatic driving control system evaluation unit includes an operation scenario generation unit that generates an operation scenario for evaluation based on the operation design domain output from the ODD adjustment unit.

7. The design support system according to claim 1, wherein: The autonomous driving ODD automatic definition unit includes an operating environment change unit that outputs design information of at least one infrastructure sensor that is installed in the environment and collects environmental information based on the information of the digital map and the autonomous driving control program.

8. The design support system according to claim 7, wherein: The design support system includes an infrastructure sensor system that collects information collected by all the infrastructure sensors and shares the information with the vehicle.

9. The design support system according to claim 1, wherein: The area environment data generating unit includes an environment change detecting unit that collects data again when the environment of the area where the service is provided changes, generates a digital map again in the area digital map generating unit, and executes a series of processes of the design support system again.

10. The design support system according to claim 1, wherein: The KPI evaluation unit has a function of adding the changed automatic driving control program to the automatic driving control function database.

11. A design support method, which obtains customer requirements and environmental data from an environmental measuring device that measures the environment of a service provision area managed by the customer, and outputs improvement values ​​of important performance evaluation indicators of the customer using a computer device, wherein: An automatic driving control function database is provided, wherein the automatic driving control function database stores an automatic driving control program for performing automatic driving control on a vehicle traveling in the service provision area, The computing unit of the computer device generates a digital map of the service provision area using the environmental data, generates an operation design domain that serves as an environmental constraint condition for operating the autonomous driving service based on information in the digital map, evaluates the adequacy of the autonomous driving control function specification stored in pair with the autonomous driving control program for the operation design domain, changes the operation design domain based on the evaluation result, evaluates the operation design domain based on the customer requirement, generates a customer requirement evaluation formula that evaluates important performance evaluation indicators of the customer based on the customer requirement, calculates the customer requirement evaluation formula based on the evaluation value of the autonomous driving control system, and obtains the operation design domain and the important performance evaluation indicators of the customer, and Based on the value of the customer requirement evaluation formula, the operation design domain is changed in such a way that the value of the customer requirement evaluation formula is improved compared with the previous value of the customer requirement evaluation formula, and based on the changed operation design domain, an improvement value of the customer's important performance evaluation indicator is obtained.

12. A design support program for acquiring customer requirements and environmental data from an environmental measurement device that measures the environment of a service provision area managed by the customer, and outputting improvement values ​​of important performance evaluation indicators of the customer, the design support program comprising the following procedures: an automatic driving control function program, which is an automatic driving control program for performing automatic driving control on a vehicle traveling in the service provision area; an area environment data generating program including an area digital map generating unit for generating a digital map of the service providing area using the environment data; An automatic definition program for autonomous driving ODD, which generates and outputs an operation design domain based on the information of the digital map, wherein the operation design domain is an environmental constraint condition for operating an autonomous driving service; an ODD adjustment program that evaluates the adequacy of the automatic driving control function specification stored in pair with the automatic driving control program for the operation design domain, and changes the operation design domain based on the evaluation result; an automated driving control system evaluation program that evaluates the operational design domain based on the customer requirements; and a KPI evaluation program that generates a customer requirement evaluation formula for evaluating a customer's important performance evaluation index based on the customer requirement, calculates the customer requirement evaluation formula based on a value output from the automatic driving control system evaluation program, and outputs the operation design domain and the customer's important performance evaluation index, The ODD adjustment program changes the operation design domain based on the value of the customer requirement evaluation formula output by the KPI evaluation program so that the value of the customer requirement evaluation formula is improved compared with the previous value of the customer requirement evaluation formula. The KPI evaluation program outputs an improvement value of an important performance evaluation indicator of a customer based on the changed operation design domain.

Images