Management method and management system for automatic driving feature of vehicle

The management server verifies the functionality of updated trained models in vehicles to ensure appropriate image recognition processing for automated driving, enhancing reliability and safety by confirming correct model functionality.

JP2024168092AActive Publication Date: 2024-12-05TOYOTA JIDOSHA KK
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
JP2023084500
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-12-05
Estimated Expiration
2043-05-23

AI Technical Summary

Technical Problem

Existing technologies do not adequately verify whether updated trained models for image recognition processing in vehicles are functioning appropriately for automatic driving control, which is crucial for reliable automated driving functions.

Method used

A management server collects verification data from multiple vehicles to assess the functionality of updated trained models, ensuring that the automatic driving functions meet the intended update criteria by analyzing the collected data.

Benefits of technology

This approach enhances the reliability of automated driving functions by confirming that updated trained models are functioning correctly, allowing for timely adjustments if necessary, thereby improving the overall performance and safety of automated driving systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024168092000001_ABST
    Figure 2024168092000001_ABST
Patent Text Reader

Abstract

To make it possible to verify a model when the model is updated, the model being generated through machine learning and employed in automatic driving control of a vehicle.SOLUTION: A management server transmits update data for use in updating a trained model that is generated through machine learning to at least two vehicles in which automatic driving control using the trained model is implemented. The management server collects verification data indicative of data acquired or generated in relation to the implementation of the automatic driving control using the trained model that has been updated from each of the at least two vehicles to which the update data has been transmitted. The management server uses the verification data collected from each of the at least two vehicles to verify the feature of the automatic driving control using the trained model that has been updated.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to methods and systems for managing automated driving functions in a vehicle. [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2015-135552 discloses a system for updating parameters of image recognition processing performed in each of a plurality of vehicles. This conventional system includes a server that communicates with the plurality of vehicles. The server collects image data captured by each of the plurality of vehicles as learning data, and performs machine learning processing using the learning data. In the machine learning processing, data for updating parameters of the image recognition processing is generated. The server also provides the data generated by the machine learning processing to each of the plurality of vehicles that have collected the learning data. The plurality of vehicles update parameters of the image recognition processing using the data provided by the server.

[0003] In addition to JP 2015-135552 A, examples of documents showing the technical state of the technical field related to the present disclosure include JP 2022-002118 A, JP 2021-532487 A, JP 2018-195184 A, and JP 2019-156171 A. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-135552 A [Patent Document 2] Patent Publication No. 2022-002118 [Patent Document 3] Patent Publication No. 2021-532487 [Patent Document 4] JP 2018-195184 A [Patent Document 5] JP 2019-156171 A Summary of the Invention [Problem to be solved by the invention]

[0005] Image recognition processing performed in a vehicle is useful for controlling the automatic driving of the vehicle. In addition to image recognition processing, examples of processing useful for automatic driving control include processing for "path planning" and "motion planning" of the vehicle. In recent years, models generated through machine learning (hereinafter also referred to as "trained models") are sometimes used for such processing.

[0006] Consider a case where a model construction element such as a trained model or a parameter of the trained model is updated. In this case, it is desirable to verify whether the process using the updated trained model is performed appropriately in the autonomous driving control. For example, it is desirable to perform this verification immediately after updating the trained model. Alternatively, it is desirable to perform this verification periodically after updating the trained model.

[0007] However, conventional documents such as JP2015-135552A do not focus on the verification of the trained model. Therefore, there is room for development of a technology that enables verification of whether computer processing using the updated trained model is being performed appropriately in autonomous driving control.

[0008] One objective of the present disclosure is to provide a technology that enables verification of an updated model in a vehicle in which autonomous driving control is performed using a model generated by machine learning. [Means for solving the problem]

[0009] A first aspect of the present disclosure is a method for managing an autonomous driving function in a vehicle, the method having the following features. The method comprises: A step in which a management server transmits update data indicating data for updating the trained model to at least two vehicles in which autonomous driving control is executed using a trained model indicating a model generated by machine learning; The management server collects, from each of the at least two vehicles to which the update data has been transmitted, validation data indicating data acquired or generated in association with the execution of the autonomous driving control using the updated trained model; The management server verifies a function of autonomous driving control using the updated trained model by using the verification data collected from the at least two vehicles; Includes.

[0010] A second aspect of the present disclosure is a system for managing an autonomous driving function in a vehicle, and has the following features. The system includes a management server that manages the autonomous driving function. The management server includes: A process of transmitting update data indicating data for updating a trained model indicating a model generated by machine learning to at least two vehicles in which autonomous driving control is performed using the trained model; A process of collecting validation data indicating data acquired or generated in association with execution of the autonomous driving control using the updated trained model from each of the at least two vehicles to which the update data has been transmitted; A process of verifying a function of autonomous driving control using the updated trained model by using the verification data collected from the at least two vehicles; Do the following. Effect of the Invention

[0011] According to the present disclosure, verification data is collected from each of at least two vehicles to which update data is transmitted. In addition, the function of the autonomous driving control is verified using the verification data. Therefore, it is possible to determine whether the updated trained model is in line with the intention of the update. Therefore, for example, when it is found that the update does not conform to the intention of the update, it is possible to take measures such as generating data for correcting the updated trained model or the model construction elements and updating the updated trained model again. In this way, according to the present disclosure, it is possible to confirm that the autonomous driving function using the updated trained model is functioning in line with the intention of the update. This contributes to improving the reliability of the autonomous driving function using the trained model. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining an overview of an embodiment. [Diagram 2] FIG. 1 is a block diagram showing a configuration example of an autonomous driving system mounted on a vehicle. [Diagram 3] 2 is a block diagram showing an example of a functional configuration of a management server; [Figure 4] FIG. 11 is a diagram showing an example of instruction data relating to management of log data. [Diagram 5] FIG. 13 is a diagram illustrating a collection time when log data is extracted and stored. [Figure 6] 1 is a flowchart showing a processing flow by an automated driving system, which is particularly related to the embodiment. [Figure 7] 10 is a flowchart showing a flow of processing by a management server, which is particularly related to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are denoted by the same reference numerals, and the description thereof will be simplified or omitted.

[0014] 1. Overview of the embodiment FIG. 1 is a diagram for explaining an outline of the present embodiment. In FIG. 1, three vehicles 1 and a management server 200 are illustrated. The three vehicles 1 are an example of "at least two vehicles" in the present disclosure. Each of the vehicles 1 (hereinafter also referred to as "managed object 1") is equipped with an automatic driving system 100 for performing automatic driving control of the managed object 1. Automatic driving refers to automatically performing at least one of steering, acceleration, and deceleration of the vehicle without the driving operation by an operator. Automatic driving control is a concept that includes not only fully automatic driving control, but also risk avoidance control, lane keep assist control, and the like. The operator may be a driver who rides in the managed object 1, or a remote operator who remotely operates the managed object 1.

[0015] The autonomous driving system 100 includes one or more processors 110 (hereinafter simply referred to as "processors" 110) and one or more storage devices 120 (hereinafter simply referred to as "storage devices 120"). The processors 110 execute various processes. Examples of the processors 110 include a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc. The storage devices 120 store various information. Examples of the storage devices 120 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, a non-volatile memory, etc.

[0016] The management server 200, together with the autonomous driving system 100, constitutes the management system of this embodiment. The management server 200 manages the functions of the autonomous driving system 100, i.e., the autonomous driving functions. The management server 200 includes one or more processors 210 (hereinafter simply referred to as "processor 210") and one or more storage devices 220 (hereinafter simply referred to as "storage device 220"). The processor 210 executes various types of processing. An example of the processor 210 is the same as that of the processor 110 described above. The storage device 220 stores various types of information. An example of the storage device 220 is the same as that of the storage device 120 described above.

[0017] The management server 200 communicates with each of the autonomous driving systems 100. In communication with the autonomous driving systems 100, the management server 200 transmits update data UPD and instruction data INS_LOG to each of the autonomous driving systems 100. The update data UPD is data for updating a trained model. The update data UPD includes model construction elements such as the trained model itself and parameters of the trained model. The trained model is included in the functions of an autonomous driving control unit possessed by the autonomous driving system 100. The functions of this autonomous driving control unit will be described later. The instruction data INS_LOG is data including instructions regarding management of the log data LOG in the autonomous driving system 100. The instruction data INS_LOG will also be described later.

[0018] When a trained model is updated in the autonomous driving system 100, it is desirable to verify whether processing using the updated trained model is performed appropriately in autonomous driving control. Therefore, in this embodiment, the management server 200 collects verification data LOG_VRF from each autonomous driving system 100. In addition, the management server 200 performs verification based on the collected verification data LOG_VRF. The verification data LOG_VRF is log data LOG acquired or generated in processing using the updated trained model and processing related to autonomous driving control.

[0019] By performing verification using the verification data LOG_VRF, it is possible to determine whether the updated trained model is in line with the intention of the update. Therefore, when it is found that the update does not conform to the intention, it is possible to take measures such as generating data for correcting the updated trained model and the model construction elements and updating the updated trained model again. In this way, according to the present embodiment, it is possible to confirm that the automated driving function using the updated trained model is functioning in line with the intention of the update. This contributes to improving the reliability of the automated driving function using the trained model.

[0020] 2. Management system configuration example 2-1. Example of autonomous driving system configuration The present embodiment will be described in more detail below. Fig. 2 is a block diagram showing a configuration example of an autonomous driving system 100 mounted on each of the vehicles 1 shown in Fig. 1. In the example shown in Fig. 2, the autonomous driving system 100 includes a sensor group 10, a recognition unit 20, a planning unit 30, a control amount calculation unit 40, and a traveling device 50.

[0021] The sensor group 10 includes a recognition sensor 11 used to recognize the surroundings of the managed object 1. Examples of the recognition sensor 11 include a camera, a LIDAR (Laser Imaging Detection and Ranging), and a radar. The sensor group 10 may further include a status sensor 12 that detects the status of the managed object 1, a position sensor 13 that detects the position of the managed object 1, and the like. Examples of the status sensor 12 include a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and the like. Examples of the position sensor 13 include a GNSS (Global Navigation Satellite System) sensor.

[0022] The sensor detection information SEN is information obtained by the sensor group 10. For example, the sensor detection information SEN includes images captured by a camera. As another example, the sensor detection information SEN may include point cloud information obtained by a LIDAR. The sensor detection information SEN may include vehicle state information indicating the state of the managed object 1. The sensor detection information SEN may include location information indicating the location of the managed object 1.

[0023] The recognition unit 20 receives the sensor detection information SEN. The recognition unit 20 recognizes the situation around the managed object 1 based on the information obtained by the recognition sensor 11. For example, the recognition unit 20 recognizes objects around the managed object 1. Examples of objects include pedestrians, other vehicles (vehicles ahead, parked vehicles, etc.), white lines, road structures (e.g., guardrails, curbs), fallen objects, traffic lights, intersections, signs, etc. The recognition result information RES indicates the recognition result by the recognition unit 20. For example, the recognition result information RES includes object information indicating the relative position and relative speed of the object with respect to the managed object 1.

[0024] The planner 30 receives the recognition result information RES from the recognition unit 20. The planner 30 may also receive vehicle state information, position information, and pre-generated map information. The map information may be high-precision three-dimensional map information. The planner 30 generates a driving plan for the managed object 1 based on the received information. The driving plan may be for reaching a pre-set destination, or for avoiding risks. Examples of the driving plan include maintaining the current driving lane, changing lanes, overtaking, turning right or left, steering, accelerating, decelerating, and stopping. Furthermore, the planner 30 generates a target trajectory TRJ required for the managed object 1 to travel according to the driving plan. The target trajectory TRJ includes a target position and a target speed.

[0025] The control amount calculation unit 40 receives the target trajectory TRJ from the planning unit 30. The control amount calculation unit 40 calculates a control amount CON required for the managed object 1 to follow the target trajectory TRJ. It can also be said that the control amount CON is a control amount required to reduce the deviation between the managed object 1 and the target trajectory TRJ. The control amount CON includes at least one of a steering control amount, a drive control amount, and a braking control amount. Examples of the steering control amount include a target steering angle, a target torque, a target motor angle, a target motor drive current, etc. Examples of the drive control amount include a target speed, a target acceleration, etc. Examples of the braking control amount include a target speed, a target deceleration, etc.

[0026] The traveling device 50 includes a steering device 51, a driving device 52, and a braking device 53. The steering device 51 steers the wheels. For example, the steering device 51 includes an electric power steering (EPS) device. The driving device 52 is a power source that generates a driving force. Examples of the driving device 52 include an engine, an electric motor, and an in-wheel motor. The braking device 53 generates a braking force. The traveling device 50 receives a control amount CON from the control amount calculation unit 40. The traveling device 50 operates the steering device 51, the driving device 52, and the braking device 53 according to the steering control amount, the driving control amount, and the braking control amount, respectively. As a result, the managed object 1 travels so as to follow the target trajectory TRJ.

[0027] The recognition unit 20 includes at least one of a rule-based model and a trained model. The rule-based model performs recognition processing based on a group of predetermined rules. Examples of the trained model include a neural network (NN), a support vector machine (SVM), a regression model, a decision tree model, and the like. The NN may be a convolutional neural network (CNN), a recurrent neural network (RNN), or a combination thereof. The type of each layer, the number of layers, and the number of nodes in the NN are arbitrary. The trained model is generated in advance or updated through machine learning. The recognition unit 20 performs recognition processing by inputting the sensor detection information SEN to the model. The recognition result information RES is output from the model, or is generated based on the output from the model.

[0028] The planner 30 also includes at least one of a rule-based model and a trained model. The planner 30 performs planning by inputting the recognition result information RES to the model. The target trajectory TRJ is output from the model or is generated based on the output from the model.

[0029] Similarly, the control amount calculation unit 40 includes at least one of a rule-based model and a trained model. The control amount calculation unit 40 performs a control amount calculation process by inputting a target trajectory TRJ to the model. The control amount CON is output from the model or is generated based on the output from the model.

[0030] Two or more of the recognition unit 20, the planner 30, and the control amount calculation unit 40 may be integrally configured. All of the recognition unit 20, the planner 30, and the control amount calculation unit 40 may be integrally configured (end-to-end configuration). For example, the recognition unit 20 and the planner 30 may be integrally configured by a NN that outputs a target trajectory TRJ from the sensor detection information SEN. Even in the case of an integral configuration, intermediate products such as recognition result information RES and target trajectory TRJ may be output. For example, when the recognition unit 20 and the planner 30 are integrally configured by a NN, the recognition result information RES may be the output of an intermediate layer of the NN.

[0031] The recognition unit 20, the planning unit 30, and the control amount calculation unit 40 constitute an "automatic driving control unit" that controls automatic driving of the managed object 1. In this embodiment, a trained model is used for at least a part of the automatic driving control unit. That is, at least one of the recognition unit 20, the planning unit 30, and the control amount calculation unit 40 includes a trained model. The automatic driving control unit performs at least a part of the automatic driving control of the managed object 1 by using the trained model.

[0032] 2-2. Management server configuration example Fig. 3 is a block diagram showing an example of a functional configuration of the management server 200 shown in Fig. 1. In the example shown in Fig. 3, the management server 200 includes a data receiving unit 230, a model generating unit 240, a data transmitting unit 250, a function verifying unit 260, and an instruction setting unit 270. Note that the functions of the model generating unit 240, the function verifying unit 260, etc. are realized, for example, by the processor 210 shown in Fig. 1 executing a predetermined program stored in the storage device 220.

[0033] The data receiving unit 230 has a function as an interface for receiving various data from outside the management server 200. The various data received by the data receiving unit 230 includes verification data LOG_VRF. Examples of the verification data LOG_VRF include the sensor detection information SEN, the control amount CON, the recognition result information RES, and the target trajectory TRJ shown in FIG. 2. The verification data LOG_VRF may include the reason for the determination in the recognition process by the recognition unit 20 shown in FIG. 2. The verification data LOG_VRF may include the reason for the determination in the planning process by the planning unit 30 shown in FIG. 2. The verification data LOG_VRF may include the presence or absence of operator intervention in the automatic driving control.

[0034] The model generation unit 240 generates data (i.e., update data UPD) for updating the trained model applied to the autonomous driving system 100. The update data UPD includes model construction elements such as the trained model itself and parameters of the trained model. Examples of the machine learning model for updating are the same as the examples of the machine learning model applied to the autonomous driving system 100. When a trained model is generated, the trained model applied to the autonomous driving system 100 is updated by replacing it with the generated trained model. When a model construction element is generated, the trained model applied to the autonomous driving system 100 is updated by modifying this model construction element.

[0035] The data transmission unit 250 has a function as an interface for receiving various data from the management server 200 to the outside. The various data transmitted by the data transmission unit 250 includes the update data UPD received from the model generation unit 240. The update data UPD includes the trained model and model construction elements generated by the model generation unit 240. The various data transmitted by the data transmission unit 250 also includes the instruction data INS_LOG received from the instruction setting unit 270.

[0036] The function verification unit 260 uses the verification data LOG_VRF to verify whether the processing using the updated trained model and the processing related to the automatic driving control are performed appropriately. As already described, in this embodiment, the trained model is used for at least a part of the automatic driving control unit described in Fig. 2. Therefore, the automatic driving function using the updated trained model is the target of verification by the function verification unit 260.

[0037] The verification of the autonomous driving function is performed after a predetermined period (e.g., one week or one month) has elapsed after the update data UPD has been transmitted. Alternatively, the verification of the autonomous driving function is performed after the mileage of the managed object 1 after the update data UPD has been transmitted reaches a predetermined distance (e.g., 5 to 10 km). A verification index set in advance is used to verify the autonomous driving function. An example of the verification index is the number of times the operator intervenes in the autonomous driving control.

[0038] As already described, the verification data LOG_VRF may include data on the operator's intervention in the automatic driving control. If the number of interventions in the automatic driving control during a predetermined period (or during driving a predetermined distance) is large, the operator may feel uncomfortable with the automatic driving function using the updated trained model. For example, if the number of interventions after the current update is significantly increased compared to that after the previous update, it is expected that the operator feels strong discomfort with the trained model after the current update.

[0039] The above-mentioned example of the verification index is an example in which one managed object 1 (one operator) is focused on. In the case of focusing on two or more managed objects 1 (two or more operators), the number of interventions per managed object 1 (i.e., the average number of interventions) can be used as the verification index. However, it is considered that the frequency of intervention in the automatic driving control varies depending on the driving proficiency of the operator. Therefore, in the case of focusing on two or more managed objects 1, it is desirable to calculate the verification index by weighting according to the driving proficiency. For example, it is considered that the higher the proficiency of manual driving, the higher the frequency of intervention, and the weight coefficient of the operator with low proficiency is increased. Alternatively, it is considered that the higher the proficiency of automatic driving, the lower the frequency of intervention, and the weight coefficient of the operator with high proficiency is decreased.

[0040] The instruction setting unit 270 sets the instruction data INS_LOG. The instruction data INS_LOG specifies a predetermined event PE, etc., from among events related to the behavior of the managed object 1, which is to be the subject of collection of the verification data LOG_VRF. The instruction data INS_LOG is provided from the management server 200 to the managed object 1 at a predetermined timing in a day (for example, when the ignition of the managed object 1 is turned ON). The original data of the instruction data INS_LOG is stored in the storage device 220. The contents of this original data are updated by the management server 200.

[0041] Fig. 4 is a diagram showing an example of instruction data INS_LOG. In the example shown in Fig. 4, the instruction data INS_LOG includes items of group ID, event ID, event name EN, priority PL, and event judgment On / Off. Note that the instruction data INS_LOG may include items other than these items. Examples of the other items include a pattern for acquiring the verification data LOG_VRF, and a list of position conditions (latitude and longitude), time conditions (morning, afternoon, late night), and weather conditions (sunny, cloudy, rainy) for acquiring the verification data LOG_VRF. The position, time, and weather conditions can set conditions for intensively collecting the verification data LOG_VRF.

[0042] The group ID is an item for identifying a group to which a predetermined event PE belongs. In the example shown in FIG. 4, GI1, GI2, and GIm (m is a natural number equal to or greater than 3) are set as group IDs. GI1 is, for example, an ID of a group related to behavior control of the vehicle 1. GI2 is, for example, an ID of a group related to external recognition by the managed object 1. GIm is, for example, an ID of a group related to abnormality occurrences in various systems mounted on the managed object 1.

[0043] The event ID is an item for identifying a predetermined event PE. In the example shown in FIG. 4, EI1 to E15 are set as the event IDs of GI1. EI1 is, for example, an ID indicating detection of a sudden acceleration or deceleration of the managed object 1. EI2 is, for example, an ID indicating detection of a driving change request (TD: Transition Demand) by the operator of the managed object 1. EI3 is, for example, an ID indicating detection of a sharp turn of the managed object 1. EI4 is, for example, an ID indicating detection of behavior of the managed object 1 that is different from the behavior of surrounding vehicles. EI5 is, for example, an ID indicating detection of a short distance between the managed object 1 and a preceding vehicle or a following vehicle.

[0044] In the example shown in FIG. 4, EI6 to EI8 are set as the event IDs of GI2. EI6 is, for example, an ID indicating erroneous recognition. EI7 is, for example, an ID indicating detection of clouds. EI8 is, for example, an ID indicating detection of raindrops. EIn is set as the event ID of GIm (n is a natural number equal to or greater than 3). EIn is, for example, an ID indicating detection of an abnormal signal.

[0045] The event name EN is an item that succinctly indicates the content of a predetermined event PE. In the example shown in Fig. 4, EN1 to ENn are set as the event names EN. EN1 to ENn correspond to EI1 to EIn, respectively.

[0046] The priority PL is an item indicating the priority order for collecting the verification data LOG_VRF into the storage device 120 when a predetermined event PE occurs. The priority PL is also an item indicating the priority order for transmitting the verification data LOG_VRF to the management server 200. In the example shown in FIG. 4, the priority PL is divided into five levels, 1 to 5. The higher the number of the priority PL, the higher the priority is given to collecting the verification data LOG_VRF for the predetermined event PE. Also, the higher the number of the priority PL, the higher the priority is given to transmitting the verification data LOG_VRF.

[0047] Event judgment On / Off is an item that indicates whether or not a specific event PE is treated as a "collection target event CTE." A collection target event STE is an event among the specific event PE, whose log data LOG during the event is collected in the storage device 21 as verification data LOG_VRF. A specific event PE whose event judgment is "On" is treated as a collection target event CTE. A specific event PE whose event judgment is "Off" is not treated as a collection target event CTE. In other words, log data LOG during a specific event PE whose event judgment is "On" is collected in the storage device 21 as verification data LOG_VRF, and log data LOG other than that is not collected in the storage device 21.

[0048] Here, the log data LOG during a predetermined event PE will be described with reference to Fig. 5. In the example shown in Fig. 5, a predetermined event PE with an event ID = EI3 occurs at time T1. Also, a predetermined event PE with an event ID = EI1 occurs at time T2. Note that both the predetermined events PE with event ID = EI1 and event ID = EI3 correspond to predetermined events PE with an "On" event judgment (i.e., collection target events CTE) (see Fig. 4).

[0049] When a predetermined event PE occurring at a certain time corresponds to a collection target event CTE, the log data LOG of this collection target event CTE is extracted for a predetermined collection time CT. The extracted log data LOG is stored in the storage device 120. This log data LOG corresponds to the verification data LOG_VRF. The upper part of Fig. 5 shows the log data LOG including the sensor detection information SEN, the control amount CON, the recognition result information RES, and the target trajectory TRJ.

[0050] The collection time CT during which the log data LOG is extracted and stored is set based on the time when it is determined that the collection target event CTE has occurred. In the example shown in Fig. 5, it is determined that two types of collection target events CTE have occurred at times T1 and T2. The collection time CT1 for the first collection target event CTE (event ID = EI3) includes a time zone Z1 before time T1 and a time zone Z2 after time T1. The collection time CT2 for the second collection target event CTE (event ID = EI1) includes a time zone Z3 before time T2 and a time zone Z4 after time T2.

[0051] The length of each of the time periods Z1, Z2, Z3, and Z4 is, for example, 1 to 10 seconds. However, it may take time to determine whether or not a specific event PE corresponds to a collection target event CTE. Therefore, it is expected that the time at which it is determined that the collection target event CTE has occurred will be slightly later than the time at which the collection target event CTE occurred. Therefore, it is desirable to set the length of each of the time periods Z1, Z2, Z3, and Z4 (i.e., the length of the collection times CT1 and CT2) for each content (event ID) of the collection target event CTE.

[0052] In addition, in the case of an operator intervention event in automatic driving control, such as a request for a driver changeover by the operator, it is expected that the operator will feel something is wrong before the reference time. In addition, it is considered that the frequency of intervention in automatic driving control varies depending on the driving proficiency of the operator. Therefore, in the case of an operator intervention event in automatic driving control, it is desirable to set the collection time CT according to the driving proficiency. For example, it is considered that the higher the proficiency of manual driving, the shorter the time to decide to intervene will be, and therefore, an example is considered in which the time period (time period Z1 or Z3 in FIG. 5) before the reference time (time T1 or T2 in FIG. 5) is set to be shorter. Alternatively, it is considered that the higher the proficiency of automatic driving, the longer the time to decide to intervene will be, and therefore, an example is considered in which the time period before the reference time is set to be longer.

[0053] 3. Example of processing by the processor 3-1. Processing example in the autonomous driving system 100 6 is a flowchart particularly related to the present embodiment, showing a flow of processing by the processor 110. The routine shown in FIG 6 is executed, for example, immediately after the autonomous driving system 100 starts operating.

[0054] In the routine shown in Fig. 6, first, the startup data IGN is transmitted to the management server 200 (step S11). The startup data IGN is data for notifying the management server 200 that the ignition of the managed object 1 is turned ON. The startup data IGN includes, for example, identification data of the managed object 1, version data of the autonomous driving system 100, and the like. The version data also includes version data of the trained model applied to the autonomous driving system 100. The startup data IGN may include verification data LOG_VRF that has not yet been transmitted to the management server 200.

[0055] Following the process of step S11, it is determined whether or not a confirmation signal SCF has been received (step S12). The confirmation signal SCF is a signal that notifies the managed object 1 that the management server 200 has confirmed the startup data IGN. The process of step S12 is repeated until a positive determination result is obtained.

[0056] If the determination result of step S12 is positive, it is determined whether or not update data UPD has been transmitted (step S13). When updating the trained model applied to the autonomous driving system 100, a signal notifying that update data UPD has been transmitted is added to the confirmation signal SCF received in the processing of step S12. Therefore, if this additional signal is recognized, it is determined that update data UPD has been transmitted.

[0057] If the determination result in step S13 is positive, the update data UPD is updated (step S14). When the process in step S14 is performed, it is desirable to notify the operator of the managed object 1 through the user interface that there is a possibility that part of the automatic driving control of the managed object 1 may not be performed until the update of the update data UPD is completed.

[0058] Following the processing of step S14, a completion signal SUP is transmitted to the management server 200, and counting of the verification distance LVRF is started (step S15). The completion signal SUP is a signal for notifying the management server 200 that the update of the update data UPD has been completed. In other words, the completion signal SUP is generated and transmitted to the management server 200 when the update of the update data UPD is completed in the managed object 1. The verification distance LVRF is set in order to collect the verification data LOG_VRF.

[0059] Following the process of step S15, it is determined whether the verification distance LVRF is equal to or less than the threshold LTH (step S16). The threshold LTH corresponds to the above-mentioned "predetermined travel distance (e.g., 5 to 10 km) of the managed object 1 after the transmission of the update data UPD." If the determination result of step S16 is negative, that is, if the verification distance LVRF exceeds the threshold LTH, the collection of the update data UPD is terminated.

[0060] If the determination result in step S16 is positive, it is determined whether the collection condition is satisfied (step S17). Whether the collection condition is satisfied can be determined based on whether a collection target event CTE has occurred. As already explained, a collection target event CTE is an event among the predetermined events PE, the log data LOG during which is stored in the storage device 21. Therefore, it is determined whether the predetermined event PE has occurred, and if it is determined that the predetermined event PE has occurred, the instruction data INS_LOG explained in FIG. 4 is referenced. This makes it possible to determine whether a collection target event CTE has occurred.

[0061] If the determination result in step S17 is negative, the process returns to step S16. On the other hand, if the determination result in step S17 is positive, the log data LOG is stored in the storage device 120 (step S18). In the process of step S18, first, based on the time when it is determined that the collection target event CTE has occurred, a predetermined period is extracted from the log data LOG of this collection target event CTE. The concept of this predetermined period is as explained in FIG. 5. Then, the extracted log data LOG is stored in the storage device 120. This log data LOG corresponds to the above-mentioned verification data LOG_VRF. The verification data LOG_VRF stored in the storage device 120 is appropriately transmitted to the management server 200.

[0062] Following the processing of step S18, it is determined whether or not the verification distance LVRF exceeds the threshold value LTH (step S19). The determination content of step S19 is the opposite of that of step S16. If the determination result of step S19 is negative, that is, if the verification distance LVRF is equal to or less than the threshold value LTH, the processing of steps S17 and thereafter is performed. If the determination result of step S19 is positive, the collection of the update data UPD is terminated. In this way, if there is an update of the update data UPD, the processing of steps S16 to S19 is repeated to collect the update data UPD.

[0063] 3-2. Example of Processing in the Management Server 200 7 is a flowchart particularly related to this embodiment, showing the flow of processing by processor 210. The routine shown in FIG 7 is repeatedly executed at a predetermined control period.

[0064] 7, first, it is determined whether or not the start-up data IGN has been received from the managed object 1 (step S21). The start-up data IGN has already been described. If the start-up data IGN includes the verification data LOG_VRF, the verification data LOG_VRF is appropriately stored in the storage device 220.

[0065] Following the process of step S21, it is determined whether or not it is necessary to transmit update data UPD (step S22). In the process of step S22, for example, it is determined whether or not it is necessary to upgrade the trained model based on the startup data IGN (the version data of the trained model) received in the process of step S21.

[0066] If the determination result in step S22 is negative, a confirmation signal SCF is sent to the managed object 1 (step S23). The confirmation signal SCF has already been described. On the other hand, if the determination result in step S22 is positive, in addition to the confirmation signal SCF, update data UPD and instruction data INS_LOG are sent to the managed object 1 (step S24). The instruction data INS_LOG has already been described.

[0067] Following the process of step S24, it is determined whether or not a completion signal SUP has been received from the managed object 1 (step S25). The completion signal SUP has already been described. The process of step S25 is repeated until a positive determination result is obtained.

[0068] If the result of the determination in step S25 is positive, counting of the verification period PVRF is started (step S26). The verification period PVRF is set for collecting verification data LOG_VRF.

[0069] Following the process of step S26, it is determined whether or not the verification conditions are satisfied (step S27). The verification conditions include, for example, the following conditions. (i) The PVRF during the verification period is equal to or greater than the threshold PTH. (ii) A significant amount of validation data LOG_VRF has been obtained. (iii) At least two managed vehicles 1 have completed the verification distance LVRF. The threshold value PTH corresponds to the above-mentioned "predetermined period (for example, one week, one month) after the update data UPD is transmitted."

[0070] For example, if conditions (i) and (ii) are satisfied, it is determined that the verification conditions are satisfied. In another example, if conditions (i) to (iii) are all satisfied, it is determined that the verification conditions are satisfied. If the judgment result of step S27 is positive, verification of the autonomous driving function is performed (step S28). The verification of the autonomous driving function is performed using a verification index set in advance. Examples of verification and examples of verification indexes have already been described. [Explanation of symbols]

[0071] 1...vehicle (managed object), 10...sensor group, 20...recognition unit, 30...planning unit, 40...control amount calculation unit, 50...driving device, 100...autonomous driving system, 110, 210...processor, 120, 220...storage device, 200...management server, CON...control amount, INS_LOG...instruction, LOG...log data, RES...recognition result information, SEN...sensor detection information, TRJ...target trajectory, UPD...update data, LOG_VRF...verification data, CT1, CT2...collection time

Claims

1. 1. A method for managing automated driving functions in a vehicle, comprising: A step in which a management server transmits update data indicating data for updating the trained model to at least two vehicles in which autonomous driving control is executed using a trained model indicating a model generated by machine learning; The management server collects, from each of the at least two vehicles to which the update data has been transmitted, validation data indicating data acquired or generated in association with the execution of the autonomous driving control using the updated trained model; The management server verifies a function of autonomous driving control using the updated trained model by using the verification data collected from the at least two vehicles; A method for managing an autonomous driving function, comprising:

2. 2. The method of claim 1 , A processor that executes the autonomous driving control of one of the managed objects included in the at least two vehicles, during the autonomous driving control using the updated trained model, when a predetermined collection condition is satisfied, collects, as the verification data for the managed object, data that the managed object has acquired or generated in relation to the execution of the autonomous driving control within a predetermined collection time based on the timing at which the collection condition is satisfied; a step of the managed object transmitting the verification data for the managed object to the management server; The method for managing an autonomous driving function further comprises:

3. 3. The method of claim 2, The collection condition includes that an operator of the managed object has intervened in the automatic driving control, A method for managing an autonomous driving function in a vehicle, characterized in that the managed object further includes a step of variably setting the length of the collection time based on the timing at which the collection condition is satisfied, based on the driving proficiency of the operator of the managed object.

4. 4. The method according to claim 2 or 3, A method for managing an autonomous driving function, characterized in that the collection conditions include that the driving environment conditions of the managed object match pre-set conditions.

5. 3. The method of claim 2, The collection condition includes that an operator of the managed object has intervened in the automatic driving control, In the step of verifying the autonomous driving function, the number of interventions in the autonomous driving control acquired during a predetermined verification period after the transmission of the update data or during a period until the mileage of the managed object after the transmission of the update data reaches a predetermined verification distance is used as a verification index. A method for managing an autonomous driving function.

6. 6. The method of claim 5, When an intervention in the automatic driving control by an operator of the managed object is present in at least two of the managed objects, in the step of verifying the automatic driving function, the number of interventions in the automatic driving control as the verification index is calculated by weighting according to the driving proficiency of the operator of the at least two managed objects. A method for managing an autonomous driving function.

7. A system for managing an autonomous driving function in a vehicle, A management server that manages the autonomous driving function, The management server, A process of transmitting update data indicating data for updating a trained model indicating a model generated by machine learning to at least two vehicles in which autonomous driving control is performed using the trained model; A process of collecting validation data indicating data acquired or generated in association with execution of the autonomous driving control using the updated trained model from each of the at least two vehicles to which the update data has been transmitted; A process of verifying a function of autonomous driving control using the updated trained model by using the verification data collected from the at least two vehicles; The management system for autonomous driving functions is characterized by:

8. 8. The system of claim 7, The at least two vehicles further include a processor that executes the autonomous driving control of one of the managed vehicles, The processor, During the autonomous driving control using the updated trained model and when a predetermined collection condition is satisfied, a process of collecting data acquired or generated by the managed object in relation to the execution of the autonomous driving control within a predetermined collection time based on the timing when the collection condition is satisfied, as the verification data for the managed object; a process of transmitting the verification data for the management target to the management server; The management system for autonomous driving functions is characterized by:

Citation Information

Patent Citations

  • Learning system, on-vehicle device, and server

    JP2015135552A

  • Information collection device, information collection method, and program

    JP2016071497A

  • Travel control device, vehicle, travel control system, travel control method, and program

    JP2019156171A

  • Re-learning data management system and re-learning data management method

    JP2022150778A

  • Customized data request system and method of using

    JP2023057532A