Model generation method, data collection method, and computer program product

Abstract

The present disclosure relates to a model generation method, a data collection method, and a computer program product, and provides a technique for obtaining a trained machine learning model that obtains the ability to perform moving body control at an appropriate reaction speed. A model generation method according to one aspect of the present disclosure includes acquiring, by a computer, a plurality of data sets each comprising a combination of training data and positive solution data, and performing machine learning of a control model using the plurality of acquired data sets. The use of the plurality of data sets includes the more preferential use of a data set that is evaluated to be appropriate due to the response speed of the control command represented by the positive solution data to the event satisfying a predetermined condition.

Description

Technical Field

[0001] The present disclosure relates to a model generation method, a data collection method and a computer program product including a control program. Background Art

[0002] Patent document 1 proposes a driving assistance device, which is configured to obtain information representing a driving operation and information representing a driving condition during the driving operation, determine whether the driving condition is suitable for learning based on the obtained information, and determine that a driving operation under a driving condition determined to be unsuitable is not an object of learning.

[0003] Prior art literature

[0004] Patent Document 1: Japanese Patent Application Publication No. 2019-127207 Summary of the invention

[0005] Problems to be solved by the invention

[0006] The purpose of the present disclosure is to provide the following technology: a technology for increasing the possibility of obtaining a trained machine learning model that has acquired the ability to perform (realize) control of a moving body at an appropriate response speed, or a control technology for a moving body using the trained machine learning model obtained thereby.

[0007] Technical solutions to solve problems

[0008] The model generation method involved in the first technical solution of the present disclosure is executed by a computer. The model generation method includes: obtaining multiple data sets (data sets) each consisting of a combination of training data and correct answer (correct answer) data, wherein the training data is data representing the environment in which the mobile body moves in a time series, and the correct answer data is data representing the control instructions for the mobile body in the environment in a time series; and using the multiple data sets obtained to implement machine learning of the control model. Implementing the machine learning includes: for each of the data sets, training the control model in such a way that the result of deriving the control instructions for the mobile body based on the training data using the control model conforms to the correct answer data. Moreover, using the multiple data sets includes: the more the reaction speed of the control instructions represented by the correct answer data to the event is evaluated as appropriate because it meets predetermined conditions, the more preferential use is given. The control model can be composed of a neural network.

[0009] The data collection method involved in the second technical solution of the present disclosure is executed by a computer. The data collection method includes: collecting multiple data sets each consisting of a combination of training data and correct answer data, wherein the training data is data representing the environment in which the mobile body moves in a time series, and the correct answer data is data representing the control instructions for the mobile body in the environment in a time series; and outputting the collected multiple data sets for machine learning. Moreover, collecting the multiple data sets includes: the more the reaction speed of the control instructions represented by the correct answer data to the event is evaluated as appropriate because it meets the predetermined conditions, the more priority is given to collecting the data sets.

[0010] The control program involved in the third technical solution of the present disclosure is used to make a computer execute: obtaining object data representing the environment in which the target mobile body moves; using a trained control model, deriving a control instruction based on the obtained object data; and controlling the action of the target mobile body according to the result of deriving the control instruction. The trained control model is a model generated by implementing machine learning using a plurality of data sets each consisting of a combination of training data and correct answer data, wherein the training data is data representing the environment in which the training mobile body moves in a time series, and the correct answer data is data representing the control instruction for the training mobile body in the environment in a time series. Implementing the machine learning includes: for each of the data sets, training the control model in such a way that the result of deriving the control instruction of the mobile body based on the training data using the control model conforms to the correct answer data. Moreover, using the plurality of data sets for the machine learning includes: the more the reaction speed of the control instruction represented by the correct answer data to the event is evaluated as appropriate because it conforms to a predetermined condition, the more preferentially the data set is used for the machine learning.

[0011] Effects of the Invention

[0012] According to the present disclosure, it is possible to provide the following technology: technology for increasing the possibility of obtaining a trained machine learning model that has acquired the ability to control a moving body at an appropriate reaction speed, or technology for controlling a moving body using the trained machine learning model obtained thereby. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 An example of a scenario to which the present disclosure is applied is schematically shown.

[0014] Figure 2 An example of the hardware configuration of the model generation device according to the embodiment is schematically shown.

[0015] Figure 3 An example of the hardware configuration of the control device according to the embodiment is schematically shown.

[0016] Figure 4 An example of the software configuration of the model generation device according to the embodiment is schematically shown.

[0017] Figure 5 An example of a software configuration of the control device according to the embodiment is schematically shown.

[0018] Figure 6 This is a flowchart showing an example of a processing procedure related to machine learning of a control model by the model generation device involved in the embodiment.

[0019] Fig. 7A An example of events according to the embodiment is schematically shown.

[0020] Figure 7B Schematically represents the evaluation Fig. 7A An example of a method for measuring the reaction speed in an event.

[0021] Fig. 8A An example of events according to the embodiment is schematically shown.

[0022] Figure 8B Schematically represents the evaluation Fig. 8A An example of a method for measuring the reaction speed in an event.

[0023] Fig. 9 An example of events according to the embodiment is schematically shown.

[0024] Fig. 10A An example of events according to the embodiment is schematically shown.

[0025] Fig. 10B Schematically represents the evaluation Fig. 10A An example of a method for measuring the reaction speed in an event.

[0026] Fig.11A An example is schematically shown in which a data set having a more appropriate reaction speed according to a predetermined condition is used with higher priority in the embodiment.

[0027] Fig. 11B An example is schematically shown in which a data set having a more appropriate reaction speed according to a predetermined condition is used with higher priority in the embodiment.

[0028] Fig.12 This is a flowchart showing an example of a processing procedure related to the motion control of a moving object by the control device according to the embodiment.

[0029] Fig.13 An example of another scenario to which the present disclosure is applied is schematically shown.

[0030] Fig.14An example of a hardware configuration of a data collection device according to another embodiment is schematically shown.

[0031] Fig.15 An example of a software configuration of a data collection device according to another embodiment is schematically shown.

[0032] Fig.16 This is a flowchart showing an example of a processing procedure related to data collection by a data collection device according to another embodiment.

[0033] Description of symbols

[0034] 1. Model generating device;

[0035] 11 control unit; 12 storage unit; 13 communication interface (I / F);

[0036] 14 input device; 15 output device; 16 driver;

[0037] 81model generation program; 91storage medium;

[0038] 111 learning data acquisition unit; 112 learning processing unit;

[0039] 113 storage processing unit; 125 learning result data;

[0040] 2. Control device;

[0041] 21 control unit; 22 storage unit; 23 communication interface (I / F);

[0042] 24 input device; 25 output device; 26 driver;

[0043] 27 external interface (I / F);

[0044] 82 control program; 92 storage medium;

[0045] 211 acquisition unit; 212 export unit;

[0046] 213 action control unit; 221 object data;

[0047] 3. Data collection device;

[0048] 31 control unit; 32 storage unit; 33 communication interface;

[0049] 34 input device; 35 output device; 36 driver;

[0050] 37 external interfaces;

[0051] 83Data collection procedures; 93Storage media;

[0052] 311 collection unit; 312 output unit;

[0053] 4 datasets;

[0054] 41 training data; 45 correct answer data;

[0055] 5. Control model;

[0056] M moving body; S sensor. DETAILED DESCRIPTION

[0057] According to the method proposed in Patent Document 1, by excluding driving operations in driving conditions determined to be inappropriate from the learning object, it is expected that an autonomous driving model capable of performing only appropriate driving operations can be generated. However, the inventors of the present application have found that the conventional method has the following problems.

[0058] That is, imagine a scenario in which a model acquires the ability to perform autonomous driving of a vehicle through machine learning. In this case, the ability acquired by the trained machine learning model depends on the learning data used for machine learning. The learning data may be collected from various drivers. At this time, the driver's ability is not necessarily fixed. For example, there are drivers who react quickly to events such as the deceleration of the vehicle in front (execution of driving operations), and on the other hand, there are drivers who react slowly to the event. Therefore, the reaction speed of the driver represented by the collected learning data may be dispersed.

[0059] In this regard, in the previous methods, driving operations that are judged as inappropriate are simply excluded from the learning object. Even if the driving operation is appropriate, there may be differences in reaction speed. Therefore, in the previous methods, it is not necessarily possible to obtain a trained machine learning model that has acquired the ability to perform autonomous driving at an appropriate reaction speed.

[0060] In addition, this problem may occur regardless of the type of vehicle (for example, the number of wheels (two-wheeled vehicles, four-wheeled vehicles, etc.), size (large, ordinary, small, etc.), power source (electricity, fuel, etc.), etc.). In addition, such problems are not limited to scenarios where vehicles are controlled. The same applies to moving objects other than vehicles in terms of controlling movement. Therefore, the same problem may occur in scenarios where all moving objects other than vehicles (for example, aircraft (drones, etc.), ships, etc.) are controlled.

[0061] In this regard, the model generation method involved in the first technical solution of the present disclosure is executed by a computer. The model generation method includes: obtaining multiple data sets each consisting of a combination of training data and correct answer data, wherein the training data is data representing the environment in which the mobile body moves in a time series, and the correct answer data is data representing the control instructions for the mobile body in the environment in a time series; and using the multiple data sets obtained to implement machine learning of the control model. Implementing the machine learning includes: for each of the data sets, training the control model in such a way that the result of deriving the control instructions for the mobile body based on the training data using the control model conforms to the correct answer data. Moreover, using the multiple data sets includes: the more the reaction speed of the control instructions represented by the correct answer data to the event is evaluated as appropriate because it meets predetermined conditions, the more priority is given to using the data set.

[0062] The ability of a trained model generated by machine learning depends on the data set used for the machine learning. In the first technical solution of the present disclosure, the more appropriate the reaction speed of the data set is, the more preferentially it is used for machine learning. Therefore, the possibility of obtaining a trained machine learning model that has acquired the ability to control a mobile body at an appropriate reaction speed can be increased.

[0063] In addition, the data collection method involved in the second technical solution of the present disclosure is executed by a computer. The data collection method includes: collecting multiple data sets each consisting of a combination of training data and correct answer data, wherein the training data is data representing the environment in which the mobile body moves in a time series, and the correct answer data is data representing the control instructions for the mobile body in the environment in a time series; and outputting the collected multiple data sets for machine learning. Moreover, collecting the multiple data sets includes: the more the reaction speed of the control instructions represented by the correct answer data to the event is evaluated as appropriate because it meets the predetermined conditions, the more priority is given to collecting the data sets.

[0064] In the second technical solution of the present disclosure, the more appropriate the reaction speed is, the more priority is given to the collection of the data set, thereby, similar to the first technical solution, the more appropriate the reaction speed is, the more priority is given to the use of the data set for machine learning. Therefore, the possibility of obtaining a trained machine learning model that has acquired the ability to control a mobile body at an appropriate reaction speed can be increased.

[0065] In addition, the control program involved in the third technical solution of the present disclosure is used to make a computer execute: obtaining object data representing the environment in which the target mobile body moves; using a trained control model, deriving a control instruction based on the obtained object data; and controlling the action of the target mobile body according to the result of deriving the control instruction. The trained control model is a model generated by implementing machine learning using a plurality of data sets each consisting of a combination of training data and correct answer data, wherein the training data is data representing the environment in which the training mobile body moves in a time series, and the correct answer data is data representing the control instruction for the training mobile body in the environment in a time series. Implementing the machine learning includes: for each of the data sets, training the control model in such a way that the result of deriving the control instruction for the mobile body based on the training data using the control model conforms to the correct answer data. Moreover, using the plurality of data sets for the machine learning includes: the more the reaction speed of the control instruction represented by the correct answer data to the event is evaluated as appropriate because it conforms to a predetermined condition, the more preferentially the data set is used for the machine learning.

[0066] As in the above-mentioned technical solutions, by giving priority to data sets whose reaction speed is evaluated as appropriate and using them for machine learning, a trained machine learning model that has acquired the ability to control a moving body at an appropriate reaction speed can be obtained. According to the third technical solution of the present disclosure, by using such a trained control model (machine learning model), it can be expected that the moving body can be controlled at an appropriate reaction speed.

[0067] Hereinafter, an embodiment (hereinafter also referred to as "this embodiment") relating to one aspect of the present disclosure is described based on the accompanying drawings. However, the present embodiment described below is merely an illustration of the present disclosure in all respects. Various improvements or modifications may be made without departing from the scope of the present disclosure. It is also possible to appropriately adopt specific structures corresponding to the embodiment when implementing the present disclosure. In addition, although the data appearing in this embodiment is described in natural language, more specifically, it is specified by pseudo-language, instructions, parameters, machine language, etc. that can be recognized by a computer.

[0068] [1 Application example]

[0069] Figure 1 An example of a scenario to which the present disclosure is applied is schematically shown. A system according to the present embodiment includes a model generation device 1 and a control device 2 .

[0070] The model generation device 1 involved in this embodiment is one or more computers configured to generate a trained control model 5 by implementing machine learning. In this embodiment, the model generation device 1 obtains a plurality of data sets 4 each consisting of a combination of training data 41 and correct answer data 45. The training data 41 is configured to represent the environment in which the mobile body (training mobile body) moves in a time series. The correct answer data 45 is configured to represent the true value of the control instruction for the mobile body in the environment represented by the corresponding training data 41 in a time series.

[0071] The model generation device 1 uses the obtained multiple data sets 4 to implement machine learning of the control model 5. The implementation of machine learning includes: for each data set 4, the control model 5 is trained in such a way that the result of deriving the control instruction of the mobile body based on the training data 41 using the control model 5 conforms to the corresponding correct data 45. Moreover, in the machine learning, the use of the multiple data sets 4 includes: the more the reaction speed of the control instruction represented by the correct data 45 to the event conforms to the predetermined condition, so that the reaction speed is evaluated as appropriate, the more preferentially used. Through this machine learning, it is possible to generate a trained control model 5 that has acquired the ability to derive control instructions according to the environment in which the mobile body moves.

[0072] On the other hand, the control device 2 according to the present embodiment is one or more computers configured to control the movement of the target moving body M using the trained control model 5. In the present embodiment, the control device 2 obtains object data 221 representing the environment in which the target moving body M moves. The control device 2 uses the trained control model 5 to derive a control command based on the obtained object data 221. Then, the control device 2 controls the movement of the target moving body M according to the result of deriving the control command.

[0073] As described above, in the present embodiment, in the model generation device 1, the more appropriate the response speed is, the more preferentially the data set is used for machine learning, thereby generating the trained control model 5. Since the ability of the trained model generated by machine learning depends on the data set used for the machine learning, according to the present embodiment, it is expected to obtain the trained control model 5 that has mastered the ability to control the moving body at an appropriate response speed. In addition, in the control device 2 according to the present embodiment, by using such a trained control model 5, it is expected that the moving body M as the object can be controlled at an appropriate response speed.

[0074] (Moving body)

[0075] As long as it can be automatically moved by mechanical control, the type of the mobile body (mobile body M) is not particularly limited and can be appropriately selected according to the embodiment. The mobile body (mobile body M) can be, for example, any movable device such as a vehicle, an aircraft, a ship, a robot device, etc. The aircraft can be at least one of an unmanned aircraft such as a drone and a manned aircraft.

[0076] In one example, Figure 1 As shown, the mobile object (mobile object M) may be a vehicle. In this case, in the model generation device 1, it is expected that a trained control model 5 that has mastered the ability to control the vehicle at an appropriate reaction speed can be obtained. In addition, in the control device 2, by using such a trained control model 5, it is expected that the target vehicle can be controlled at an appropriate reaction speed.

[0077] In addition, when the mobile body is a vehicle, the type of vehicle can be arbitrarily selected. The vehicle can be selected, for example, from a two-wheeled vehicle, a three-wheeled vehicle, a four-wheeled vehicle, etc. The power source of the vehicle can be selected, for example, from electricity, fuel, etc. In the case where the vehicle is a car, the size of the vehicle can be selected from large, medium, quasi-medium, ordinary, special large, special small, etc. In the case where the vehicle is a two-wheeled vehicle, the size of the vehicle can be selected from large, ordinary, etc. As a typical example, the mobile body (mobile body M) can be a car with an autonomous driving capability of level 2 (L2) or above.

[0078] (Environment / Sensor)

[0079] The environment is an event / phenomenon observed by at least one of the mobile body itself and the surroundings. In one example, at least a part of the environment can be observed by one or more sensors S disposed inside or outside the mobile body (mobile body M). Accordingly, the training data 41 and the object data 221 can each include sensor data SD obtained by one or more sensors S.

[0080] As long as it can observe any environment in which the moving body moves, the type of sensor S is not particularly limited and can be appropriately selected according to the implementation method. In one example, one or more sensors S may include a camera (image sensor), a radar, a LiDAR (Light Detection And Ranging), a sonar (ultrasonic sensor), an infrared sensor, a GNSS (Global Navigation Satellite System) / GPS (Global Positioning Satellite) module, etc.

[0081] (Control Instructions)

[0082] The control instruction is related to the movement of the mobile body. The structure of the control instruction can be appropriately determined according to the implementation method. For example, the control instruction can be composed of acceleration, deceleration, steering, or a combination thereof. Acceleration and deceleration can include gear changes. In this case, in the model generation device 1, it can be expected to obtain a trained control model 5 that has mastered the ability to perform acceleration, deceleration, steering, or a combination thereof at an appropriate reaction speed. In addition, in the control device 2, by using such a trained control model 5, it can be expected that acceleration, deceleration, steering, or a combination thereof can be performed at an appropriate reaction speed.

[0083] In one example, when the mobile body (mobile body M) is a vehicle, the control instruction may include at least one of acceleration, deceleration, and steering of the vehicle. When at least one of acceleration, deceleration, and steering is included, the control instruction may be represented by a path. Accordingly, the control model 5 may be represented as a path planner.

[0084] In addition, the control instructions may also include instructions related to the operation of the mobile body. As an example, when the mobile body (mobile body M) is a vehicle, the control instructions may include vehicle operations such as turn indicators, hazard warning lights, horns, and communication processing (for example, sending data to a center, sending an emergency call, etc.).

[0085] (Dataset)

[0086] Each data set 4 can be generated appropriately. Each data set 4 can be automatically generated by the operation of a computer, or can be manually generated by at least partially including the operation of an operator. Typically, the operation data and environmental data of the mobile body can be collected during the period when the subject controls the mobile body entirely by manual operation. The environmental data (observation data) can be obtained by a sensor S mounted on the mobile body. The operation data can be obtained by recording the manual operation of the subject. Moreover, the training data 41 of each data set 4 can be generated based on the environmental data. The correct answer data 45 can be generated based on the operation data. That is, typically, the data set 4 can be generated based on the result of the subject's manual operation (for example, manual driving of a vehicle). In the case of obtaining the data set 4 using a mobile body, the mobile body that becomes the object of obtaining the data set 4 may include the mobile body M that becomes the object of using the generated trained control model 5, or may not include the mobile body M. That is, the training mobile body may include the mobile body M as the object, or may not include the mobile body M as the object. The training mobile body may include a mobile body used by an end user. In this case, the end user may be a subject. In addition, the training mobile body may include a mobile body used experimentally.

[0087] However, the method of generating the data set 4 may not be limited to such an example, and may be appropriately selected according to the implementation method. In another example, the data set 4 may be generated according to the result of the manual operation of the subject in the same manner as in the above example, but the manual operation may include the subject's operation in the partial automatic control action such as the override (override, temporarily invalidating the automatic control and using manual control instead) operation for any automatic control. In another example, at least a portion of the data set 4 may be obtained by virtual methods such as simulation (emulation). In another example, at least a portion of the data set 4 may also be obtained by the framework of reinforcement learning. In addition, in another example, at least a portion of the data set 4 may also be obtained by data augmentation (Data Augmentation) of any data set. Data augmentation is formed by generating new training data by changing the attribute values ​​of the training data. For example, in the case where the training data includes an image, the parameter change may be composed of image processing such as parallel movement, enlargement, reduction, rotation, and noise addition to the image. In the case of obtaining at least a portion of the data set 4 by data expansion, the values ​​of other attributes other than at least one of the attributes dependent on the reaction speed and the attributes of the training data of any data set (original data set) may be changed, and the corresponding correct answer data may be assigned, thereby generating more than one new data set. The plurality of data sets 4 may include the generated new data sets.

[0088] (Control Model)

[0089] The control model 5 is composed of a machine learning model having one or more operation parameters that can be adjusted by machine learning. One or more operation parameters are used for the operation of the target reasoning (in this case, the derivation of control instructions). Machine learning refers to the use of learning data (in this case, multiple data sets 4) to adjust (optimize) the value of the operation parameter. The structure and type of the machine learning model are not particularly limited and can be appropriately selected according to the implementation method. The machine learning model can be composed of, for example, a neural network, a support vector machine, a regression model, a decision tree model, etc.

[0090] As an example, the control model 5 can be composed of a neural network. The structure of the neural network can be appropriately determined according to the implementation method. The structure of the neural network can be determined, for example, by the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) contained in each layer, and the connection relationship between the nodes of each layer. In one example, the neural network can also have a recursive structure. In addition, the neural network can include any layer such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, a depooling layer, a normalization layer, a dropout layer, and LSTM (Long short-term memory). The neural network can also have any mechanism such as an Attention mechanism. The control model 5 (neural network) can include any model such as GNN (Graph neural network), diffusion model, generation model (for example, Generative Adversarial Network, Transformer, etc.). When the neural network is used for the control model 5, the connection weights between the nodes contained in the control model 5 (neural network) and the threshold of each node are examples of operation parameters.

[0091] As long as the control instructions can be derived according to the environment of the mobile body, the input and output methods of the control model 5 may not be particularly limited and may be appropriately selected according to the implementation method. For example, the control model 5 may be configured to derive control instructions for more than one time point based on environmental data at more than one time point. In addition, the control model 5 may be configured to accept time series data according to its structure. As an example, by being configured as a recursive type, the control model 5 may be configured to accept time series data. As another example, the control model 5 may also be configured to input environmental data of multiple time points at a time. Alternatively, the control model 5 may be configured to be unable to accept time series data in terms of structure. For example, the control model 5 may be configured to derive control instructions for one time point based on environmental data at one time point. In this case, the control model 5 may be used in the following manner: sequentially receive data at each time point in the time series data, and sequentially output the calculation results, thereby obtaining the calculation results relative to the time series data. Furthermore, the control model 5 may be configured to derive control instructions in real time. Alternatively, the control model 5 may be configured to derive control instructions for multiple future time points at a time. In this case, at least a part of the control command derived together can be used for controlling the moving object (moving object M).

[0092] As long as the control model 5 participates in at least a part of the reasoning process of deriving the control command according to the environment of the mobile body, the processing content performed by the control model 5 may not be particularly limited and may be appropriately selected according to the implementation method. In one example, the control model 5 may be configured to perform surrounding recognition and path planning (path / track planning). The control model 5 may also be configured to also perform motion planning (action / control planning). That is, the control model 5 may be an end-to-end model.

[0093] In addition, as long as the action of the mobile body can be controlled according to the output of the control model 5, the output form of the control model 5 can be appropriately selected according to the implementation method. The control model 5 can also be configured to directly output the control instruction. Alternatively, the control instruction can also be obtained by performing any information processing (interpretation processing) on ​​the output of the control model 5. The control instruction can also be configured to directly represent the control amount of the mobile body such as the acceleration control amount, the braking control amount, the steering wheel steering angle (control instruction value, control output amount). Alternatively, the control instruction can be configured to indirectly represent the control amount of the mobile body such as the path, the state after control, etc. In this case, the control amount of the mobile body can be obtained according to the control instruction by performing any information processing. In one example, when the mobile body is a vehicle, the control amount of the vehicle can be obtained by applying the inference result obtained from the control model 5 to the vehicle model. The vehicle model can have various parameters such as acceleration, braking, steering wheel steering, etc., and can be appropriately configured to derive the control amount based on indirect information (path, state after control, etc.).

[0094] In addition, the correct answer data 45 of each data set 4 can be appropriately configured to directly or indirectly represent the control instruction. In the case where the control instruction is derived from the output of the control model 5 by performing any operation processing, the correct answer data 45 can also be given for the control instruction derived from the output of the control model 5 (that is, the correct answer data 45 can be configured to directly represent the control instruction). Alternatively, the correct answer data 45 can be given for the output of the control model 5 (that is, the correct answer data 45 can be configured to indirectly represent the control instruction).

[0095] (event)

[0096] Events may include all events / phenomena that may be associated with the operation of a mobile body. In addition, events may include all events / phenomena that can be detected by a sensor S. Detection by a sensor S means making a determination based on a sensor value. That is, the start time of an event can be determined based on sensor data. The detection (determination, determination) method can be appropriately determined based on the event. The sensor data can be analyzed by any method, thereby determining the start time of the event (that is, the occurrence of the event can be detected).

[0097] Accordingly, the mobile body (mobile body M) can be equipped with a sensor S. The training data 41 can include sensor data SD obtained by the sensor S. Moreover, the start time of the event in the training data 41 can be determined based on the sensor data SD. Thus, the reaction speed in each data set 4 can be mechanically evaluated, so that the efficiency of the judgment of the data set 4 that is preferentially used in machine learning can be achieved. That is, the work of selecting the data set 4 that is preferentially used can be at least partially automated, thereby reducing time and labor. In addition, the start time of the operation for the event is represented in the correct answer data 45. Therefore, in this method, the reaction speed to the event (that is, the time from the time of occurrence of the event to the start time of the operation) can be determined based on the training data 41 and the correct answer data 45.

[0098] (Example of event)

[0099] As an example, when the moving body (moving body M) is a vehicle, the event may include at least one of the deceleration of the preceding vehicle relative to the vehicle, the cutting in (cut in, merging) of a parallel (parallel running) vehicle, the appearance of a parked / stopped vehicle, the generation of an obstacle, and the change of a signal (traffic light). Obstacles may include all objects that have the potential to hinder the travel of the vehicle. Obstacles may be, for example, pedestrians, bicycles, etc. In this case, in the model generation device 1, it is expected to obtain a trained control model 5 that has mastered the ability to control the vehicle at an appropriate reaction speed for at least any one of these events. In addition, in the control device 2, by using such a trained control model 5, it is expected that the control of the target vehicle can be implemented at an appropriate reaction speed for at least any one of these events.

[0100] In addition, in the case where the event as the object of automatic driving (automatic control) includes the deceleration of the leading vehicle, the control instruction may include an instruction for deceleration corresponding to the leading vehicle. In the case where the event as the object includes the cutting in of a parallel vehicle, the control instruction may include an instruction for deceleration corresponding to the parallel vehicle. In the case where the event as the object includes the appearance of a parked / parked vehicle, the control instruction may include at least any one of deceleration and turning corresponding to the parked / parked vehicle. In the case where the event as the object includes the generation of an obstacle, the control instruction may include at least any one of deceleration and turning corresponding to the obstacle. In the case where the event as the object includes the change of a traffic light, the control instruction may include an instruction for acceleration or deceleration corresponding to the traffic light. These events can be adapted for other types of moving objects (e.g., aircraft, ships, etc.).

[0101] (Meet the reservation conditions)

[0102] The predetermined condition may be appropriately specified so that an appropriate reaction speed can be evaluated according to an event. In one example, the predetermined condition may be specified so that the faster the reaction speed, the more appropriate it is evaluated to be. In this case, the faster the data set that reacts to the event (evaluated as an appropriate reaction speed) can be used for machine learning with priority. However, the predetermined condition may not be limited to such an example. In another example, the predetermined condition may specify a range (upper limit and lower limit) of appropriate reaction speed. In this case, a data set whose reaction speed falls within the range specified by the predetermined condition can be used preferentially for machine learning, and other data sets (i.e., data sets whose reaction speed is faster or slower than the reaction speed within the range specified by the predetermined condition) are not used preferentially for machine learning.

[0103] (Preferred use)

[0104] The method of giving priority to a data set whose reaction speed is evaluated as more appropriate can be constituted by any method that makes it easier to reflect the data set to be given priority in the training of the control model 5 than the data set not to be given priority.

[0105] As an example, whether to use or not can be simply determined based on whether or not to prioritize. That is, the more appropriate the reaction speed is, the more priority is given to using the dataset whose reaction speed is evaluated as appropriate. This can be configured by using the dataset whose reaction speed is evaluated as appropriate for training the control model 5, and not using the dataset whose reaction speed is not evaluated as appropriate for training the control model 5. According to this method, the evaluation of reaction speed can be reflected in machine learning extremely simply.

[0106] As another example, a data set whose reaction speed is evaluated as appropriate among the multiple data sets 4 is set as the first data set, and a data set whose reaction speed is not evaluated as appropriate (i.e., does not meet the predetermined conditions) is set as the second data set. The data set whose reaction speed is evaluated as more appropriate is used more preferentially, which can be constituted by increasing the sampling probability of the first data set in the multiple data sets 4 in the machine learning, and making the sampling probability of the second data set in the machine learning lower than that of the first data set.

[0107] In this case, the more appropriate the reaction speed of a data set is, the more priority it is used, and it can be further constructed by excluding at least a part of the second data set from the object of machine learning (that is, setting the sampling probability to 0). Alternatively, the more appropriate the reaction speed of a data set is, the more priority it is used, and it can be further constructed by not excluding the second data set from the object of machine learning (that is, not setting the sampling probability to 0). In actual scenarios, due to external factors, operations with slow reaction speeds may be performed as appropriate operations. For example, emergency operations such as emergency braking and emergency steering are assumed to be operations with slow reaction speeds. In addition, since the period from the detection of an event to the need for operation is long, it may also be evaluated as a slow reaction speed on the surface. In this regard, by not excluding the second data set from the object of machine learning, the control model 5 can also grasp these operations. Therefore, it can be expected to improve the robustness of operations against events.

[0108] As another example, the more appropriate the reaction speed is, the more priority is given to the data set, which can be constituted by increasing the training weight of the first data set among the multiple data sets 4 and reducing the training weight of the second data set. For example, increasing the training weight can be constituted by increasing the learning rate, and reducing the training weight can be constituted by reducing the learning rate. In other words, the more appropriate the reaction speed is, the more priority is given to the data set, which can be constituted by making the update amount of the parameters of the control model 5 in one training larger for the data set that is prioritized. In another example, the priority use can include the method of both the above-mentioned sampling probability and weight.

[0109] In addition, the priority of use can be set appropriately. In one example, the priority can be two levels (i.e., two levels of priority / non-priority). In another example, the priority can be set to three or more levels. In this case, the priority can be different between the data sets that are used with priority (i.e., there can be advantages and disadvantages). Similarly, the priority can also be different between the data sets that are not used with priority.

[0110] (Control the action)

[0111] In one example, the control of the movement of the moving body M as the object may be constituted by directly controlling the moving body M as the object. In another example, the moving body (moving body M) may be provided with a dedicated control device such as a controller. In this case, the control of the movement of the moving body M as the object by the control device 2 may be constituted by indirectly controlling the moving body M as the object by giving the derived result to the dedicated control device.

[0112] (System Configuration)

[0113] In one example, Figure 1As shown, the model generation device 1 and the control device 2 can be configured to communicate (connect) with each other via a network. The type of network is not particularly limited, and can be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, etc. However, the method of exchanging data between the model generation device 1 and the control device 2 is not limited to such an example, and can be appropriately selected according to the implementation method. In another example, data can be exchanged using a storage medium.

[0114] In addition, Figure 1 In the example, the model generation device 1 and the control device 2 are independent computers. However, the system configuration is not limited to such an example. In another example, the model generation device 1 and the control device 2 may be composed of an integrated computer. In addition, at least one of the model generation device 1 and the control device 2 may be composed of multiple computers.

[0115] In addition, Figure 1 In the example of FIG. 1 , the control device 2 is mounted inside the mobile body M. However, the configuration of the control device 2 is not limited to such an example. The control device 2 may also be configured outside the mobile body M as long as it can directly or indirectly control the action of the mobile body M.

[0116] [2 Structure example]

[0117] [Hardware configuration example]

[0118] <Model generation device>

[0119] Figure 2 An example of the hardware structure of the model generation device 1 according to the present embodiment is schematically shown. Figure 2 As shown, the model generation device 1 according to the present embodiment is a computer in which a control unit 11, a storage unit 12, a communication interface 13, an input device 14, an output device 15, and a driver 16 are electrically connected.

[0120] The control unit 11 includes a CPU (Central Processing Unit) as a hardware processor, a RAM (Random Access Memory), a ROM (Read Only Memory), etc., and is configured to execute information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource.

[0121] The storage unit 12 may be composed of, for example, a hard disk drive, a solid state drive, etc. The storage unit 12 (and RAM, ROM) is an example of a memory resource. In this embodiment, the storage unit 12 stores various information such as the model generation program 81, a plurality of data sets 4, and the learning result data 125.

[0122] The model generation program 81 is used to cause the model generation device 1 to execute information processing related to machine learning of the control model 5 (described later). Figure 6 The model generation program 81 includes a series of instructions for the information processing. The learning result data 125 is configured to represent information related to the generated trained control model 5. In the present embodiment, the learning result data 125 is generated as a result of executing the model generation program 81.

[0123] The communication interface 13 is an interface for performing wired or wireless communication via a network. The communication interface 13 may be composed of, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc. The model generation device 1 may perform data communication with other computers (for example, the control device 2) via the communication interface 13.

[0124] The input device 14 is a device for performing input, such as a mouse and a keyboard. The output device 15 is a device for performing output, such as a display and a speaker. The operator can operate the model generation device 1 by using the input device 14 and the output device 15. The input device 14 and the output device 15 may be integrally formed by, for example, a touch panel display or the like.

[0125] The driver 16 is a device for reading various information such as programs stored in the storage medium 91. Instead of the storage unit 12 or together with the storage unit 12, at least one of the above-mentioned model generation program 81, multiple data sets 4, and learning result data 125 may be stored in the storage medium 91. The storage medium 91 is configured to accumulate various information (stored programs, etc.) by electrical, magnetic, optical, mechanical or chemical action in a manner that allows a machine such as a computer to read the information. The model generation device 1 can obtain at least one of the model generation program 81 and multiple data sets 4 from the storage medium 91.

[0126] Here, in Figure 2 In the embodiment, as an example of the storage medium 91, a disk-type storage medium such as a CD and a DVD is illustrated. However, the type of the storage medium 91 is not limited to the disk type. As a storage medium other than the disk type, for example, a semiconductor memory such as a flash memory can be cited. The type of the drive 16 can be appropriately selected according to the type of the storage medium 91.

[0127] In addition, regarding the specific hardware structure of the model generation device 1, the constituent elements can be appropriately omitted, replaced, and added according to the implementation method. For example, the control unit 11 may also include multiple hardware processors. The hardware processor may be composed of a microprocessor, an FPGA (field-programmable gate array), a DSP (digital signal processor), an ECU (Electronic Control Unit), a GPU (Graphics Processing Unit), etc. At least one of the communication interface 13, the input device 14, the output device 15, and the driver 16 may also be omitted. The model generation device 1 may be composed of multiple computers. In this case, the hardware structure of each computer may be consistent or inconsistent. In addition to being designed as a computer specifically for the service provided, the model generation device 1 may also be a general-purpose server device, a general-purpose PC (Personal Computer), an industrial PC, a terminal device (for example, a tablet PC, etc.), etc.

[0128] <Control device>

[0129] Figure 3 An example of the hardware configuration of the control device 2 according to the present embodiment is schematically shown. Figure 3 As shown, the control device 2 according to the present embodiment is a computer in which a control unit 21, a storage unit 22, a communication interface 23, an input device 24, an output device 25, a driver 26, and an external interface 27 are electrically connected.

[0130] The control unit 21 to the driver 26 and the storage medium 92 of the control device 2 can be respectively configured similarly to the control unit 11 to the driver 16 and the storage medium 91 of the above-mentioned model generation device 1. The control unit 21 (CPU) is an example of a processor resource of the control device 2, and the storage unit 22 (and RAM, ROM) is an example of a memory resource of the control device 2. In the present embodiment, the storage unit 22 stores various information such as the control program 82 and the learning result data 125.

[0131] The control program 82 is for causing the control device 2 to execute information processing (described later) related to automatic control of the moving object M as a target based on the trained control model 5. Fig.12) program. The control program 82 includes a series of commands for the information processing. At least one of the control program 82 and the learning result data 125 may be stored in the storage medium 92 instead of the storage unit 22 or together with the storage unit 22. The control device 2 may obtain at least one of the control program 82 and the learning result data 125 from the storage medium 92.

[0132] The control device 2 can communicate data with other computers (for example, the model generation device 1) via the communication interface 23. The operator can operate the control device 2 by using the input device 24 and the output device 25. The input device 24 and the output device 25 may be integrally formed by, for example, a touch panel display.

[0133] The external interface 27 is an interface for connecting to an external device. The external interface 27 may be, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like. The type and number of the external interface 27 may be appropriately determined according to the type and number of the external devices connected. In the present embodiment, the control device 2 may be connected to the sensor S via the external interface 27. At least a portion of the object data 221 may be composed of sensor data obtained by the sensor S. In addition, the connection method of the sensor S may not be limited to such an example. In another example, the sensor S may be connected via the communication interface 23.

[0134] In addition, regarding the specific hardware structure of the control device 2, the components can be appropriately omitted, replaced, and added according to the implementation method. For example, the control unit 21 may also include multiple hardware processors. The hardware processor may be composed of a microprocessor, an FPGA, a DSP, an ECU, a GPU, etc. At least one of the communication interface 23, the input device 24, the output device 25, the driver 26, and the external interface 27 may also be omitted. The control device 2 may be composed of multiple computers. In this case, the hardware structure of each computer may be consistent or inconsistent. In addition to being designed as a computer specifically for the services provided, the control device 2 may also be a general-purpose computer, a portable phone including a smart phone, a tablet PC (Personal Computer), etc. In the case where the mobile body M is a vehicle, the control device 2 may be a vehicle-mounted device.

[0135] [Software structure example]

[0136] <Model generation device>

[0137] Figure 4An example of the software structure of the model generation device 1 according to the present embodiment is schematically shown. The control unit 11 of the model generation device 1 expands the model generation program 81 stored in the storage unit 12 to the RAM, and executes the commands included in the model generation program 81 through the CPU. Thus, the model generation device 1 operates as a computer having a learning data acquisition unit 111, a learning processing unit 112, and a storage processing unit 113 as software modules. That is, in the present embodiment, each software module of the model generation device 1 is implemented by the control unit 11 (CPU).

[0138] The learning data acquisition unit 111 is configured to acquire a plurality of data sets 4 each consisting of a combination of training data 41 and correct answer data 45. The learning processing unit 112 is configured to implement machine learning of the control model 5 using the acquired plurality of data sets 4. In the present embodiment, implementing machine learning includes: for each data set 4, the control model 5 is trained in such a manner that the result of deriving the control instruction of the moving body based on the training data 41 using the control model 5 conforms to the corresponding correct answer data 45. In addition, in the machine learning, using the plurality of data sets 4 includes: the more the reaction speed of the control instruction represented by the correct answer data 45 to the event conforms to a predetermined condition, and thus the reaction speed is evaluated as appropriate, the more preferentially the data set is used. By executing the machine learning, a trained control model 5 is generated.

[0139] The storage processing unit 113 is configured to store the trained control model 5 generated by machine learning. In one example, the storage processing unit 113 can be configured to generate learning result data 125, which represents the trained control model 5 generated as a result of machine learning. As long as the information used to perform the operation processing of the trained control model 5 can be maintained, the structure of the learning result data 125 is not particularly limited and can be appropriately determined according to the implementation method. As an example, the learning result data 125 can be configured to include information representing the value of the operation parameter adjusted by machine learning. Depending on the situation, the learning result data 125 can also be configured to include information representing the structure of the trained control model 5 (for example, the structure of the neural network, etc.). The storage processing unit 113 can be configured to save the generated learning result data 125 in a predetermined storage area. The learning result data 125 can be provided to the control device 2 at any timing.

[0140] <Control device>

[0141] Figure 5 An example of the software structure of the control device 2 according to the present embodiment is schematically shown. The control unit 21 of the control device 2 expands the control program 82 stored in the storage unit 22 to the RAM, and executes the commands included in the control program 82 through the CPU. Figure 5As shown, the control device 2 according to the present embodiment operates as a computer having an acquisition unit 211, a derivation unit 212, and an action control unit 213 as software modules. That is, in the present embodiment, similarly to the model generation device 1, each software module of the control device 2 is also implemented by the control unit 21 (CPU).

[0142] The acquisition unit 211 is configured to acquire object data 221 representing an environment in which the target moving body M moves. The derivation unit 212 has the trained control model 5 generated by the above-mentioned model generation device 1 by holding the learning result data 125. The derivation unit 212 is configured to derive a control instruction based on the acquired object data 221 using the trained control model 5. The motion control unit 213 is configured to control the motion of the target moving body M according to the result of deriving the control instruction (that is, the control instruction derived by the trained control model 5).

[0143] <Others>

[0144] In this embodiment, an example is described in which each software module of the model generation device 1 and the control device 2 is implemented by a general-purpose CPU. However, part or all of the above software modules may also be implemented by one or more dedicated processors. The above modules may also be implemented as hardware modules. Regarding the software structure of the model generation device 1 and the control device 2, modules may be omitted, replaced, or added appropriately according to the embodiment.

[0145] [3 Work Examples]

[0146] [Model generation device]

[0147] Figure 6 1 is a flowchart showing an example of processing steps related to machine learning of the control model 5 of the model generation device 1 involved in this embodiment. The following processing steps are an example of a model generation method executed by a computer. However, the following processing steps of the model generation device 1 are just an example, and each step can be changed as much as possible. In addition, for the following processing steps, steps can be appropriately omitted, replaced, and added according to the embodiment.

[0148] <Step S101>

[0149] In step S101, the control unit 11 functions as the learning data acquisition unit 111. That is, the control unit 11 acquires a plurality of data sets 4 each consisting of a combination of the training data 41 and the correct answer data 45.

[0150] The training data 41 is configured to represent the environment in which the mobile body moves in a time series. In one example, the training data 41 may include sensor data SD obtained by the sensor S. In addition, the training data 41 may include any information that may be related to control, such as a set speed, a speed limit, map information, and navigation information. The correct answer data 45 is configured to directly or indirectly represent the control instructions for the mobile body in the environment represented by the corresponding training data 41 in a time series.

[0151] As described above, each data set 4 can be appropriately generated (collected). Each generated data set 4 can be stored in the model generation device 1 (at least one of the storage unit 12 and the storage medium 91). Alternatively, each data set 4 can also be stored in other computers such as a network server (for example, NAS: Network Attached Storage). In this case, when implementing machine learning, the control unit 21 can obtain each data set 4 via a network, an external storage device, the storage medium 91, etc. Each data set 4 can be stored in the form of a database.

[0152] The generation of at least a portion of the plurality of data sets 4 may be performed by the model generation device 1. The generation of at least a portion of the plurality of data sets 4 may also be performed by another computer other than the model generation device 1. When the data set 4 is generated by another computer, the control unit 11 may obtain the data set 4 generated by the other computer, for example, via a network, an external storage device, a storage medium 91, or the like.

[0153] The number of acquired data sets 4 can be appropriately determined according to the embodiment. When a plurality of data sets 4 are acquired, the control unit 11 advances the process to the next step S102.

[0154] <Step S102>

[0155] In step S102, the control unit 11 operates as a learning processing unit 112. That is, the control unit 11 uses the acquired plurality of data sets 4 to perform machine learning of the control model 5. In the machine learning, the control unit 11 trains the control model 5 for each data set 4 in such a manner that the result of deriving the control command of the moving body using the control model 5 based on the training data 41 conforms to the corresponding correct answer data 45.

[0156] The control model 5 (machine learning model) includes one or more operation parameters for performing operation processing to solve the reasoning task. Training the control model 5 means optimizing (adjusting) the values ​​of the operation parameters of the control model 5 (machine learning model) according to the given learning data (multiple data sets 4). The machine learning method can be appropriately determined according to the type, structure and other implementation methods of the machine learning model used for the control model 5. The method for adjusting the operation parameters can adopt any method such as the error back propagation method and solving the optimization problem.

[0157] (An example of machine learning method)

[0158] As a typical example, when the control model 5 is composed of a neural network, the control unit 11 first prepares the control model 5 as the processing object of the machine learning. The structure of the prepared control model 5, the initial value of the connection weight between each neuron, and the initial value of the threshold of each neuron can be given by a template or by the input of an operator. In the case of re-learning, the control unit 11 can prepare the control model 5 based on the learning result data obtained by the past machine learning. Next, the control unit 11 uses the training data 41 of each data set 4 as input data, and uses the correct answer data 45 as a teacher signal (label) to perform the learning process of the control model 5 (learning with a teacher).

[0159] like Figure 4 As shown, as an example of learning processing, in the first step, the control unit 11 inputs the training data 41 of each data set 4 and performs the forward propagation operation processing of the control model 5. As a result of the operation processing, the control unit 11 obtains an output value corresponding to the inference result (direct or indirect derivation result of the control instruction) for the training data 41 from the control model 5. In the second step, the control unit 11 calculates the error between the obtained output value and the corresponding correct data 45. In the third step, the control unit 11 calculates the gradient of the calculated error. Moreover, the control unit 11 back-propagates the calculated gradient of the error by using the error back-propagation method, thereby calculating the error of the value of the operation parameter (connection weights between nodes, thresholds of each node, etc.) of the control model 5. In the fourth step, the control unit 11 updates the value of the operation parameter based on the calculated error. The degree of updating the operation parameter value can be adjusted by the learning rate.

[0160] The control unit 11 adjusts the value of the operation parameter of the control model 5 for each data set 4 in such a way that the sum of the errors between the output value output from the control model 5 and the correct answer data 45 becomes smaller by repeatedly performing the above-mentioned steps 1 to 4. The adjustment of the value of the operation parameter can be repeated, for example, until the predetermined conditions such as the adjustment of the set number of repetitions and the sum of the calculated errors being below the threshold are satisfied. The threshold value can be appropriately set according to the implementation method. In addition, the machine learning conditions such as the objective function (cost function, loss function, error function) for calculating the error, the learning rate, and the optimization algorithm can be appropriately set according to the implementation method.

[0161] The adjustment of the values ​​of the operation parameters of the control model 5 can be performed on a mini-batch. As an example, before executing the processing of the above-mentioned steps 1 to 4, the control unit 11 can generate a mini-batch by extracting any sample (data set) from multiple data sets 4. The size of the mini-batch can be appropriately set according to the implementation method. Moreover, the control unit 11 can execute the processing of the above-mentioned steps 1 to 4 for the data set 4 contained in the generated mini-batch. In the case of repeatedly performing steps 1 to 4, the control unit 11 can generate a mini-batch again and execute the processing of steps 1 to 4 for the generated new mini-batch.

[0162] In addition, the machine learning method may not be limited to such an example of learning with a teacher, and other methods may be at least partially adopted. As another example, deep reinforcement learning may be adopted. In this case, a portion of the multiple data sets 4 may be obtained as a result of an episode in reinforcement learning. Alternatively, the control unit 11 may also generate multiple new data sets based on the results of the episode in reinforcement learning, and use the generated multiple new data sets and multiple data sets 4 to adjust (optimize) the operation parameters of the control model 5. Deep reinforcement learning may adopt any method such as R2D3 (Recurrent ReplayDistributed DQN from Demonstrations).

[0163] (Specific example of incident)

[0164] In the present embodiment, in the above-mentioned machine learning, the control unit 11 preferentially uses a data set in which the reaction speed of the control command represented by the correct answer data 45 to the event meets a predetermined condition and the reaction speed is evaluated to be appropriate.

[0165] As described above, the event may include all events / phenomena that may be associated with the operation of the moving body. In addition, the event may include all events / phenomena that can be detected by the sensor S. As an example, the moving body (moving body M) may be a vehicle, and the event may include at least one of the deceleration of the preceding vehicle relative to the vehicle, the cutting in of a parallel vehicle, the appearance of a parked / stopped vehicle, the generation of an obstacle, and the change of a signal.

[0166] The predetermined condition may be appropriately defined as evaluating an appropriate reaction speed according to an event. The start time of the event may be determined based on sensor data obtained by the sensor S. The reaction speed may be defined by the time from the start time of the event to the start time of the action control (operation) for the event. A specific example is shown below.

[0167] (A) Slowdown of the vehicle ahead

[0168] Fig. 7A An example of an event (deceleration of the preceding vehicle) is schematically shown. Fig. 7A In the example of , vehicle MA is the vehicle that encounters the event. That is, in the learning phase, it can be assumed that data set 4 is obtained from vehicle MA. On the other hand, vehicle MB is the preceding vehicle of vehicle MA.

[0169] In the case where the preceding vehicle MB decelerates, any operation to cope with the deceleration of the preceding vehicle MB can be performed in the vehicle MA. An example of a control operation that can be taken in the vehicle MA is a deceleration operation corresponding to the preceding vehicle MB. Therefore, in one example, the reaction speed can be defined by the time from the moment when the preceding vehicle MB starts to decelerate (the moment when the deceleration is detected) to the moment when the deceleration operation starts in the vehicle MA.

[0170] The deceleration of the preceding vehicle MB can be detected by a sensor (sensor S) related to speed, position or vehicle-to-vehicle distance, such as a camera, radar, or LiDAR. An index for detecting the deceleration of the preceding vehicle MB can be appropriately specified. In one example, collision risk indexes such as time-to-collision (TTC) and margin-to-collision (MTC) can be used as the index for detecting the deceleration of the preceding vehicle MB.

[0171] On the other hand, the deceleration operation in the vehicle MA can be detected based on at least one of the speed, acceleration and braking amount of the vehicle MA. As an example, the timing of the deceleration operation performed in the vehicle MA can be detected by evaluating a threshold value for at least one of the speed, acceleration and braking amount of the vehicle MA.

[0172] Therefore, in one example of the present embodiment, the reaction speed of the deceleration operation to the deceleration of the preceding vehicle MB can be defined by the time from the moment of detecting the deceleration of the preceding vehicle MB (the start time of the event) to the moment of detecting the deceleration operation of the vehicle MA (the start time of the operation). The shorter the time, the faster the reaction speed is evaluated, and the longer the time, the slower the reaction speed is evaluated.

[0173] Figure 7B An example of a method for evaluating the reaction speed of a response operation to a deceleration of a preceding vehicle is schematically shown. Figure 7B In the example, a collision risk index is used as an index for detecting the deceleration of the front vehicle MB, and the acceleration (negative acceleration) of the vehicle MA is used as an index for detecting the deceleration operation. The threshold TA is set for the collision risk index in order to detect the start time of the event (deceleration of the front vehicle MB), and the threshold TB is set for the acceleration of the vehicle MA in order to detect the start time of the deceleration operation. The threshold TB can be set, for example, by taking into account external factors such as the influence of the road surface. Each threshold (TA, TB) can be set manually by the operator, or can be at least partially automatically set based on statistics, etc. Figure 7B In the example, the reaction speed can be estimated based on the time from the start of the event detected based on the threshold TA (based on the collision risk indicator) to the start of the deceleration operation detected based on the threshold TB (based on the acceleration of the vehicle MA).

[0174] In addition, the operation (action control) of the vehicle MA with respect to the preceding vehicle MB may not be limited to deceleration as in the above-mentioned example, and may be appropriately selected according to the implementation method. In another example, in the case of deceleration of the preceding vehicle MB, the vehicle MA may perform a lane change (lane change). In this case, the start time of the lane change operation may be detected based on at least any one of the speed, acceleration, braking amount, acceleration amount, and steering amount in the vehicle MA, and the reaction speed may be estimated based on this. The steering amount may be measured, for example, based on the steering torque. In another example, the start time of the lane change operation may also be detected based on vehicle operations such as the operation of the turn signal lamp. In addition, the signal of the preceding vehicle MB, such as whether the brake lights of the preceding vehicle MB are on, may also be used as an indicator for detecting the deceleration of the preceding vehicle MB.

[0175] (B) Cutting in of parallel vehicles

[0176] Fig. 8A An example of an event (a parallel vehicle cutting in) is schematically shown. Fig. 8A In the example, vehicle MC is the vehicle that encounters the event. That is, in the learning phase, it can be assumed that data set 4 is obtained from vehicle MC. On the other hand, vehicle MD is a parallel vehicle of vehicle MC. Fig. 8A In the example of FIG. 1 , it is assumed that the parallel vehicle MD inserts (cuts in line) in front of the vehicle MC.

[0177] In the case where the parallel vehicle MD cuts in, any operation to cope with the cut-in of the parallel vehicle MD can be performed in the vehicle MC. An example of a control operation that can be taken in the vehicle MC is a deceleration operation corresponding to the cut-in of the parallel vehicle MD. Therefore, in one example, the reaction speed can be defined by the time from the moment when the parallel vehicle MD starts to cut in (the moment when the cut-in is detected) to the moment when the deceleration operation starts in the vehicle MC.

[0178] The cutting-in of the parallel vehicle MD can be detected, for example, by sensors (sensors S) related to speed, position or vehicle distance, such as cameras, radars, and LiDARs. The index for detecting the cutting-in of the parallel vehicle MD can be appropriately specified. In one example, distance indexes such as the overlap amount and the distance between the parallel vehicle MD and the white line can be used in the index for detecting the cutting-in of the parallel vehicle MD. For example, the cutting-in of the parallel vehicle MD can be detected based on the timing when the overlap amount becomes less than a certain value, the timing when the parallel vehicle MD reaches the white line, etc. In addition, the overlap amount is the distance in the vehicle width direction (the left and right direction relative to the vehicle's traveling direction, the up and down direction in the figure) between the predicted forward path MCA of the other vehicle (parallel vehicle MD) and the vehicle (vehicle MC). The predicted forward path MCA can be, for example, the predicted forward path range of the vehicle MC shown by the dotted line in the figure. On the other hand, similarly to the above (A), the deceleration operation in the vehicle MC can be detected based on at least one of the speed, acceleration, and braking amount of the vehicle MC. Therefore, in one example of this embodiment, the reaction speed of the deceleration operation to the cutting-in of the parallel vehicle MD can be defined by the time from the moment the cutting-in of the parallel vehicle MD is detected (the start time of the event) to the moment the deceleration operation of the vehicle MC is detected (the start time of the operation).

[0179] Figure 8B An example of a method for evaluating the reaction speed of a response operation to a cut-in of a parallel vehicle is schematically shown. Figure 8B In the example, the amount of overlap is used as an indicator for detecting the cut-in of the parallel vehicle MD, and the acceleration (negative acceleration) of the vehicle MC is used as an indicator for detecting the deceleration operation. The threshold TC is set for the amount of overlap in order to detect the start time of the event (the cut-in of the parallel vehicle MD), and the threshold TD is set for the acceleration of the vehicle MC in order to detect the start time of the deceleration operation. The threshold TD can be set, for example, by taking into account external factors such as the influence of the road surface. Each threshold (TC, TD) can be set manually by the operator, or can be at least partially set automatically based on statistics, etc. Figure 8BIn the example of , the reaction speed can be estimated based on the time from the event start time detected based on the threshold value TC (based on the overlap amount) to the start time of the deceleration operation detected based on the threshold value TD (based on the acceleration of the vehicle MC).

[0180] In addition, the operation (action control) of the vehicle MC with respect to the parallel vehicle MD may not be limited to deceleration as in the above-mentioned example, and may be appropriately selected according to the implementation method. In another example, in the event of a parallel vehicle MD cutting in, the vehicle MC may change lanes. In this case, the start time of the lane change operation may be detected based on at least any one of the speed, acceleration, braking amount, acceleration amount, and steering amount in the vehicle MC, and the reaction speed may be estimated based on this. In another example, the start time of the lane change operation may also be detected based on vehicle operations such as the operation of the turn signal lamp. In addition, the signals of the parallel vehicle MD, such as whether the brake lights of the parallel vehicle MD are on, whether the turn signal lights are on, etc., may be used as indicators for detecting the cutting in of the parallel vehicle MD.

[0181] In addition, the form of the parallel vehicle MD's cut-in may differ depending on the scene. In one example, when cutting into a place where the parallel vehicle MD cannot drive, such as a merging road or a construction section, the vehicle MC may be required to respond at an earlier timing than other cut-ins (e.g., normal lane changes) in order to ensure that the parallel vehicle MD can be inserted. Therefore, the cut-in event can also be classified according to the scene.

[0182] (C) The presence of parked vehicles

[0183] Fig. 9 An example of an event (the presence of a parked vehicle) is schematically shown. Fig. 9 In the example, vehicle ME is the vehicle that encounters the event. That is, in the learning phase, it is assumed that the data set 4 is obtained from vehicle ME. On the other hand, vehicle MF is a parked vehicle. Fig. 9 In the example of FIG. 5 , it is assumed that vehicle MF is parked in front of vehicle ME.

[0184] In the case where a parked vehicle MF appears, any operation to deal with the parked vehicle MF can be performed in the vehicle ME. In one example, the front vehicle MB can be replaced with the parked vehicle MF for the appearance of the parked vehicle MF, and the same treatment as the deceleration of the front vehicle MB in (A) above can be performed. That is, in one example, an example of the control operation that can be taken in the vehicle ME is a deceleration or avoidance (lane change, etc.) operation corresponding to the appearance of the parked vehicle MF. The reaction speed can be defined by the time from the moment when the parked vehicle MF appears (the moment when the parked vehicle MF is detected) to the moment when the deceleration or avoidance operation starts in the vehicle ME.

[0185] The presence of the parked vehicle MF can be detected by, for example, a sensor (sensor S) related to speed, position, or inter-vehicle distance, such as a camera, radar, or LiDAR. In one example, the above-mentioned collision risk index can be used as an index for detecting the parked vehicle MF. In addition to the above-mentioned collision margin time and collision margin, the distance to the parked vehicle MF, inter-vehicle distance / travel speed (THW: Time Head Way) and other indexes can be used in the collision risk index for detecting the parked vehicle MF. Therefore, in one example of the present embodiment, the reaction speed to the presence of the parked vehicle MF can be estimated based on the time from the event start time to the operation start time, the event start time can be detected by the threshold determination of the value of the collision risk index, and the operation start time can be detected by the threshold determination of the value of the motion amount (at least one of the speed, acceleration, braking amount, acceleration amount, and steering amount) of the vehicle ME. In another example, the start time of the avoidance operation can also be detected based on the vehicle operation such as the turn signal operation. Furthermore, a signal of the parked vehicle MF, such as whether or not the hazard warning lights of the parked vehicle MF are on, can be used as an index for detecting the presence of the parked vehicle MF.

[0186] (D) Generation of obstacles

[0187] The generation of an obstacle is the same as the appearance of the above-mentioned parked / parked vehicle MF. By replacing the parked / parked vehicle MF with an obstacle in the above-mentioned example, the reaction speed of the operation to the generation of the obstacle can be evaluated in the same way as the reaction speed of the operation to the appearance of the above-mentioned parked / parked vehicle MF. In one example of the present embodiment, the reaction speed to the generation of the obstacle can be estimated based on the time from the start time of the event to the start time of the operation, and the event start time is detected by the threshold determination of the value of the collision risk index, and the operation start time is detected by the threshold determination of the value of the vehicle's motion amount. Regarding the collision risk index, the above-mentioned parked / parked vehicle MF can be replaced by an obstacle for measurement. In another example, the start time of the avoidance operation can also be detected based on the vehicle operation such as the turn signal operation. In addition, as described above, the obstacle can be, for example, a pedestrian, a bicycle, etc.

[0188] (E) Changes in signal lights

[0189] Fig. 10A This is a schematic representation of an example of an event (a change in a traffic signal). Fig. 10A In the example of , the vehicle MG is the vehicle that encounters the event. That is, in the learning phase, it is assumed that the data set 4 is obtained from the vehicle MG.

[0190] In the case of encountering a change in the signal MT, in the vehicle MG, any operation that responds to the change in the signal MT can be performed. As an example, in the case where the signal MT changes from a going signal to a deceleration signal or a caution signal (from a green signal to a yellow signal), an example of a control operation that the vehicle MG can take is a deceleration operation for stopping in front of the signal MT or an acceleration operation for passing a road where the signal MT is set. Therefore, in one example, the reaction speed can be defined by the time from the moment when the signal MT starts to change (the moment when the change is detected) to the moment when the deceleration or acceleration operation starts in the vehicle MG.

[0191] The change of the signal MT can be detected by a sensor such as a camera (sensor S), for example. Among the indicators for detecting the change of the signal MT, the result of identifying the color of the lighted signal in the signal MT can be used. The color of the lighted signal in the signal MT can be identified by any method. On the other hand, the deceleration or acceleration operation in the vehicle MG can be detected based on at least one of the speed, acceleration, acceleration amount and braking amount of the vehicle MG. Therefore, in one example of the present embodiment, the reaction speed of the deceleration or acceleration operation to the change of the signal MT can be defined by the time from the moment when the change of the signal MT is detected (the start time of the event) to the moment when the deceleration or acceleration operation of the vehicle MG is detected (the start time of the operation). Thresholds can be set separately for the deceleration operation and the acceleration operation.

[0192] Fig. 10B An example of a method for evaluating the reaction speed of a response operation to a change in a traffic light is schematically shown. Fig. 10B In the example, it is assumed that the color change of the signal MT occurs in an instant. Fig. 10B In the example, the recognition result of the color of the lit signal in the traffic light MT is used as an indicator for detecting changes in the traffic light MT, and the acceleration of the vehicle MG is used as an indicator for detecting deceleration or acceleration operations. The threshold TE is set for the acceleration of the vehicle MG in order to detect the start time of the deceleration operation, and the threshold TF is set for the acceleration of the vehicle MG in order to detect the start time of the acceleration operation. Each threshold (TE, TF) can be set by taking into account external factors such as the influence of the road surface. Each threshold (TE, TF) can be set manually by the operator, or can be at least partially automatically set based on statistics, etc. Fig. 10BIn the example of , the reaction speed of the deceleration operation can be estimated based on the time from the time when the signal MT changes (the event start time) to the start time of the deceleration operation detected by the threshold value TE (based on the acceleration of the vehicle MG). In addition, the reaction speed of the acceleration operation can be estimated based on the time from the time when the signal MT changes (the event start time) to the start time of the acceleration operation detected by the threshold value TF (based on the acceleration of the vehicle MG).

[0193] In addition, the event of the change of the signal MT is not limited to the above example (switching from a green signal to a yellow signal), and can be appropriately set according to the implementation. In another example, the timing of switching to a red signal can be detected instead of the timing of switching to a yellow signal as the start time of the change event of the signal MT.

[0194] In addition, in the above example, the timing of the completion of the change of the signal MT (for example, the timing of the transition from the green signal on state to the green signal off state and the yellow signal on state) is defined as the change time of the signal MT (the start time of the event). However, the definition of the start time of the event is not limited to such an example, and can be appropriately set according to the implementation method. In another example, the signal MT may include a signal for pedestrians in addition to a signal for vehicles ( Fig. 10A In this case, any timing from the start of flashing of the pedestrian signal to the completion of the change of the vehicle signal can be defined as the change time of the signal MT (start time of the event).

[0195] (Specific example of priority use)

[0196] The control unit 11 preferentially uses the data sets 4 whose reaction speed is evaluated as appropriate for machine learning. That is, the control unit 11 reflects the data sets whose reaction speed is evaluated as appropriate in the training of the control model 5 more than the data sets whose reaction speed is not evaluated as appropriate. In this embodiment, the preferential use can be composed of at least any one of the following three methods.

[0197] (1) Method 1

[0198] Fig.11A An example of the first method of using a data set with a reaction speed that is evaluated as appropriate is schematically shown. As the first method, the control unit 11 can use a data set with a reaction speed that is evaluated as appropriate for training the control model 5, and not use a data set with a reaction speed that is not evaluated as appropriate for training the control model 5.

[0199] Fig.11AThe graph of FIG. 1 shows the number of events (samples) relative to the reaction speed in a histogram. The number of events can be the number of data sets, or can be counted at least partially differently from the collection of data sets. Whether the reaction speed is appropriate can be evaluated based on any statistic. In one example, the statistic can be a mean, a mode, an nth percentile value, etc. n can be an arbitrary value. The nth percentile value can be, for example, a 50th percentile value (median value), a 25th percentile value, etc.

[0200] In one example, whether the reaction speed is appropriate (whether it meets the predetermined conditions) can be distinguished based on the statistic (equivalent to Fig.11A In the example of setting the value to 0). In the case where the faster the reaction speed, the more appropriate the evaluation is, the data set whose reaction speed is faster than the statistic that serves as the benchmark can be used as a data set for machine learning as an appropriate reaction speed. That is, the control unit 11 can use the statistic that serves as the benchmark as the upper limit value of the reaction speed. In the case of specifying a range of appropriate reaction speeds, the upper limit value and the lower limit value can be set according to the corresponding statistics, respectively. As a specific example, in the case where it is assumed that the statistics are started from the fast reaction speed, the lower limit value of the reaction speed can be specified by the 25th percentile value, and the upper limit value of the reaction speed can be specified by the 75th percentile value.

[0201] In another example, the statistic may not be used directly for the benchmark, but the control unit 11 may derive the benchmark value by performing an arbitrary operation on the statistic. In simple terms, the control unit 11 may derive the benchmark value by adding or subtracting a set value from the statistic. Moreover, whether the reaction speed is appropriate may be distinguished based on the derived benchmark value rather than the above-mentioned statistic. That is, in a case where the faster the reaction speed, the more appropriate it is evaluated to be, a data set with a reaction speed faster than the benchmark value may be used for machine learning as a data set with appropriate reaction speed. In the case of specifying a range of appropriate reaction speeds, the upper limit value and the lower limit value may be derived from the same statistic, respectively. Alternatively, the upper limit value and the lower limit value may be derived from different statistics, respectively. In each case, the size of the set value used to derive the upper limit value may be the same as the size of the set value used to derive the lower limit value, or may be different.

[0202] exist Fig.11A In the example, the mode is used as the evaluation index. The reference value is calculated by subtracting the set value from the reaction speed of the mode (the time from the event start time to the operation start time). Then, the data set with a reaction speed faster than the reference value is evaluated as a data set with an appropriate reaction speed ( Fig.11A In the first method, the control unit 11 may exclude the data sets whose reaction speed is not evaluated as appropriate, and perform the above-mentioned machine learning processing using only the data sets whose reaction speed is evaluated as appropriate.

[0203] (2) Method 2

[0204] Fig. 11B An example of the second method of giving priority to a data set whose reaction speed is evaluated as appropriate is schematically shown. As the second method, the control unit 11 can increase the sampling probability of the first data set whose reaction speed is evaluated as appropriate among the plurality of data sets 4, and set the sampling probability of the second data set whose reaction speed is not evaluated as appropriate to a lower level. That is, the control unit 11 can increase the number of sampling times in machine learning for a data set whose reaction speed is evaluated as appropriate.

[0205] In one example, the sampling probability (number of times) corresponds to the probability (number of times) of extraction from a small batch in the above-mentioned machine learning. Therefore, if the sampling probability is high (the number of sampling times is large), the number of times the operation parameter value for the control model 5 is adjusted increases. As a result, it is possible to reflect a data set whose reaction speed is evaluated as appropriate in the training of the control model 5 more than a data set whose reaction speed is not evaluated as appropriate.

[0206] The method of setting the sampling probability (number of times) according to the reaction speed can be appropriately determined according to the implementation method. In one example, the control unit 11 can use a function appropriately designed so that the sampling probability (number of times) becomes larger as the reaction speed meets the predetermined condition to determine the sampling probability (number of times) of each data set 4.

[0207] In addition, in the second method, the control unit 11 may exclude at least a portion of the second data set whose reaction speed is not evaluated as appropriate from the object of machine learning (that is, set the sampling probability to 0). Alternatively, the control unit 11 may not exclude the second data set whose reaction speed is not evaluated as appropriate from the object of machine learning (that is, not set the sampling probability to 0).

[0208] exist Fig. 11B In the example, the predetermined condition is that the faster the reaction speed, the more appropriate the evaluation. That is, the faster the reaction speed of the data set, the higher the sampling probability is set. In addition, the second data set is not excluded from the object of machine learning. As a specific example, the sampling probability P(i) of the data set with the i-th reaction speed among the multiple data sets 4 can be specified by the following formula 1.

[0209]

[0210] a is a positive real number and can be set arbitrarily. rank(i) represents the rank of the reaction speed. N represents the rank of the slowest reaction speed. The control unit 11 can generate a small batch according to the sampling probability P(i) specified by the above formula 1, and use the generated small batch to perform the above machine learning processing. In addition, the function formula for specifying the sampling probability is not limited to the example of the above formula 1, and can be appropriately designed according to the implementation method.

[0211] In addition, as an optional measure, the training weight of each data set 4 can be adjusted according to the sampling probability. In one example, a data set with a low sampling probability may not be reflected in the adjustment of the operation parameter value of the control model 5. In order to avoid this situation, the training weight of the data set with a lower sampling probability can be increased without invalidating the priority of the sampling probability. As described above, increasing the training weight can be formed by increasing the learning rate. As a specific example, the training weight w of the data set with the i-th reaction speed is i It can be calculated according to the sampling probability P(i) using the following formula 2.

[0212]

[0213] In addition, the function formula for defining the weight of the training of the data set may not be limited to the example of Formula 2, and may be appropriately designed according to the implementation. In another example, the weight of the training of the data set may not be defined based on the sampling probability.

[0214] (3) Method 3

[0215] As a third method, the control unit 11 may perform the above-mentioned machine learning processing after setting a larger training weight for the first data set whose reaction speed is evaluated as appropriate among the multiple data sets 4, and setting a smaller training weight for the second data set whose reaction speed is not evaluated as appropriate.

[0216] In one example, the control unit 11 can set the training weight to be larger by increasing the learning rate in the above-mentioned machine learning, and can set the training weight to be smaller by decreasing the learning rate. If the value of the learning rate becomes larger, the update amount when adjusting the value of the operation parameter of the control model 5 in the fourth step of the above-mentioned machine learning becomes larger. In this way, it is possible to reflect the training of the control model 5 on the data set whose reaction speed is evaluated as appropriate more than the data set whose reaction speed is not evaluated as appropriate.

[0217] The method of setting the training weight according to the reaction speed can be appropriately determined according to the implementation method. In one example, the control unit 11 can use a function appropriately designed so that the more the reaction speed meets the predetermined condition, the larger the training weight becomes to determine the training weight of each data set 4.

[0218] In addition, in the third method, the control unit 11 may exclude at least a portion of the second data set whose reaction speed is not evaluated as appropriate from the object of machine learning (i.e., set the training weight to 0). The data set whose training weight is 0 may not be used for machine learning. Alternatively, the control unit 11 may not exclude the second data set whose reaction speed is not evaluated as appropriate from the object of machine learning (i.e., not set the training weight to 0).

[0219] In addition, in the above-mentioned first method to the third method, the extraction process (calculation process) of the data set to be used preferentially can be performed by any component. In one example, the extraction process can be performed by the control unit 11. That is, the control unit 11 can refer to the reaction speed of each data set 4 and assign the advantages and disadvantages related to the use of machine learning to each data set 4 according to the above-mentioned first method to the third method. In another example, even if the control unit 11 does not perform any processing, the extraction process can be implemented by a mechanism in the storage area that stores the data set 4. As a specific example, when the data set 4 is stored in a database, the extraction process can be implemented as a database process. That is, the control unit 11 can extract each data set 4 from the database in a state where the advantages and disadvantages related to the use of machine learning are assigned according to the reaction speed (for example, the data set is sorted in the order of reaction speed in the database). Not using for the above-mentioned machine learning can be achieved by not extracting from the database. The control unit 11 can implement the above-mentioned machine learning in which the more appropriate the reaction speed of the data set is evaluated, the more preferentially used the data set is. In the third method, the control unit 11 may set the training weight according to the order of extraction (for example, when extracting in order of reaction speed from fast to slow, the control unit 11 may set a heavier training weight for the data set that is extracted earlier).

[0220] In the above-mentioned first method to the third method, it can be appropriately selected whether to include the data set ( Fig.11A and Fig. 11B The data set located to the left of the event start time) is used for machine learning.

[0221] The control unit 11 can use at least any one of the first method to the third method to give priority to the data set 4, which is evaluated as having a more appropriate reaction speed, for machine learning. The first method to the third method can be used in combination. In this embodiment, the control unit 11 can generate a trained control model 5 by executing the machine learning process as described above. When the machine learning of the control model 5 is completed, the control unit 11 causes the process to enter the next step S103.

[0222] In addition, the operation required for each event may be different. Therefore, the control unit 11 may also generate a trained control model 5 for each event. In addition, the multiple data sets 4 obtained may include data sets related to other events / phenomena other than the target event that is ranked according to the reaction speed. In this case, the data sets related to other events / phenomena may also be ranked in any way. Alternatively, the data sets related to other events / phenomena may be used for machine learning without being ranked.

[0223] <Step S103>

[0224] return Figure 6 In step S103, the control unit 11 operates as the storage processing unit 113. That is, the control unit 11 generates information related to the trained control model 5 generated by machine learning as learning result data 125. The learning result data 125 can be appropriately configured to include information for reproducing the trained control model 5. The control unit 11 stores the generated learning result data 125 in a predetermined storage area.

[0225] The predetermined storage area may be, for example, a RAM in the control unit 11, a storage unit 12, an external storage device, a storage medium, or a combination thereof. The storage medium may be, for example, a CD, a DVD, a semiconductor memory, etc., and the control unit 11 may save the learning result data 125 to the storage medium via the driver 16. The external storage device may be, for example, a data server such as NAS. In this case, the control unit 11 may save the learning result data 125 to the data server via the network using the communication interface 13. In addition, the external storage device may be, for example, an external storage device. The external storage device may be appropriately connected to the model generation device 1. For example, the model generation device 1 may also have an external interface, which may be connected to the external storage device via the external interface.

[0226] When the storage of the machine learning results is completed, the control unit 11 ends the processing steps of the model generation device 1 according to this operation example.

[0227] In addition, the generated learning result data 125 can be provided to the control device 2 at any timing and method. For example, as the processing of the above-mentioned step S103 or separately from the processing of step S103, the control unit 11 can transmit the learning result data 125 to the control device 2. The control device 2 can obtain the learning result data 125 by receiving the transmission. In another example, the control device 2 can obtain the learning result data 125 by accessing the model generation device 1 or the data server via the network using the communication interface 23. In another example, the control device 2 can obtain the learning result data 125 via the storage medium 92. In another example, the learning result data 125 can also be pre-programmed into the control device 2.

[0228] In addition, the control unit 11 can update or newly generate the learning result data 125 by repeatedly performing the processing of the above-mentioned steps S101 to S103 regularly or irregularly. During the repetition, at least a part of the data set 4 used for machine learning can be appropriately changed, corrected, added, deleted, etc. In addition, the control unit 11 can update the learning result data 125 maintained by the control device 2 by providing the updated or newly generated learning result data 125 to the control device 2 in an arbitrary manner.

[0229] [Control device]

[0230] Fig.12 1 is a flowchart showing an example of processing steps of the control device 2 involved in the present embodiment related to automatic control of the moving object M as an object using the trained control model 5. The following processing steps are an example of a control method executed by a computer. However, the following processing steps of the control device 2 are only examples, and each step can be changed as much as possible. In addition, for the following processing steps, steps can be appropriately omitted, replaced, and added according to the embodiment.

[0231] <Step S201>

[0232] In step S201, the control unit 21 functions as an acquisition unit 211. That is, the control unit 21 acquires object data 221 representing the environment in which the moving body M as the object moves. In one example, the object data 221 may include sensor data obtained by the sensor S. In addition, the object data 221 may include any information that may be related to control, such as a set speed, a speed limit, map information, and navigation information. The control unit 21 may acquire various information in any manner. When the object data 221 is acquired, the control unit 21 advances the processing to the next step S202.

[0233] <Step S202>

[0234] In step S202 , the control unit 21 functions as the derivation unit 212 . That is, the control unit 21 derives a control command from the acquired object data 221 using the trained control model 5 .

[0235] In addition, the control unit 21 can refer to the learning result data 125 at any timing before executing step S202, and set the trained control model 5 in a usable state (i.e., a state in which the operation processing can be performed). In addition, when the trained control model 5 is generated for each event, the control unit 21 can determine the event that the moving body M as the object encounters or may encounter. The encounter of the event or the possibility thereof can be detected by the sensor S. The control unit 21 can select a trained control model 5 corresponding to the determined event from the multiple trained control models 5 maintained. Moreover, the control unit 21 can use the selected trained control model 5 to derive a control instruction based on the acquired object data 221.

[0236] The calculation process of the trained control model 5 can be appropriately determined according to the type, structure, and configuration of the control model 5. In one example, when the control model 5 is composed of a neural network, the control unit 21 inputs the object data 221 into the trained control model 5 and performs the calculation process of the forward propagation of the trained control model 5. As a result of performing the calculation process, the control unit 21 can obtain an output value corresponding to the result of deriving the control instruction from the trained control model 5. When the output of the control model 5 is configured to indirectly represent the control instruction, the control unit 21 can derive the control instruction by performing a predetermined calculation process on the output of the control model 5. When the derivation of the control instruction is completed, the control unit 21 causes the process to enter the next step S203.

[0237] <Step S203>

[0238] In step S203, the control unit 21 functions as the motion control unit 213. That is, the control unit 21 controls the motion of the target moving object M according to the result of deriving the control command. The control unit 21 may control the motion of the target moving object M directly or indirectly.

[0239] When the control of the moving body M as the object is completed, the control unit 21 ends the processing steps of the control device 2 involved in this work example. In addition, the control unit 21 can repeatedly perform a series of information processing from step S201 to step S203. The timing of repetition can be appropriately determined according to the implementation method. In one example, the control unit 21 can repeatedly perform a series of information processing from step S201 to step S203 during a predetermined period (for example, from the start to the stop of the power source of the moving body M). Thereby, the control device 2 can continuously perform automatic control of the moving body M.

[0240] [feature]

[0241] In the present embodiment, in the process of the above step S102, the more appropriate the reaction speed is evaluated, the more preferentially the data set is used for machine learning, thereby generating the trained control model 5. Since the ability of the trained model generated by machine learning depends on the data set used for the machine learning, according to the present embodiment, it can be expected that the trained control model 5 that has mastered the ability to control the moving body at an appropriate reaction speed can be obtained. In addition, in the process of the above step S202, by using such a trained control model 5, it can be expected that the moving body M as a target can be controlled at an appropriate reaction speed.

[0242] [4 Modifications]

[0243] The embodiments of the present disclosure have been described in detail above, and the above description is merely an illustration of the present disclosure in all aspects. Various improvements or modifications can be made without departing from the scope of the present disclosure. For example, the following changes can be made.

[0244] Fig.13 An example of another scenario in which the present disclosure is applied is schematically shown. The system involved in the modification example further includes a data collection device 3 in addition to the above-mentioned model generation device 1 and control device 2. The data collection device 3 is one or more computers configured to collect data sets 4. In this modification example, the data collection device 3 collects multiple data sets 4 each consisting of a combination of training data 41 and correct answer data 45. Collecting multiple data sets 4 may include: the more the reaction speed of the control instruction represented by the correct answer data 45 to the event is evaluated as appropriate because it meets predetermined conditions, the more priority is given to collecting the data set. Moreover, the data collection device 3 outputs the collected multiple data sets 4 for machine learning.

[0245] exist Fig.13 In the example of , it is assumed that the data collection device 3 is mounted on the mobile body MZ, and the data set 4 is collected according to the operation of the mobile body MZ performed by the subject. In this example, the data collection device 3 can generate training data 41 based on the sensor data obtained from the sensor S and other information (for example, set speed, speed limit, map information, navigation information, etc.). In addition, the data collection device 3 can generate correct answer data 45 based on the results of the subject's operation during the period in which the information constituting the training data 41 is obtained. Moreover, the data collection device 3 can obtain the data set 4 by associating the generated training data 41 with the correct answer data 45. The data collection device 3 can be configured to perform the processing of collecting the data set 4 according to instructions from an external computer such as the model generation device 1. Data exchange between each device 1 to 3 can be performed in any method.

[0246] In addition, the form of the data collection device 3 may not be limited to Fig.13 In another example, the data collection device 3 may not be mounted on the moving body MZ, but may be configured separately from the moving body MZ. In another example, the data collection device 3 may be integrally formed with the control device 2 or the model generation device 1. That is, the control device 2 may also serve as the data collection device 3 (in this case, the moving body MZ is the moving body M). Alternatively, the model generation device 1 may also serve as the data collection device 3.

[0247] [Hardware structure]

[0248] Fig.14 An example of the hardware structure of the data collection device 3 according to this modification is schematically shown. Fig.14As shown, the data collection device 3 according to the present modification is a computer in which a control unit 31 , a storage unit 32 , a communication interface 33 , an input device 34 , an output device 35 , a driver 36 , and an external interface 37 are electrically connected.

[0249] The control unit 31 to the external interface 37 and the storage medium 93 of the data collection device 3 can be respectively configured similarly to the control unit 21 to the external interface 27 and the storage medium 92 of the control device 2 described above. The control unit 31 (CPU) is an example of a processor resource of the data collection device 3, and the storage unit 32 (and RAM, ROM) is an example of a memory resource of the data collection device 3. In this modified example, the storage unit 32 stores various information such as the data collection program 83 and the data set 4.

[0250] The data collection program 83 is used to cause the data collection device 3 to perform information processing related to the collection of the data set 4 (described later). Fig.16 ). The data collection program 83 includes a series of instructions for the information processing. The data set 4 can be accumulated as a result of the execution of the data collection program 83. At least one of the data collection program 83 and the data set 4 can also be stored in the storage medium 93 instead of the storage unit 32 or together with the storage unit 32. The data collection device 3 can obtain the data collection program 83 from the storage medium 93.

[0251] The data collection device 3 can perform data communication with other computers (e.g., the model generation device 1) via the communication interface 33. The operator (e.g., the subject) can operate the data collection device 3 by using the input device 34 and the output device 35. The input device 34 and the output device 35 can also be integrally formed by, for example, a touch panel display. The data collection device 3 can be connected to the sensor S via the external interface 37. However, the connection method of the sensor S is not limited to such an example. In another example, the data collection device 3 can also be connected to the sensor S via the communication interface 23.

[0252] In addition, regarding the specific hardware structure of the data collection device 3, the constituent elements can be appropriately omitted, replaced, and added according to the implementation method. For example, the control unit 31 may also include multiple hardware processors. The hardware processor may be composed of a microprocessor, an FPGA, a DSP, an ECU, a GPU, etc. At least one of the communication interface 33, the input device 34, the output device 35, the driver 36, and the external interface 37 may also be omitted. The data collection device 3 may be composed of multiple computers. In this case, the hardware structure of each computer may be consistent or inconsistent. In addition to being designed as a computer specifically for the services provided, the data collection device 3 may also be a general-purpose server device, a general-purpose computer, a portable phone including a smart phone, a tablet PC (Personal Computer), etc. In the case where the mobile body MZ is a vehicle, the data collection device 3 may be a vehicle-mounted device.

[0253] [Software Structure]

[0254] Fig.15 An example of the software structure of the data collection device 3 according to the present embodiment is schematically shown. The control unit 31 of the data collection device 3 expands the data collection program 83 stored in the storage unit 32 to the RAM, and executes the commands included in the data collection program 83 through the CPU. Fig.15 As shown, the data collection device 3 according to the present embodiment operates as a computer including a collection unit 311 and an output unit 312 as software modules.

[0255] The collecting unit 311 is configured to collect a plurality of data sets 4 each consisting of a combination of training data 41 and correct answer data 45. The collecting of the plurality of data sets 4 in the collecting unit 311 may include: the more appropriate data sets are evaluated as appropriate because the reaction speed of the control instruction represented by the correct answer data 45 to the event meets the predetermined conditions, the more preferentially collected. The output unit 312 is configured to output the collected plurality of data sets 4 for use in machine learning.

[0256] In addition, in this variant, similar to the above-mentioned model generation device 1 and control device 2, each software module of the data collection device 3 is also implemented by the control unit 31 (CPU). That is, an example in which each software module of the data collection device 3 is implemented by a general-purpose CPU is described. However, part or all of the software modules of the data collection device 3 can also be implemented by one or more dedicated processors. The above-mentioned modules can also be implemented as hardware modules. Regarding the software structure of the data collection device 3, modules can be appropriately omitted, replaced, and added according to the implementation method.

[0257] [Working example]

[0258] Fig.161 is a flowchart showing an example of processing steps related to the collection of data sets 4 by the data collection device 3 involved in this modification. The following processing steps are an example of a data collection method executed by a computer. However, the following processing steps are only an example, and each step can be changed as much as possible. In addition, for the following processing steps, steps can be appropriately omitted, replaced, or added according to the implementation method.

[0259] (Step S301)

[0260] In step S301, the control unit 31 operates as a collection unit 311 to collect a plurality of data sets 4 each consisting of a combination of training data 41 and correct answer data 45. At this time, in one example, the control unit 31 may collect the data sets 4 regardless of whether the reaction speed is appropriate. In another example, the control unit 31 may give priority to collecting data sets whose reaction speed to an event represented by the correct answer data 45 is evaluated as appropriate because it meets a predetermined condition.

[0261] The collected data set 4 may be a newly acquired data set or selected from an already acquired data set. The newly acquired data set may be a newly generated data set or a data set obtained from another computer. Priority collection may be performed in at least one of the stages of newly acquired data sets and the stages of selecting from an already acquired data set. In addition, the stage of newly acquired data sets may be equivalent to the initial entry stage of saving (storing) data.

[0262] Prioritized collection can be performed by increasing the amount of data sets whose reaction speed is evaluated as appropriate and reducing the amount of data sets whose reaction speed is not evaluated as appropriate. In one example, reducing the amount of data sets whose reaction speed is not evaluated as appropriate may include not collecting data sets related to at least a portion of the reaction speeds that are not evaluated as appropriate.

[0263] When a device for obtaining a data set (first device) and a device for storing the obtained data set (second device) are separately provided, priority collection can be performed at any of the following three stages. (1) Whether the object data set can continue to be maintained in the storage of the first device; (2) Whether the first device sends (transmits) the object data set to the second device; (3) Whether the object data set can continue to be maintained in the storage of the second device. In addition, an example of the first device is a terminal device (user terminal, vehicle-mounted device, etc.), and an example of the second device is a server device. The data collection device 3 can be either the first device or the second device. Fig.13 In the example of , it is assumed that the data collection device 3 is the first device.

[0264] In one example, the data collection device 3 may be the first device or the second device, and may perform priority collection at the above-mentioned stage (1) or (3). In this case, the data set whose reaction speed is evaluated as appropriate is collected with higher priority, which may include: in the data set temporarily stored in the storage area (RAM, storage unit 32, storage medium 93, etc.) of the data collection device 3, the data set whose reaction speed is evaluated as appropriate according to the predetermined conditions is maintained, and the data set whose reaction speed is evaluated as inappropriate according to the predetermined conditions is deleted. Thus, by not maintaining the data set whose reaction speed is evaluated as inappropriate, the storage area of ​​the data collection device 3 can be efficiently used for the collection of appropriate data sets.

[0265] In another example, the data collection device 3 may be the first device, and priority collection may be performed in the above-mentioned stage (2). In this case, the second device is, for example, an external storage device such as an external server. In one example, the external server may be the model generation device 1 or a network server (NAS, etc.). The sending process is performed in step S302 described later. Therefore, when this method is adopted, in this step S301, the control unit 31 may evaluate the response speed of the obtained data set in order to determine whether to send it. That is, priority collection may include evaluating the response speed.

[0266] In addition, each data set 4 can be obtained by any method. As described above, the data set 4 can be obtained by the operation of the moving body MZ performed by the subject. In addition to the complete manual operation, the operation of the moving body MZ can also include an override operation for any automatic control. In addition, the data set 4 can be obtained by simulation, data expansion, etc. When multiple data sets 4 are obtained, the control unit 31 causes the processing to enter the next step S302.

[0267] (Step S302)

[0268] In step S302, the control unit 31 functions as the output unit 312. That is, the control unit 31 outputs the collected plurality of data sets 4 for use in machine learning.

[0269] Outputting for machine learning may be: keeping the collected multiple data sets 4 in a distinguishable (identifiable) state so that they can be used for machine learning. Therefore, outputting the collected multiple data sets 4 can be constituted by storing the collected multiple data sets 4 in any storage area. The arbitrary storage area may be RAM, storage unit 32, external storage device, etc. The external storage device may include an external server such as the model generation device 1 and a network server (NAS, etc.).

[0270] In one example, the control unit 31 may send a plurality of data sets 4 to the model generation device 1 via a network. The model generation device 1 may execute the above steps S102 and S103 accordingly, thereby generating a trained control model 5. In addition, when the data collection device 3 and the model generation device 1 are integrally formed, the collected data set 4 in the above step S301 may be reflected in the acquisition of a data set of a specified batch size in the above step S101 or machine learning. In this case, the output processing of step S302 may be included in the above step S102.

[0271] In another example, as described above, the data collection device 3 can reflect the priority collection in the above-mentioned stage (2). In this case, outputting multiple data sets 4 may include: sending a data set evaluated as having appropriate reaction speed according to a predetermined condition to the second device, and omitting sending a data set evaluated as having inappropriate reaction speed according to a predetermined condition to the second device. In one example, the second device may be the model generation device 1, an external server such as a network server, etc. Thus, by omitting the communication processing of the data set evaluated as having inappropriate reaction speed, it is possible to reduce the communication cost required for outputting the data set 4.

[0272] When the output of multiple data sets 4 is completed, the control unit 31 ends the processing steps of the data collection device 3 involved in this work example. In addition, the control unit 31 can repeatedly perform a series of information processing from step S301 to step S302. The timing of repetition can be appropriately determined according to the implementation method. In a typical example, the control unit 31 can start the execution of the processing of the above-mentioned step S301 according to a data collection instruction from an external computer such as the model generation device 1. Moreover, the control unit 31 can repeatedly perform a series of information processing from step S301 to step S302 until a collection stop instruction from an external computer is received. Thus, the data collection device 3 can be configured to continuously collect data sets 4 during the period when the data collection instruction is given.

[0273] (feature)

[0274] In this variant, the data collection device 3 prioritizes the collection of data sets whose reaction speed is evaluated as appropriate in the processing of step S301. As a result, after the processing of step S302, data sets whose reaction speed is evaluated as appropriate are prioritized for machine learning. Therefore, through this variant, it is also possible to expect to obtain a trained machine learning model (control model 5) that has acquired the ability to control a moving body at an appropriate reaction speed.

[0275] [5 Supplement]

[0276] The processes and units described in the present disclosure may be implemented in combination as long as no technical contradiction occurs.

[0277] Furthermore, a process described as being performed by one device may be shared and executed by a plurality of devices. Alternatively, a process described as being performed by different devices may be executed by one device. In a computer system, the hardware structure used to implement each function can be flexibly changed.

[0278] The present disclosure can also be implemented by supplying a computer with a computer program that implements the functions described in the above-mentioned embodiments, and one or more processors possessed by the computer read out and execute the program. Such a computer program can be provided to the computer via a non-transient computer-readable storage medium that can be connected to the system bus of the computer, or it can be provided to the computer via a network. Non-transient computer-readable storage media, for example, include any type of disks such as disks (floppy disks (floopy) (registered trademark), hard disk drives (HDD), etc.), optical disks (CD-ROM, DVD disks, Blu-ray disks, etc.), read-only memories (ROM), random access memories (RAM), EPROM, EEPROM, magnetic cards, flash memories, optical cards, semiconductor drives (solid-state drives, etc.), and any type of medium suitable for storing electronic commands.

Claims

1. A model generation method, which is executed by a computer, The model generation method comprises: Acquire a plurality of data sets each consisting of a combination of training data and correct answer data, wherein the training data is data representing an environment in which a mobile body moves in a time series, and the correct answer data is data representing a control instruction for the mobile body in the environment in a time series; and Using the obtained multiple data sets, performing machine learning of the control model, Implementing the machine learning includes: for each of the data sets, training the control model in such a way that the result of deriving the control instruction of the moving body according to the training data using the control model conforms to the correct answer data, and, The use of the plurality of data sets includes: using a data set whose reaction speed of the control instruction represented by the correct answer data to an event is evaluated to be more appropriate because it meets a predetermined condition.

2. The model generation method according to claim 1, The more appropriate data sets that are evaluated for the reaction rate, the more preferred ones include: increasing the sampling probability of the first data set in the machine learning, in which the reaction speed is evaluated as appropriate, among the plurality of data sets, and The second data set whose reaction speed is not evaluated as appropriate is not excluded from the object of the machine learning, but the sampling probability of the second data set in the machine learning is made lower than that of the first data set.

3. The model generation method according to claim 1 or 2, The mobile body includes a sensor, The training data includes sensor data obtained by the sensor, The start time of the event in the training data is determined based on the sensor data.

4. The model generation method according to claim 1 or 2, The moving object is a vehicle.

5. The model generation method according to claim 4, The event includes at least any one of deceleration of a preceding vehicle, cutting in of a parallel vehicle, appearance of a parked / stopped vehicle, generation of an obstacle, and change of a traffic light.

6. The model generation method according to claim 4 The control command includes at least any one of acceleration, deceleration, and steering of the vehicle.

7. A data collection method, which is executed by a computer, The data collection methods include: Collecting a plurality of data sets each consisting of a combination of training data and correct answer data, wherein the training data is data representing an environment in which a moving body moves in a time series, and the correct answer data is data representing a control instruction for the moving body in the environment in a time series; and Output the collected data sets for machine learning. The collecting of the plurality of data sets includes: giving priority to collecting data sets whose reaction speed of the control instruction represented by the correct answer data to an event is evaluated to be more appropriate because it meets a predetermined condition.

8. The data collection method according to claim 7, The more the reaction rate is evaluated, the more appropriate data sets are prioritized for collection, including: Among the data sets temporarily stored in the storage area of ​​the computer, the data sets whose reaction speed is evaluated as being appropriate according to the predetermined condition are maintained, and the data sets whose reaction speed is evaluated as being inappropriate according to the predetermined condition are deleted.

9. The data collection method according to claim 7, Outputting the plurality of data sets includes: sending the data set evaluated as having the appropriate reaction speed according to the predetermined condition to an external server, and The transmission of the data set evaluated as having the inappropriate response speed according to the predetermined condition to the external server is omitted.

10. A computer program product comprising a control program, The control program is used to make the computer execute: Acquire object data representing an environment in which the object moving body moves; Using the trained control model, deriving control instructions based on the acquired object data; and According to the result of deriving the control instruction, controlling the movement of the object moving body, The trained control model is a model generated by implementing machine learning using a plurality of data sets each consisting of a combination of training data and correct answer data, wherein the training data is data representing in a time series an environment in which a training mobile body moves, and the correct answer data is data representing in a time series a control instruction for the training mobile body in the environment, Implementing the machine learning includes: For each of the data sets, the control model is trained in such a way that the result of deriving the control instruction of the moving body according to the training data using the control model conforms to the correct answer data, and, Using the plurality of data sets for the machine learning includes: a data set whose reaction speed of the control instruction represented by the correct answer data to an event is evaluated to be appropriate because it meets a predetermined condition is given priority in the machine learning.

11. The computer program product according to claim 10, The target moving object is a vehicle.

12. The computer program product according to claim 11, The event includes at least any one of deceleration of a preceding vehicle, cutting in of a parallel vehicle, appearance of a parked / stopped vehicle, generation of an obstacle, and change of a traffic light.

13. The computer program product according to claim 11, The control command includes at least any one of acceleration, deceleration, and steering of the vehicle.

Images