Model generation method, data acquisition method and control program

Abstract

To provide a technology to obtain a trained machine learning model that has an ability to control a moving object with an appropriate reaction speed.SOLUTION: A model generation method according to one aspect of the present disclosure includes: causing a computer to acquire a plurality of data sets, each of which is composed of a combination of training data and correct answer data; and causing the computer to use the acquired plurality of data sets to perform machine learning on a control model. Using the plurality of data sets includes that the data set is evaluated as more appropriate based on a reaction speed for an event of a control command indicated by correct answer data meeting a specified condition will be used with priority.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a model generation method, a data collection method, and a control program. [Background technology]

[0002] Patent document 1 proposes a driving assistance device that is configured to acquire information indicating driving operations and information indicating the driving conditions at the time of the driving operations, determine whether the driving conditions are appropriate for learning based on the acquired information, and determine that driving operations in driving conditions that are judged to be inappropriate are not subject to learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-127207 A Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide a technology for increasing the probability of obtaining a trained machine learning model that has acquired the ability to control a moving object with an appropriate reaction speed, or a technology for controlling a moving object using the trained machine learning model obtained thereby. [Means for solving the problem]

[0005] A model generation method according to a first aspect of the present disclosure is executed by a computer. The model generation method includes acquiring a plurality of data sets each composed of a combination of training data showing an environment in which a moving object moves in a time series and ground truth data showing a control command for the moving object in the environment in a time series, and performing machine learning of a control model using the acquired plurality of data sets. The performing of the machine learning includes training the control model for each of the data sets such that a result of deriving a control command for the moving object from the training data using the control model matches the ground truth data. The using of the plurality of data sets includes preferentially using a data set that is evaluated as being appropriate because the reaction speed of the control command to an event shown by the ground truth data matches a predetermined condition. The control model may be composed of a neural network.

[0006] A data collection method according to a second aspect of the present disclosure is executed by a computer. The data collection method includes collecting a plurality of data sets each composed of a combination of training data indicating an environment in which a moving object moves in time series and ground truth data indicating a control command for the moving object in the environment in time series, and outputting the collected plurality of data sets for use in machine learning. The collecting of the plurality of data sets includes preferentially collecting data sets that are evaluated as more appropriate because the reaction speed of the control command to an event indicated by the ground truth data meets a predetermined condition.

[0007] A control program according to a third aspect of the present disclosure is a program for causing a computer to execute the following: acquiring target data indicating an environment in which a target moving object moves; deriving a control command from the acquired target data using a trained control model; and controlling the operation of the target moving object according to the result of deriving the control command. The trained control model is a combination of training data indicating an environment in which a training moving object moves in a time series and ground truth data indicating a control command for the training moving object in the environment in a time series. The vehicle control system is generated by performing machine learning using a plurality of data sets each of which is configured by combining a vehicle control command and a vehicle control information. Performing the machine learning includes training the control model for each of the data sets such that a result of deriving a control command for the vehicle from the training data using the control model matches the ground truth data. Using the plurality of data sets for the machine learning includes preferentially using for the machine learning a data set that is evaluated as more appropriate because the reaction speed of the control command to an event, which is indicated by the ground truth data, matches a predetermined condition. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide a technology for increasing the probability of obtaining a trained machine learning model that has acquired the ability to control a moving object with an appropriate reaction speed, or a technology for controlling a moving object using the trained machine learning model obtained thereby. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a situation to which the present disclosure is applied. [Diagram 2] FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of the model generating device according to the embodiment. [Diagram 3] FIG. 3 illustrates an example of a hardware configuration of a control device according to an embodiment. [Figure 4]FIG. 4 illustrates an example of a software configuration of the model generating device according to the embodiment. [Diagram 5] FIG. 5 illustrates an example of a software configuration of the control device according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of a processing procedure related to machine learning of a control model by the model generating device according to the embodiment. [Figure 7A] FIG. 7A illustrates an example of an event according to an embodiment. [Figure 7B] FIG. 7B illustrates a schematic diagram of one example of a technique for evaluating the reaction rate of the events of FIG. 7A. [Figure 8A] FIG. 8A illustrates an example of an event according to an embodiment. [Figure 8B] FIG. 8B illustrates a schematic diagram of one example of a technique for evaluating the reaction rate of the events of FIG. 8A. [Figure 9] FIG. 9 illustrates an example of an event according to the embodiment. [Figure 10A] FIG. 10A illustrates an example of an event according to an embodiment. [Figure 10B] FIG. 10B illustrates a schematic diagram of an example of a technique for evaluating the reaction rate of the event of FIG. 10B. [Figure 11A] FIG. 11A is a schematic diagram showing an example of how a data set having a more appropriate reaction rate is used preferentially under predetermined conditions in an embodiment. [Figure 11B] FIG. 11B is a schematic diagram showing an example of how a data set having a more appropriate reaction rate is used preferentially under predetermined conditions in an embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of a processing procedure regarding the operational control of a moving object by the control device according to the embodiment. [Figure 13] FIG. 13 illustrates an example of another scene to which the present disclosure is applied. [Figure 14] FIG. 14 is a diagram illustrating an example of a hardware configuration of a data collection device according to another embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of a software configuration of a data collection device according to another embodiment. [Figure 16] FIG. 16 is a flowchart illustrating an example of a processing procedure regarding data collection by a data collection device according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] According to the method proposed in Patent Document 1, it is expected that an autonomous driving model that has acquired the ability to perform only appropriate driving operations can be generated by excluding driving operations in driving situations that are determined to be inappropriate from the learning target. However, the inventors of the present invention have found that the conventional method has the following problems.

[0011] That is, a scenario is assumed in which a model is made to acquire the ability to perform automatic driving of a vehicle through machine learning. In this case, the ability acquired by a trained machine learning model depends on the learning data used for machine learning. The learning data may be collected from various drivers. In this case, the driver's ability is not necessarily constant. For example, while there are drivers who react quickly to an event such as the deceleration of a preceding vehicle (perform driving operations), there are also drivers who react slowly to the event. Therefore, the reaction speeds of drivers that appear in the collected learning data may vary.

[0012] In this regard, the conventional method simply excludes driving operations that are judged to be inappropriate from the learning target. Even appropriate driving operations can have different reaction times. Therefore, the conventional method does not necessarily obtain a trained machine learning model that has acquired the ability to perform autonomous driving with an appropriate reaction time.

[0013] This problem can arise regardless of the type of vehicle (e.g., number of wheels (two-wheeled, four-wheeled, etc.), size (large, normal, small, etc.), power source (electricity, fuel, etc.), etc.). Furthermore, such problems do not only arise in situations where a vehicle is being controlled. In terms of controlling movement, the same applies to moving objects other than vehicles. Therefore, similar problems can arise in situations where any moving object other than a vehicle (e.g., an aircraft (drone, etc.), a ship, etc.) is being controlled.

[0014] In contrast, a model generation method according to a first aspect of the present disclosure is executed by a computer. The model generation method includes acquiring a plurality of data sets each composed of a combination of training data showing an environment in which a moving object moves in time series and ground truth data showing a control command for the moving object in the environment in time series, and performing machine learning of a control model using the acquired plurality of data sets. The performing of the machine learning includes training the control model for each of the data sets such that a result of deriving a control command for the moving object from the training data using the control model matches the ground truth data. The using of the plurality of data sets includes preferentially using a data set that is evaluated as more appropriate because the reaction speed of the control command to an event shown by the ground truth data matches a predetermined condition.

[0015] The ability of a trained model generated by machine learning depends on the data set used for the machine learning. In the first aspect of the present disclosure, a data set with a more appropriate reaction speed is preferentially used for machine learning. Therefore, it is possible to increase the probability of obtaining a trained machine learning model that has acquired the ability to control a moving object with an appropriate reaction speed.

[0016] A data collection method according to a second aspect of the present disclosure is executed by a computer. The data collection method includes collecting a plurality of data sets each composed of a combination of training data indicating an environment in which a moving object moves in time series and ground truth data indicating a control command for the moving object in the environment in time series, and outputting the collected plurality of data sets for use in machine learning. The collecting of the plurality of data sets includes preferentially collecting data sets that are evaluated as more appropriate because the reaction speed of the control command to an event indicated by the ground truth data meets a predetermined condition.

[0017] In the second embodiment of the present disclosure, a data set having a more appropriate reaction rate is collected preferentially. By doing so, as in the first aspect, it is possible to preferentially use in machine learning a data set for which the reaction speed is evaluated as being more appropriate, thereby increasing the probability of obtaining a trained machine learning model that has acquired the ability to control a moving object with an appropriate reaction speed.

[0018] A control program according to a third aspect of the present disclosure is a program for causing a computer to execute the following: acquiring target data indicating an environment in which a target moving object moves; deriving a control command from the acquired target data using a trained control model; and controlling the operation of the target moving object according to the result of deriving the control command. The trained control model is generated by performing machine learning using a plurality of data sets each composed of a combination of training data indicating an environment in which a training moving object moves in a time series and ground truth data indicating a control command for the training moving object in the environment in a time series. The performing of the machine learning includes training the control model for each of the data sets such that a result of deriving a control command for the moving object from the training data using the control model matches the ground truth data. The using of the plurality of data sets for the machine learning includes preferentially using a data set for the machine learning that is evaluated as being more appropriate because the reaction speed of the control command to an event indicated by the ground truth data matches a predetermined condition.

[0019] As in the above-mentioned aspects, a trained machine learning model that has acquired the ability to control a moving object at an appropriate reaction speed can be obtained by preferentially using a data set that is evaluated as having an appropriate reaction speed for machine learning. According to the third aspect of the present disclosure, it is expected that the use of such a trained control model (machine learning model) will enable the control of a moving object to be performed at an appropriate reaction speed.

[0020] Hereinafter, an embodiment according to one aspect of the present disclosure (hereinafter also referred to as "the present embodiment") will be described with reference to the drawings. However, the present embodiment described below is merely an example of the present disclosure in all respects. Various improvements or modifications may be made without departing from the scope of the present disclosure. In implementing the present disclosure, a specific configuration according to the embodiment may be appropriately adopted. Note that while the data appearing in this embodiment is described in natural language, more specifically, it is specified in pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.

[0021] [1 Application example] 1 is a schematic diagram showing an example of a situation in which the present disclosure is applied. The system according to the present embodiment includes a model generating device 1 and a control device 2.

[0022] The model generation device 1 according to this embodiment is one or more computers configured to generate a trained control model 5 by performing machine learning. In this embodiment, the model generation device 1 acquires a plurality of data sets 4, each of which is configured by a combination of training data 41 and supervised answer data 45. The training data 41 is configured to chronologically represent an environment in which a moving object (training moving object) moves. The supervised answer data 45 is configured to chronologically represent true values ​​of control commands for a moving object in an environment represented by the corresponding training data 41.

[0023] The model generating device 1 uses the acquired multiple data sets 4 to perform machine learning of the control model 5. Performing this machine learning includes training the control model 5 so that, for each data set 4, a control command for a moving object is derived from the training data 41 using the control model 5 to match the corresponding correct answer data 45. In this machine learning, the use of multiple data sets 4 is This includes giving priority to the use of data sets for which the reaction speed to the control command events conforms to a predetermined condition and is evaluated as being appropriate. This machine learning makes it possible to generate a trained control model5 that has acquired the ability to derive control commands according to the environment in which the mobile object moves.

[0024] On the other hand, the control device 2 according to this embodiment is one or more computers configured to control the movement of a target moving object M using a trained control model 5. In this embodiment, the control device 2 acquires target data 221 indicating an environment in which the target moving object M moves. The control device 2 derives a control command from the acquired target data 221 using the trained control model 5. Then, the control device 2 controls the operation of the target moving object M according to the result of deriving the control command.

[0025] As described above, in this embodiment, in the model generation device 1, a data set evaluated as having a more appropriate reaction speed is preferentially used for machine learning to generate a trained control model 5. Since the ability of a trained model generated by machine learning depends on the data set used for the machine learning, according to this embodiment, it is expected to obtain a trained control model 5 that has acquired the ability to control a moving object at an appropriate reaction speed. Then, in the control device 2 according to this embodiment, it is expected that the control of the target moving object M can be performed at an appropriate reaction speed by using such a trained control model 5.

[0026] (Mobile) As long as the moving body (moving body M) can move automatically by mechanical control, the type of the moving body (moving body M) is not particularly limited and may be appropriately selected according to the embodiment. The moving body (moving body M) may be any movable device such as a vehicle, an aircraft, a ship, a robot device, etc. The aircraft may be at least one of an unmanned aircraft such as a drone and a manned aircraft.

[0027] 1, the moving body (moving body M) may be a vehicle. In this case, the model generation device 1 can be expected to obtain a trained control model 5 that has acquired the ability to control the vehicle with an appropriate reaction speed. In addition, the control device 2 can be expected to use such a trained control model 5 to be able to control the target vehicle with an appropriate reaction speed.

[0028] In addition, when the moving body is a vehicle, the type of the vehicle may be selected arbitrarily. The vehicle may be selected from, for example, a two-wheeled vehicle, a three-wheeled vehicle, a four-wheeled vehicle, etc. The power source of the vehicle may be selected from, for example, electricity, fuel, etc. When the vehicle is an automobile, the size of the vehicle may be selected from large, medium, semi-medium, normal, large special, small special, etc. When the vehicle is a two-wheeled vehicle, the size of the vehicle may be selected from large, normal, etc. As a typical example, the moving body (moving body M) may be an automobile with an autonomous driving capability of level 2 or higher.

[0029] (Environment / Sensor) The environment is an event observed by the moving body itself and / or its surroundings. In one example, at least a part of the environment may be observed by one or more sensors S disposed inside or outside the moving body (moving body M). Accordingly, the training data 41 and the target data 221 may each include sensor data SD obtained by one or more sensors S.

[0030] The type of the sensor S is not particularly limited as long as it can observe any environment in which the mobile object moves, and may be appropriately selected depending on the embodiment. In one example, the one or more sensors S may include a camera (image sensor), a radar, a LiDAR (Light Detection And Ranging), a sonar (ultrasonic sensor), an infrared sensor, a GNSS (Global Navigation Satellite System) / GPS (Global Positioning Satellite) module, etc.

[0031] (Control command) The control command is related to the operation of the moving object. The configuration of the control command may be appropriately determined depending on the embodiment. For example, the control command may be configured by acceleration, deceleration, steering, or a combination thereof. Acceleration and deceleration may include gear changes. In this case, the model generation device 1 can be expected to acquire a trained control model 5 that has acquired the ability to control acceleration, deceleration, steering, or a combination thereof with an appropriate reaction speed. In addition, the control device 2 can be expected to be able to control acceleration, deceleration, steering, or a combination thereof with an appropriate reaction speed by using such a trained control model 5.

[0032] In one example, when the moving body (moving body M) is a vehicle, the control command may include at least one of acceleration, deceleration, and steering of the vehicle. When at least one of acceleration, deceleration, and steering is included, the control command may be expressed by a path. Accordingly, the control model 5 may be expressed as a path planner.

[0033] The control command may further include a command related to the operation of the moving body. For example, if the moving body (moving body M) is a vehicle, the control command may include vehicle operations such as turn signals, hazard lights, horn, and communication processing (for example, transmitting data to a center, making an emergency call, etc.).

[0034] (Dataset) Each data set 4 may be generated appropriately. Each data set 4 may be generated automatically by the operation of a computer, or may be generated manually by at least partially including the operation of an operator. Typically, while a subject controls a moving object entirely by manual operation, operation data of the moving object and environmental data may be collected. The environmental data (observation data) may be obtained by a sensor S mounted on the moving object. The operation data may be obtained by recording the manual operation by the subject. Then, the training data 41 of each data set 4 may be generated from the environmental data. The correct answer data 45 may be generated from the operation data. That is, typically, the data set 4 may be generated from the result of a manual operation by the subject (e.g., manual driving of a vehicle). When the data set 4 is obtained using a moving object, the target moving object from which the data set 4 is obtained may include the moving object M that is the target of using the generated trained control model 5, or may not include the moving object M. That is, the training moving object may include the target moving object M, or may not include the target moving object M. The training moving object may include a moving object used by an end user. In this case, the end user may be the subject. Additionally, the training vehicles may include vehicles used for experimental purposes.

[0035] However, the method of generating the dataset 4 is not limited to this example, and may be appropriately selected depending on the embodiment. In another example, the dataset 4 may be generated from the result of a manual operation by the subject, as in the above example, but the manual operation may include an operation by the subject during the operation of partial automatic control, such as an override operation for any automatic control. In another example, at least a portion of the dataset 4 may be obtained by a virtual method such as a simulation. In another example, at least a portion of the dataset 4 may be acquired by a reinforcement learning framework. In another example, at least a portion of the dataset 4 may be generated by data augmentation for any dataset. Data augmentation is performed by generating new training data by changing attribute values ​​of the training data. For example, if the training data includes an image, the parameter change may be performed by image processing such as translation, enlargement, reduction, rotation, and noise addition to the image. When at least a part of the dataset 4 is obtained by data augmentation, reaction rates and reaction rate-dependent parameters for the training data of an arbitrary dataset (original dataset) may be obtained. One or more new datasets may be generated by changing the values ​​of attributes other than at least one of the attributes and adding correct answer data accordingly. The multiple datasets 4 may include the generated new dataset.

[0036] (Control Model) The control model 5 is configured by a machine learning model having one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used for the calculation of the desired inference (in this case, derivation of a control command). Machine learning is to adjust (optimize) the value of the calculation parameter using learning data (in this case, a plurality of data sets 4). The configuration and type of the machine learning model are not particularly limited and may be appropriately selected depending on the embodiment. The machine learning model may be configured by, for example, a neural network, a support vector machine, a regression model, a decision tree model, etc.

[0037] As an example, the control model 5 may be configured by a neural network. The structure of the neural network may be appropriately determined depending on the embodiment. The structure of the neural network may be specified by, for example, the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationship between the nodes in each layer, etc. In one example, the neural network may have a recursive structure. In addition, the neural network may include any layer, such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, an unpooling layer, a normalization layer, a dropout layer, or a Long short-term memory (LSTM). The neural network may have any mechanism, such as an Attention mechanism. The control model 5 (neural network) may include any model such as a graph neural network (GNN), a diffusion model, a generative model (e.g., a generative adversarial network, a transformer, etc.), etc. When a neural network is used for the control model 5, the weight of the connection between each node included in the control model 5 (neural network) and the threshold value of each node are examples of calculation parameters.

[0038] As long as the control command can be derived from the environment of the moving body, the input / output form of the control model 5 is not particularly limited and may be appropriately selected according to the embodiment. For example, the control model 5 may be configured to derive a control command at one or more time points from environmental data at one or more time points. Furthermore, the control model 5 may be configured to be able to accept time series data due to its structure. As an example, the control model 5 may be configured to be recursive, thereby being able to accept time series data. As another example, the control model 5 may be configured to input environmental data at multiple time points in a lump. Alternatively, the control model 5 may be configured structurally to be unable to accept time series data. For example, the control model 5 may be configured to derive a control command at one time point from environmental data at one time point. In this case, the control model 5 may be used to obtain a calculation result for the time series data by sequentially receiving data at each time point in the time series data and sequentially outputting the calculation results. Furthermore, the control model 5 may be configured to instantly derive a control command. Alternatively, the control model 5 may be configured to derive control commands at multiple future time points in a lump. In this case, at least a part of the control commands derived collectively may be used to control the moving body (moving body M).

[0039] As long as the control model 5 is involved in at least a part of the inference process of deriving a control command from the environment of the moving body, the process contents executed by the control model 5 are not particularly limited and may be appropriately selected depending on the embodiment. In one example, the control model 5 may be configured to execute surrounding recognition and path planning (route / trajectory planning). The control model 5 may be further configured to execute motion planning (motion / control planning). In other words, the control model 5 may be an end-to-end model.

[0040] In addition, if the output of the control model 5 can control the movement of the moving object, The force form may be appropriately selected according to the embodiment. The control model 5 may be configured to directly output a control command. Alternatively, the control command may be obtained by performing any information processing (interpretation processing) on ​​the output of the control model 5. The control command may be configured to directly indicate the control amount (control instruction value, control output amount) of the moving body, such as, for example, an accelerator control amount, a brake control amount, or a steering wheel steering angle. Alternatively, the control command may be configured to indirectly indicate the control amount of the moving body, such as, for example, a path, a state after control, or the like. In this case, the control amount of the moving body may be obtained from the control command by performing any information processing. In one example, when the moving body is a vehicle, the control amount of the vehicle may be obtained by applying the inference result obtained from the control model 5 to a vehicle model. The vehicle model may have various parameters such as an accelerator, a brake, a steering wheel steering, etc., and may be appropriately configured to derive the control amount from indirect information (path, state after control, etc.).

[0041] The correct answer data 45 of each data set 4 may be appropriately configured to directly or indirectly indicate a control command. When a control command is derived by executing any arithmetic processing from the output of the control model 5, the correct answer data 45 may be provided for the control command derived from the output of the control model 5 (i.e., the correct answer data 45 may be configured to directly indicate the control command). Alternatively, the correct answer data 45 may be provided for the output of the control model 5 (i.e., the correct answer data 45 may be configured to indirectly indicate the control command).

[0042] (event) An event may include any occurrence that may involve the operation of a mobile object. An event may also include any occurrence that can be detected by a sensor S. Detection by a sensor S means determination based on a sensor value. That is, the start time of an event may be identified by sensor data. A detection (determination, identification) method may be appropriately determined depending on the event. The sensor data may be analyzed by any method, whereby the start time of an event may be identified (that is, the occurrence of an event may be detected).

[0043] Accordingly, the mobile object (mobile object M) may include a sensor S. The training data 41 may include sensor data SD obtained by the sensor S. The start time of an event in the training data 41 may be identified by the sensor data SD. This allows the reaction speed in each data set 4 to be mechanically evaluated, so that the efficiency of determining the data set 4 to be preferentially used in machine learning can be improved. In other words, the task of selecting the data set 4 to be preferentially used can be at least partially automated, thereby reducing the amount of work. Note that the start time of an operation for an event appears in the correct answer data 45. Therefore, in this embodiment, the reaction speed for an event (i.e., the time from the occurrence time of the event to the start time of the operation) can be identified from the training data 41 and the correct answer data 45.

[0044] (Example of an event) As an example, when the moving body (moving body M) is a vehicle, the event may include at least one of the following: deceleration of a preceding vehicle relative to the vehicle, cutting in of a vehicle traveling parallel to the vehicle, occurrence of a parked vehicle, occurrence of an obstacle, and a change in a traffic light. The obstacle may include any object that may impede the travel of the vehicle. The obstacle may be, for example, a pedestrian, a bicycle, etc. In this case, the model generation device 1 can be expected to acquire a trained control model 5 that has acquired the ability to control the vehicle with an appropriate reaction speed for at least one of these events. Furthermore, the control device 2 can be expected to use such a trained control model 5 to be able to control the target vehicle with an appropriate reaction speed for at least one of these events.

[0045] In addition, if the event subject to automatic driving (automatic control) includes deceleration of the leading vehicle, The control command may include a command to decelerate in response to the preceding vehicle. If the target event includes a cut-in by a parallel vehicle, the control command may include a command to decelerate in response to the parallel vehicle. If the target event includes the occurrence of a parked vehicle, the control command may include at least one of deceleration and steering in response to the parked vehicle. If the target event includes the occurrence of an obstacle, the control command may include at least one of deceleration and steering in response to the obstacle. If the target event includes a change in a traffic light, the control command may include a command to accelerate or decelerate in response to the traffic light. These events may also be applied to other types of moving bodies (e.g., aircraft, ships, etc.).

[0046] (Meets specified conditions) The predetermined conditions may be appropriately defined so that an appropriate reaction rate can be evaluated according to the event. In one example, the predetermined conditions may be defined so that a faster reaction rate is evaluated as more appropriate. In this case, a dataset having a faster reaction rate to an event (evaluated as an appropriate reaction rate) may be preferentially used for machine learning. However, the predetermined conditions may not be limited to such an example. In another example, the predetermined conditions may define a range (upper limit and lower limit) of an appropriate reaction rate. In this case, a dataset whose reaction rate falls within the range defined by the predetermined conditions may be preferentially used for machine learning, and other datasets (i.e., datasets whose reaction rate is faster or slower than the reaction rate in the range defined by the predetermined conditions) may not be preferentially used for machine learning.

[0047] (Preferential use) The preferential use of datasets for which the reaction rate is evaluated to be more appropriate may be configured in any manner that makes the preferentially used datasets more likely to be reflected in the training of the control model 5 than datasets that are not preferentially used.

[0048] As an example, whether to use a dataset may be determined simply depending on whether it is prioritized. That is, preferential use of a dataset with a more appropriate reaction rate may be achieved by using the dataset with an appropriate reaction rate for training the control model 5, and not using the dataset with an inappropriate reaction rate for training the control model 5. According to this method, the reaction rate evaluation can be reflected in machine learning very easily.

[0049] As another example, a dataset having a reaction rate evaluated as appropriate among the multiple datasets 4 is set as a first dataset, and a dataset having a reaction rate not evaluated as appropriate (i.e., not meeting a predetermined condition) is set as a second dataset. Preferential use of a dataset having a reaction rate evaluated as more appropriate may be achieved by increasing the sampling probability in machine learning of the first dataset among the multiple datasets 4 and lowering the sampling probability in machine learning of the second dataset compared to the first dataset.

[0050] In this case, the use of a dataset with a higher reaction speed may be further configured by excluding at least a part of the second dataset from the machine learning target (i.e., setting the sampling probability to 0). Alternatively, the use of a dataset with a higher reaction speed may be further configured by not excluding the second dataset from the machine learning target (i.e., not setting the sampling probability to 0). In actual situations, due to external factors, an operation with a slow reaction speed may be executed as an appropriate operation. For example, sudden operations such as sudden braking and sudden steering are assumed to be operations with a slow reaction speed. In addition, since it takes a long time for an operation to be required after an event is detected, the reaction speed may be evaluated as being slow in appearance. In this regard, by not excluding the second dataset from the machine learning target, these operations can also be learned by the control model 5. Therefore, it is expected that the robustness of the operation against the event will be improved.

[0051] As yet another example, preferential use of a dataset having a more appropriate reaction rate may be configured by increasing the training weight of a first dataset among the multiple datasets 4 and decreasing the training weight of a second dataset. For example, increasing the training weight may be configured by increasing the learning rate, and decreasing the training weight may be configured by decreasing the learning rate. In other words, preferential use of a dataset having a more appropriate reaction rate may be configured by increasing the amount of update of the parameters of the control model 5 in one training session for the dataset having a higher priority. In yet another example, preferential use may include both the sampling probability and weighting techniques described above.

[0052] The priority of use may be set appropriately. In one example, the priority may be set in two stages (i.e., two stages of preferred / non-preferred). In another example, the priority may be set in three or more stages. In this case, the priority may differ between the data sets that are used preferentially (i.e., there may be a difference in merit). Similarly, the priority may differ between the data sets that are not used preferentially.

[0053] (Controlling the movement) In one example, controlling the operation of the target moving body M may be configured by directly controlling the target moving body M. In another example, the moving body (moving body M) may be equipped with a dedicated control device such as a controller. In this case, controlling the operation of the target moving body M by the control device 2 may be configured by indirectly controlling the target moving body M by providing the derived result to the dedicated control device.

[0054] (System Configuration) In one example, as shown in Fig. 1, the model generating device 1 and the control device 2 may be configured to be able to communicate (connect) with each other via a network. The type of network is not particularly limited and may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, and the like. However, the method of exchanging data between the model generating device 1 and the control device 2 is not limited to this example and may be appropriately selected depending on the embodiment. In another example, data may be exchanged using a storage medium.

[0055] 1, the model generating device 1 and the control device 2 are separate computers. However, the system configuration need not be limited to this example. In another example, the model generating device 1 and the control device 2 may be configured by a single computer. Furthermore, at least one of the model generating device 1 and the control device 2 may be configured by multiple computers.

[0056] 1, the control device 2 is mounted inside the moving body M. However, the location of the control device 2 is not limited to this example. The control device 2 may be disposed outside the moving body M as long as it is capable of directly or indirectly controlling the operation of the moving body M.

[0057] [2 Configuration example] [Hardware configuration example] <Model generation device> Fig. 2 is a schematic diagram showing an example of a hardware configuration of the model generating device 1 according to the present embodiment. As shown in Fig. 2, the model generating device 1 according to the present embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0058] The control unit 11 includes a hardware processor, such as a CPU (Central Processing Unit), The control unit 11 includes a random access memory (RAM), a read only memory (ROM), etc., and is configured to execute information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource.

[0059] The storage unit 12 may be configured, for example, by a hard disk drive, a solid state drive, or the like. The storage unit 12 (and RAM, ROM) is an example of a memory resource. In this embodiment, the storage unit 12 stores various information such as a model generation program 81, a plurality of data sets 4, learning result data 125, and the like.

[0060] The model generation program 81 is a program for causing the model generation device 1 to execute information processing (FIG. 6 described below) related to machine learning of the control model 5. The model generation program 81 includes a series of instructions for the information processing. The learning result data 125 is configured to indicate information related to the generated trained control model 5. In this embodiment, the learning result data 125 is generated as a result of executing the model generation program 81.

[0061] The communication interface 13 is an interface for performing wired or wireless communication via a network. The communication interface 13 may be configured, for example, by a wired LAN (Local Area Network) module, a wireless LAN module, etc. The model generation device 1 may perform data communication with another computer (for example, the control device 2) via the communication interface 13.

[0062] The input device 14 is a device for performing input, such as a mouse, a keyboard, etc. The output device 15 is a device for performing output, such as a display, a speaker, etc. An operator can operate the model generation device 1 by using the input device 14 and the output device 15. The input device 14 and the output device 15 may be integrally configured, for example, by a touch panel display or the like.

[0063] The drive 16 is a device for reading various information such as a program stored in the storage medium 91. At least one of the model generation program 81, the multiple data sets 4, and the learning result data 125 may be stored in the storage medium 91 instead of the storage unit 12 or together with the storage unit 12. The storage medium 91 is configured to accumulate various information (stored programs, etc.) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The model generation device 1 may acquire at least one of the model generation program 81 and the multiple data sets 4 from the storage medium 91.

[0064] 2 illustrates a disk-type storage medium such as a CD or a DVD as an example of the storage medium 91. However, the type of the storage medium 91 is not limited to the disk type. Examples of storage media other than the disk type include semiconductor memories such as flash memories. The type of the drive 16 may be appropriately selected depending on the type of the storage medium 91.

[0065] In addition, with regard to the specific hardware configuration of the model generating device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), It consists of an ECU (Electronic Control Unit), a GPU (Graphics Processing Unit), etc. At least one of the communication interface 13, the input device 14, the output device 15, and the drive 16 may be omitted. The model generating device 1 may be composed of a plurality of computers. In this case, the hardware configuration of each computer is the same. The model generating device 1 may be a computer designed specifically for the service to be provided, a general-purpose server device, a general-purpose PC (Personal Computer), etc. ), an industrial PC, a terminal device (e.g., a tablet PC, etc.), etc.

[0066] <Control device> Fig. 3 is a schematic diagram showing an example of a hardware configuration of the control device 2 according to the present embodiment. As shown in Fig. 3, the control device 2 according to the present embodiment is a computer in which a control unit 21, a storage unit 22, a communication interface 23, an input device 24, an output device 25, a drive 26, and an external interface 27 are electrically connected.

[0067] The control unit 21 to the drive 26 and the storage medium 92 of the control device 2 may be configured similarly to the control unit 11 to the drive 16 and the storage medium 91 of the above-described model generating device 1. The control unit 21 (CPU) is an example of a processor resource of the control device 2, and the storage unit 22 (and RAM, ROM) is an example of a memory resource of the control device 2. In this embodiment, the storage unit 22 stores various information such as a control program 82 and learning result data 125.

[0068] The control program 82 is a program for causing the control device 2 to execute information processing (see FIG. 12 described below) relating to the automatic control of the target moving object M by the trained control model 5. The control program 82 includes a series of instructions for the information processing. At least one of the control program 82 and the learning result data 125 may be stored in a storage medium 92 instead of or together with the storage unit 22. The control device 2 may acquire at least one of the control program 82 and the learning result data 125 from the storage medium 92.

[0069] The control device 2 may perform data communication with another computer (e.g., the model generating device 1) via the communication interface 23. An operator can operate the control device 2 by using the input device 24 and the output device 25. The input device 24 and the output device 25 may be integrally configured with, for example, a touch panel display or the like.

[0070] The external interface 27 is an interface for connecting to an external device. The external interface 27 may be, for example, a USB (Universal Serial Bus) port, a dedicated port, etc. The type and number of the external interfaces 27 may be appropriately determined according to the type and number of the connected external devices. In this embodiment, the control device 2 may be connected to the sensor S via the external interface 27. At least a part of the target data 221 may be composed of sensor data obtained by the sensor S. Note that the connection method of the sensor S may not be limited to this example. In another example, the sensor S may be connected via the communication interface 23.

[0071] In addition, regarding the specific hardware configuration of the control device 2, components can be omitted, replaced, and added as appropriate depending on the embodiment. For example, the control unit 21 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, FPGA, DSP, ECU, GPU, etc. At least one of the communication interface 23, the input device 24, the output device 25, the drive 26, and the external interface 27 may be omitted. The control device 2 may be configured with multiple computers. In this case, the hardware configurations of the computers may be the same or may not be the same. The control device 2 may be a computer designed exclusively for the service provided, a general-purpose computer, a mobile phone including a smartphone, a tablet PC (Personal Computer), etc. In the case where the moving object M is a vehicle, the control device 2 may be an in-vehicle device.

[0072] [Software configuration example] <Model generation device> 4 shows an example of a software configuration of the model generation device 1 according to this embodiment. The control unit 11 of the model generation device 1 loads the model generation program 81 stored in the storage unit 12 into the RAM, and executes instructions included in the model generation program 81 by the CPU. As a result, the model generation device 1 operates as a computer including a learning data acquisition unit 111, a learning processing unit 112, and a storage processing unit 113 as software modules. That is, in this embodiment, each software module of the model generation device 1 is realized by the control unit 11 (CPU).

[0073] The learning data acquisition unit 111 is configured to acquire a plurality of data sets 4 each composed of a combination of training data 41 and correct answer data 45. The learning processing unit 112 is configured to perform machine learning of the control model 5 by using the acquired plurality of data sets 4. In this embodiment, performing machine learning includes training the control model 5 so that, for each data set 4, a result of deriving a control command for a moving object from the training data 41 using the control model 5 matches the corresponding correct answer data 45. In addition, in the machine learning, using the plurality of data sets 4 includes preferentially using a data set whose reaction speed to an event of a control command indicated by the correct answer data 45 matches a predetermined condition and is evaluated as being appropriate. By performing this machine learning, a trained control model 5 is generated.

[0074] The storage processing unit 113 is configured to store the trained control model 5 generated by machine learning. In one example, the storage processing unit 113 may be configured to generate learning result data 125 indicating the trained control model 5 generated as a result of machine learning. As long as information for executing the arithmetic processing of the trained control model 5 can be held, the configuration of the learning result data 125 is not particularly limited and may be appropriately determined according to the embodiment. As one example, the learning result data 125 may be configured to include information indicating values ​​of arithmetic parameters adjusted by machine learning. In some cases, the learning result data 125 may be configured to include information indicating the configuration of the trained control model 5 (for example, the structure of a neural network, etc.). The storage processing unit 113 may be configured to store the generated learning result data 125 in a predetermined storage area. The learning result data 125 may be provided to the control device 2 at any timing.

[0075] <Control device> Fig. 5 shows a schematic diagram of an example of a software configuration of the control device 2 according to the present embodiment. The control unit 21 of the control device 2 loads the control program 82 stored in the storage unit 22 into the RAM, and executes instructions included in the control program 82 by the CPU. As a result, as shown in Fig. 5, the control device 2 according to the present embodiment operates as a computer including an acquisition unit 211, a derivation unit 212, and an operation control unit 213 as software modules. That is, in the present embodiment, similarly to the model generating device 1, each software module of the control device 2 is also realized by the control unit 21 (CPU).

[0076] The acquisition unit 211 is configured to acquire target data 221 indicating an environment in which a target moving object M moves. The derivation unit 212 includes a trained control model 5 generated by the model generation device 1 by holding the learning result data 125. The derivation unit 212 is configured to derive a control command from the acquired target data 221 using the trained control model 5. The operation control unit 213 is configured to control the operation of the target moving object M according to the result of deriving the control command (i.e., the control command derived by the trained control model 5).

[0077] <Other> In this embodiment, an example is described in which each software module of the model generating device 1 and the control device 2 is realized by a general-purpose CPU. However, some or all of the above software modules may be realized by one or more dedicated processors. Each of the above modules may be realized as a hardware module. With regard to the software configurations of the model generating device 1 and the control device 2, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0078] [3 Example of operation] [Model generation device] 6 is a flowchart showing an example of a processing procedure related to machine learning of the control model 5 by the model generation device 1 according to this embodiment. The following processing procedure is an example of a model generation method executed by a computer. However, the following processing procedure of the model generation device 1 is merely an example, and each step may be changed as much as possible. In addition, steps of the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0079] <Step S101> In step S101, the control unit 11 operates as a learning data acquisition unit 111. That is, the control unit 11 acquires a plurality of data sets 4, each of which is composed of a combination of training data 41 and answer data 45.

[0080] The training data 41 is configured to chronologically indicate the environment in which the moving object moves. In one example, the training data 41 may include sensor data SD obtained by a sensor S. In addition, the training data 41 may include any information that may be involved in control, such as a set speed, a speed limit, map information, and navigation information. The correct answer data 45 is configured to directly or indirectly indicate a control command for the moving object in the environment indicated by the corresponding training data 41 in a chronological order.

[0081] As described above, each dataset 4 may be generated (collected) as appropriate. Each generated dataset 4 may be stored in the model generating device 1 (at least one of the storage unit 12 and the storage medium 91). Alternatively, each dataset 4 may be stored in another computer such as a network server (for example, NAS: Network Attached Storage). In this case, when performing machine learning, the control unit 21 may acquire each dataset 4 via a network, an external storage device, the storage medium 91, or the like. Each dataset 4 may be stored in the form of a database.

[0082] At least a part of the multiple data sets 4 may be generated by the model generating device 1. At least a part of the multiple data sets 4 may be generated by a computer other than the model generating device 1. When the data sets 4 are generated by another computer, the control unit 11 may acquire the data sets 4 generated by the other computer via, for example, a network, an external storage device, a storage medium 91, or the like.

[0083] The number of data sets 4 to be acquired may be appropriately determined depending on the embodiment. When a plurality of data sets 4 are acquired, the control unit 11 advances the process to the next step S102.

[0084] <Step S102> In step S102, the control unit 11 operates as the learning processing unit 112. That is, the control unit 11 performs machine learning of the control model 5 using the acquired multiple data sets 4. In the machine learning, the control unit 11 trains the control model 5 so that, for each data set 4, a result of deriving a control command for a moving object from the training data 41 using the control model 5 matches the corresponding ground truth data 45.

[0085] The control model 5 (machine learning model) has one or more computation parameters for executing computation processing to solve an inference task. Training the control model 5 is to optimize (adjust) values ​​of computation parameters of the control model 5 (machine learning model) according to given learning data (plurality of data sets 4). The machine learning method may be appropriately determined according to the embodiment such as the type and structure of the machine learning model used in the control model 5. Any method such as backpropagation, solving an optimization problem, etc. may be adopted as a method for adjusting the computation parameters.

[0086] (An example of a machine learning method) As a typical example, when the control model 5 is configured by a neural network, the control unit 11 first prepares the control model 5 to be processed by machine learning. The structure of the prepared control model 5, the initial values ​​of the weights of the connections between the neurons, and the initial values ​​of the thresholds of the neurons may be provided by a template or by an operator's input. When performing re-learning, the control unit 11 may prepare the control model 5 based on learning result data obtained by performing past machine learning. Next, the control unit 11 uses the training data 41 of each data set 4 as input data and the correct answer data 45 as a teacher signal (label) to execute a learning process (supervised learning) of the control model 5.

[0087] As shown in FIG. 4, as an example of the learning process, in the first step, the control unit 11 inputs the training data 41 of each data set 4 and executes a forward propagation arithmetic process of the control model 5. As a result of this arithmetic process, the control unit 11 acquires an output value corresponding to an inference result (a direct or indirect derivation result of a control command) for the training data 41 from the control model 5. In the second step, the control unit 11 calculates an error between the acquired output value and the corresponding correct answer data 45. In the third step, the control unit 11 calculates a gradient of the calculated error. Then, the control unit 11 backpropagates the gradient of the calculated error by the error backpropagation method to calculate an error in the value of the arithmetic parameter (weight of the connection between each node, threshold value of each node, etc.) of the control model 5. In the fourth step, the control unit 11 updates the value of the arithmetic parameter based on the calculated error. The degree to which the value of the arithmetic parameter is updated may be adjusted by a learning rate.

[0088] The control unit 11 adjusts the value of the calculation parameter of the control model 5 so that the sum of errors between the output value output from the control model 5 and the ground truth data 45 for each data set 4 is reduced by repeating the above first to fourth steps. The adjustment of the value of this calculation parameter may be repeated until a predetermined condition is satisfied, such as performing the adjustment a set number of times or the calculated sum of errors being equal to or less than a threshold value. The threshold value may be set appropriately depending on the embodiment. In addition, machine learning conditions such as an objective function (cost function, loss function, error function) for calculating the error, a learning rate, and an optimization algorithm may be set appropriately depending on the embodiment.

[0089] The adjustment of the values ​​of the calculation parameters of the control model 5 may be performed on the mini-batch. As an example, before performing the processes of the first to fourth steps, the control unit 11 may generate a mini-batch by extracting any samples (datasets) from a plurality of data sets 4. The size of the mini-batch may be set appropriately depending on the embodiment. Then, the control unit 11 may perform the processes of the first to fourth steps on the data set 4 included in the generated mini-batch. When repeating the first to fourth steps, the control unit 11 may generate a mini-batch again, and perform the processes of the first to fourth steps on the new generated mini-batch.

[0090] Note that the machine learning method does not have to be limited to such supervised learning examples, and other A method may be adopted at least partially. As another example, deep reinforcement learning may be adopted. In this case, some of the multiple data sets 4 may be obtained as a result of an episode in reinforcement learning. Alternatively, the control unit 11 may generate multiple new data sets from the result of an episode in reinforcement learning, and adjust (optimize) the operation parameters of the control model 5 using the generated multiple new data sets and the multiple data sets 4. For example, any method such as R2D3: Recurrent Replay Distributed DQN from Demonstrations may be adopted for deep reinforcement learning.

[0091] (Examples of events) In this embodiment, in the above machine learning, the control unit 11 preferentially uses a data set whose reaction speed to a control command event indicated by the correct answer data 45 is evaluated as appropriate because the reaction speed meets certain conditions.

[0092] As described above, the event may include any occurrence that may involve the operation of the moving body. The event may also include any occurrence that can be detected by the sensor S. As an example, the moving body (moving body M) may be a vehicle, and the event may include at least one of the following: deceleration of a preceding vehicle relative to the moving body M, a cut-in of a vehicle traveling parallel to the moving body M, the occurrence of a parked vehicle, the occurrence of an obstacle, and a change in a traffic light.

[0093] The predetermined conditions may be appropriately defined so as to evaluate an appropriate reaction speed depending on the event. The start time of the event may be identified by sensor data obtained by a sensor S. The reaction speed may be defined as the time between the start time of the event and the start time of the action control (operation) for the event. A specific example is shown below.

[0094] (A) Slowing of the preceding vehicle 7A shows an example of an event (slowing down of a leading vehicle) in a schematic manner. In the example of FIG. 7A, vehicle MA is the vehicle that encounters the event. That is, in the learning phase, it may be assumed that the dataset 4 is obtained from vehicle MA. Meanwhile, vehicle MB is the leading vehicle of vehicle MA.

[0095] When the vehicle MA encounters the deceleration of the preceding vehicle MB, the vehicle MA may execute any operation to deal with the deceleration of the preceding vehicle MB. One example of a control operation that the vehicle MA may take is a deceleration operation in response to the preceding vehicle MB. Therefore, in one example, the reaction speed may be defined as the time from the time when the preceding vehicle MB starts deceleration (the time when the deceleration is detected) to the time when the deceleration operation is started in the vehicle MA.

[0096] The deceleration of the leading vehicle MB is determined by, for example, the speed, position or distance of a camera, radar, LiDAR, etc. The deceleration of the preceding vehicle MB can be detected by a sensor (sensor S) related to the preceding vehicle MB. An index for detecting the deceleration of the preceding vehicle MB may be defined as appropriate. In one example, a collision risk index such as a time-to-collision (TTC) or a margin-to-collision (MTC) may be used as an index for detecting the deceleration of the preceding vehicle MB.

[0097] On the other hand, the deceleration operation of the vehicle MA may be detected based on at least one of the speed, acceleration, and braking amount of the vehicle MA. As an example, the timing of the execution of the deceleration operation of the vehicle MA may be detected by a threshold evaluation of at least one of the speed, acceleration, and braking amount of the vehicle MA.

[0098] Therefore, in one example of this embodiment, the reaction speed of the deceleration operation to the deceleration of the preceding vehicle MB may be defined as the time from the time when the deceleration of the preceding vehicle MB is detected (the start time of the event) to the time when the deceleration operation of the vehicle MA is detected (the start time of the operation). The reaction rate is evaluated as fast, and the longer the time, the slower the reaction rate.

[0099] FIG. 7B is a schematic diagram showing an example of a method for evaluating the reaction speed of a response operation to the deceleration of a preceding vehicle. In the example of FIG. 7B, a collision risk index is adopted as an index for detecting the deceleration of the preceding vehicle MB, and the acceleration (negative acceleration) of the vehicle MA is adopted as an index for detecting the deceleration operation. A threshold value TA is set for the collision risk index to detect the start time of an event (deceleration of the preceding vehicle MB), and a threshold value TB is set for the acceleration of the vehicle MA to detect the start time of the deceleration operation. The threshold value TB may be set, for example, taking into consideration external factors such as the influence of the road surface. Each threshold value (TA, TB) may be set manually by an operator, or may be set at least partially automatically based on statistics or the like. In the example of FIG. 7B, the reaction speed can be calculated by the time from the event start time detected by the threshold value TA (based on the collision risk index) to the start time of the deceleration operation detected by the threshold value TB (based on the acceleration of the vehicle MA).

[0100] In addition, the operation (operation control) of the vehicle MA with respect to the preceding vehicle MB may not be limited to deceleration as in the above example, and may be appropriately selected depending on the embodiment. In another example, when encountering deceleration of the preceding vehicle MB, the vehicle MA may perform a lane change. In this case, the start time of the lane change operation may be detected by at least one of the speed, acceleration, braking amount, acceleration amount, and steering amount of the vehicle MA, and the reaction speed may be calculated accordingly. The steering amount may be measured, for example, by steering torque. In another example, the start time of the lane change operation may be detected by a vehicle operation such as a turn signal operation. In addition, a signal of the preceding vehicle MB, such as the presence or absence of the stop lamp of the preceding vehicle MB, may be used as an indicator for detecting the deceleration of the preceding vehicle MB.

[0101] (B) Cut-in of a vehicle running parallel to the ground FIG. 8A shows an example of an event (cut-in of a parallel vehicle). In the example of FIG. 8A, vehicle MC is a vehicle that encounters the event. That is, in the learning stage, it may be assumed that dataset 4 is obtained from vehicle MC. Meanwhile, vehicle MD is a vehicle that runs parallel to vehicle MC. In the example of FIG. 8A, it is assumed that vehicle MD runs parallel to vehicle MC and cuts in front of vehicle MC.

[0102] When the vehicle MC encounters the cut-in of the parallel running vehicle MD, any operation may be performed in the vehicle MC to deal with the cut-in of the parallel running vehicle MD. One example of a control operation that the vehicle MC may take is a deceleration operation in response to the cut-in of the parallel running vehicle MD. Therefore, in one example, the reaction speed may be defined as the time from the time when the parallel running vehicle MD starts to cut in (the time when the cut-in is detected) to the time when the deceleration operation is started in the vehicle MC.

[0103] The cut-in of the parallel running vehicle MD is performed based on, for example, the speed, position or distance of a camera, radar, LiDAR, etc. The cut-in of the parallel running vehicle MD can be detected by a sensor (sensor S) related to the distance between the vehicle MD and the parallel running vehicle MD. An index for detecting the cut-in of the parallel running vehicle MD may be defined as appropriate. In one example, a distance index such as an overlap amount or a distance between the parallel running vehicle MD and the white line may be used as an index for detecting the cut-in of the parallel running vehicle MD. For example, the cut-in of the parallel running vehicle MD may be detected by the timing when the overlap amount becomes equal to or less than a certain value, the timing when the parallel running vehicle MD reaches the white line, or the like. The overlap amount is the distance in the vehicle width direction (the left-right direction with respect to the traveling direction of the vehicle, the up-down direction in the figure) between the other vehicle (parallel running vehicle MD) and the predicted course MCA of the host vehicle (vehicle MC). The predicted course MCA may be, for example, the predicted course range of the vehicle MC indicated by the dotted line in the figure. On the other hand, similar to the above (A), the deceleration operation of the vehicle MC may be detected by at least one of the speed, acceleration, and braking amount of the vehicle MC. Therefore, in one example of this embodiment, the reaction speed of the deceleration operation to the cut-in of the parallel running vehicle MD is determined by the time from the time when the cut-in of the parallel running vehicle MD is detected (the start time of the event) to the time when the deceleration operation of the vehicle MC is detected (the start time of the operation). may be defined.

[0104] FIG. 8B is a schematic diagram showing an example of a method for evaluating the reaction speed of a response operation to a cut-in of a parallel vehicle. In the example of FIG. 8B, an lap amount is adopted as an index for detecting the cut-in of the parallel vehicle MD, and an acceleration (negative acceleration) of the vehicle MC is adopted as an index for detecting a deceleration operation. The threshold value TC is set for the lap amount to detect the start time of an event (cut-in of the parallel vehicle MD), and the threshold value TD is set for the acceleration of the vehicle MC to detect the start time of the deceleration operation. The threshold value TD may be set in consideration of external factors such as the influence of the road surface. Each threshold value (TC, TD) may be set manually by an operator, or may be set at least partially automatically based on statistics or the like. In the example of FIG. 8B, the reaction speed can be calculated based on the time from the event start time detected by the threshold value TC (based on the lap amount) to the start time of the deceleration operation detected by the threshold value TD (based on the acceleration of the vehicle MC).

[0105] The operation (operation control) of the vehicle MC with respect to the parallel vehicle MD is not limited to deceleration as in the above example, and may be appropriately selected depending on the embodiment. In another example, when the vehicle MC encounters the cut-in of the parallel vehicle MD, the vehicle MC may perform a lane change. In this case, the start time of the lane change operation may be detected from at least one of the speed, acceleration, braking amount, acceleration amount, and steering amount of the vehicle MC, and the reaction speed may be calculated accordingly. In another example, the start time of the lane change operation may be detected by a vehicle operation such as a turn signal operation. In addition, signals of the parallel vehicle MD, such as whether or not the stop lamps of the parallel vehicle MD are turned on and whether or not the turn signal lamps are turned on, may be used as indicators for detecting the cut-in of the parallel vehicle MD.

[0106] In addition, the manner in which the parallel vehicle MD cuts in may differ depending on the situation. For example, when cutting in at a place where driving is not possible, such as a merging road or a construction section, the vehicle MC may be required to take action at an earlier timing in order to ensure that the parallel vehicle MD cuts in, compared to other cut-ins (e.g., normal lane changes). Therefore, the cut-in event may be classified into types according to the situation.

[0107] (C) Occurrence of parked vehicles FIG. 9 shows a schematic example of an event (occurrence of a parked vehicle). In the example of FIG. 9, the vehicle ME is a vehicle that encounters the event. That is, in the learning phase, it may be assumed that the data set 4 is obtained from the vehicle ME. Meanwhile, the vehicle MF is a parked vehicle. In the example of FIG. 9, it is assumed that the vehicle MF has parked in front of the vehicle ME.

[0108] When encountering the occurrence of a parked vehicle MF, the vehicle ME may execute any operation to deal with the parked vehicle MF. In one example, the occurrence of the parked vehicle MF may be handled in the same manner as the deceleration of the preceding vehicle MB in (A) above, by replacing the preceding vehicle MB with the parked vehicle MF. That is, in one example, an example of a control operation that the vehicle ME may take is a deceleration or avoidance operation (such as changing lanes) in response to the occurrence of the parked vehicle MF. The reaction speed may be defined as the time from the time when the parked vehicle MF occurs (the time when the parked vehicle MF is detected) to the time when the deceleration or avoidance operation is started in the vehicle ME.

[0109] The occurrence of parked vehicles MF is, for example, based on the speed, position, or distance from the vehicle using cameras, radars, LiDAR, etc. The parked vehicle MF can be detected by a sensor (sensor S) related to the distance. In one example, the above-mentioned collision risk index may be used as an index for detecting the parked vehicle MF. In addition to the above-mentioned collision margin time and collision margin, indices such as the distance to the parked vehicle MF, the inter-vehicle distance / traveling speed (THW: Time Head Way) may be used as a collision risk index for detecting the parked vehicle MF. Therefore, in this embodiment, In one embodiment, the reaction speed to the occurrence of a parked vehicle MF is a function of the collision risk index value. It may be calculated based on the time from the event start time detected by threshold judgment to the operation start time detected by threshold judgment for the value of the operation amount of the vehicle ME (at least one of speed, acceleration, braking amount, acceleration amount, and steering amount). In another example, the start time of the avoidance operation may be detected by a vehicle operation such as a turn signal operation. Furthermore, a signal of the parked vehicle MF, such as whether or not the hazard lamps of the parked vehicle MF are on, may be used as an index for detecting the occurrence of the parked vehicle MF.

[0110] (D) Obstacle occurrence The occurrence of an obstacle is similar to the occurrence of the parked vehicle MF. In the above example, by replacing the parked vehicle MF with an obstacle, the reaction speed of the operation to the occurrence of the obstacle can be evaluated in the same manner as the reaction speed of the operation to the occurrence of the parked vehicle MF. In one example of this embodiment, the reaction speed to the occurrence of the obstacle may be calculated based on the time from the event start time detected by threshold judgment on the value of the collision risk index to the start time of the operation detected by threshold judgment on the value of the vehicle movement amount. The collision risk index may be measured by replacing the parked vehicle MF with an obstacle. In another example, the start time of the avoidance operation may be detected by a vehicle operation such as a turn signal operation. As described above, the obstacle may be, for example, a pedestrian, a bicycle, or the like.

[0111] (E) Changes in traffic lights Fig. 10A shows an example of an event (a change in a traffic light) in a schematic manner. In the example of Fig. 10A, the vehicle MG is the vehicle that encounters the event. That is, in the learning phase, it may be assumed that the dataset 4 is obtained from the vehicle MG.

[0112] When encountering a change in the traffic light MT, the vehicle MG may execute any operation to deal with the change in the traffic light MT. As an example, when the traffic light MT changes from a proceeding signal to a deceleration signal or a caution signal (changing from a green signal to a yellow signal), one example of a possible control operation of the vehicle MG is a deceleration operation to stop in front of the traffic light MT or an acceleration operation to pass through the road where the traffic light MT is installed. Therefore, as an example, the reaction speed may be defined as the time from the time when the traffic light MT starts to change (the time when the change is detected) to the time when the vehicle MG starts to decelerate or accelerate.

[0113] The change in the traffic light MT can be detected by, for example, a sensor (sensor S) such as a camera. The result of identifying the color of the lighted signal in the traffic light MT may be used as an index for detecting the change in the traffic light MT. The color of the lighted signal in the traffic light MT may be identified by any method. Meanwhile, the deceleration or acceleration operation of the vehicle MG may be detected by at least one of the speed, acceleration, accelerator amount, and brake amount of the vehicle MG. Therefore, in one example of this embodiment, the reaction speed of the deceleration or acceleration operation to the change in the traffic light MT may be defined by the time from the time when the change in the traffic light MT is detected (the start time of the event) to the time when the deceleration or acceleration operation of the vehicle MG is detected (the start time of the operation). Separate thresholds may be set for the deceleration operation and the acceleration operation.

[0114] FIG. 10B shows a schematic example of a method for evaluating the reaction speed of a response operation to a change in a traffic light. In the example of FIG. 10B, it is assumed that the color change of the traffic light MT occurs instantaneously. In this example of FIG. 10B, the identification result of the color of the illuminated signal of the traffic light MT is adopted as an index for detecting the change in the traffic light MT, and the acceleration of the vehicle MG is adopted as an index for detecting a deceleration or acceleration operation. A threshold value TE is set for the acceleration of the vehicle MG to detect the start time of a deceleration operation, and a threshold value TF is set for the acceleration of the vehicle MG to detect the start time of an acceleration operation. Each threshold value (TE, TF) may be set taking into consideration external factors such as the influence of the road surface. Each threshold value (TE, TF) may be set manually by an operator, or at least partially automatically based on statistics or the like. In the example of Fig. 10B, the reaction speed of the deceleration operation can be calculated based on the time from the time when the traffic light MT changes (event start time) to the start time of the deceleration operation detected by the threshold value TE (based on the acceleration of the vehicle MG). Also, the reaction speed of the acceleration operation can be calculated based on the time from the time when the traffic light MT changes (event start time) to the start time of the acceleration operation detected by the threshold value TF (based on the acceleration of the vehicle MG).

[0115] The event of the change of the traffic light MT is not limited to the above example (changing from a green light to a yellow light) and may be set appropriately depending on the embodiment. In another example, instead of the timing of changing to a yellow light, the timing of changing to a red light may be detected as the start time of the change event of the traffic light MT.

[0116] Also, in the above example, the timing at which the traffic light MT completes its change (for example, the timing at which the green light changes from a lit state to an off state and the yellow light changes to a lit state) is defined as the time at which the traffic light MT changes (the start time of the event). However, the definition of the start time of the event need not be limited to this example, and may be set appropriately depending on the embodiment. In another example, the traffic light MT may include a traffic light for pedestrians in addition to a traffic light for vehicles (FIG. 10A illustrates an image of a traffic light for vehicles). In this case, any timing from when the pedestrian traffic light starts flashing to when the traffic light for vehicles completes its change may be defined as the time at which the traffic light MT changes (the start time of the event).

[0117] (Examples of preferential use) The control unit 11 preferentially uses, for machine learning, a dataset having a more appropriate reaction rate among the multiple datasets 4. That is, the control unit 11 reflects a dataset having an appropriate reaction rate in the training of the control model 5 more than a dataset having a reaction rate that is not evaluated as appropriate. In this embodiment, preferential use may be configured by at least one of the following three methods.

[0118] (1) First Method 11A is a schematic diagram showing an example of a first method for preferentially using a data set having a reaction rate evaluated as being appropriate. As the first method, the control unit 11 may use a data set having a reaction rate evaluated as being appropriate for training the control model 5, and may not use a data set having a reaction rate not evaluated as being appropriate for training the control model 5.

[0119] The graph in FIG. 11A shows a histogram of the number of events (samples) versus the reaction rate. The number of events may be the number of data sets, or may be counted at least partially differently from the collection of the data sets. The reaction rate may be evaluated by any statistic. In one example, the statistic may be the mean, mode, n-th percentile, etc., where n may be any number. The n-th percentile may be, for example, the 50th percentile (median), the 25th percentile, etc.

[0120] In one example, whether the reaction rate is appropriate (whether it complies with a predetermined condition) may be classified by this statistic (corresponding to the case where the set value is 0 in the example of FIG. 11A). When evaluating that the faster the reaction rate is, the more appropriate it is, a data set with a faster reaction rate than the reference statistic may be used for machine learning as a data set with an appropriate reaction rate. That is, the control unit 11 may use the reference statistic as the upper limit value of the reaction rate. When the range of the appropriate reaction rate is specified, the upper limit value and the lower limit value may be set from the corresponding statistic. As a specific example, assuming that the reaction rates are tallied from those with the fastest reaction rates, the lower limit value of the reaction rate may be specified by the 25th percentile value, and the upper limit value of the reaction rate may be specified by the 75th percentile value.

[0121] In another example, the control unit 11 may derive a reference value by performing any operation on the statistics, without using the statistics as it is. Simply, the control unit 11 may derive a reference value by adding or subtracting a set value to the statistics. Then, whether the reaction rate is appropriate or not may be classified by the derived reference value instead of the statistics. That is, when the faster the reaction rate is evaluated as being appropriate, a data set having a faster reaction rate than the reference value may be used for machine learning as a data set having an appropriate reaction rate. When a range of an appropriate reaction rate is specified, the upper limit value and the lower limit value may each be derived from the same statistics. Alternatively, the upper limit value and the lower limit value may each be derived from different statistics. In each case, the magnitude of the set value for deriving the upper limit value may be the same as or different from the magnitude of the set value for deriving the lower limit value.

[0122] In the example of FIG. 11A, the mode is used as the evaluation index. A reference value is calculated by subtracting a set value from the mode reaction speed (the time between the event start time and the operation start time). Then, a data set having a reaction speed faster than the reference value is evaluated as a data set having an appropriate reaction speed (hatched range in FIG. 11A). In this first method, the control unit 11 may exclude data sets having reaction speeds that are not evaluated as appropriate, and perform the above machine learning process using only data sets having reaction speeds that are evaluated as appropriate.

[0123] (2) Second Method 11B is a schematic diagram showing an example of the second method in which a dataset having a more appropriate reaction rate is used with higher priority. As the second method, the control unit 11 may increase the sampling probability of a first dataset having an appropriate reaction rate among the multiple datasets 4, and may set a lower sampling probability of a second dataset having an inappropriate reaction rate. In other words, the control unit 11 may increase the number of sampling times in machine learning for a dataset having an appropriate reaction rate.

[0124] In one example, the sampling probability (number of times) corresponds to the probability (number of times) of being extracted into a mini-batch in the above machine learning. Therefore, if the sampling probability is high (if the number of sampling times is large), the number of times used to adjust the values ​​of the calculation parameters of the control model 5 will be increased. This makes it possible to achieve that the training of the control model 5 reflects a data set whose reaction rate is evaluated as appropriate, compared to a data set whose reaction rate is not evaluated as appropriate.

[0125] The method of setting the sampling probability (number of times) according to the reaction rate may be appropriately determined according to the embodiment. In one example, the control unit 11 may determine the sampling probability (number of times) of each data set 4 using a function formula appropriately designed so that the sampling probability (number of times) increases as the reaction rate conforms to a predetermined condition.

[0126] Furthermore, in the second method, the control unit 11 may exclude at least a part of the second data set in which the reaction rate is not evaluated as appropriate from the target of machine learning (i.e., set the sampling probability to 0). Alternatively, the control unit 11 may not exclude the second data set in which the reaction rate is not evaluated as appropriate from the target of machine learning (i.e., do not set the sampling probability to 0).

[0127] In the example of FIG. 11B, the predetermined condition is defined so that the faster the reaction rate, the more appropriate the evaluation. In other words, the sampling probability is set higher for a data set with a faster reaction rate. In addition, the second data set is not excluded from the machine learning target. As a specific example, the sampling probability P(i) of a data set with the i-th reaction rate among the multiple data sets 4 is , may be defined by Equation 1 below:

[0128]

number

[0129] As an optional measure, the training weight of each data set 4 may be adjusted according to the sampling probability. In one example, it may happen that a data set with a low sampling probability is not reflected at all in the adjustment of the values ​​of the calculation parameters of the control model 5. To avoid this, the training weight may be increased for data sets with a lower sampling probability, without invalidating the priority of the sampling probability. As described above, increasing the training weight may be achieved by increasing the learning rate. As a specific example, if the reaction rate of the i-th data set is The weight of training w i may be calculated from the sampling probability P(i) according to Equation 2 below:

[0130]

number

[0131] (3) The third method As a third method, the control unit 11 may set a large training weight for a first data set among the multiple data sets 4, in which the reaction speed is evaluated as appropriate, and set a small training weight for a second data set in which the reaction speed is not evaluated as appropriate, and then perform the above machine learning processing.

[0132] In one example, the control unit 11 may set the weight of training to be large by increasing the learning rate in the machine learning, and may set the weight of training to be small by decreasing the learning rate. If the value of the learning rate is large, the update amount when adjusting the value of the calculation parameter of the control model 5 in the fourth step of the machine learning becomes large. This makes it possible to achieve that the data set whose reaction speed is evaluated as appropriate is reflected by the training of the control model 5, compared to the data set whose reaction speed is not evaluated as appropriate.

[0133] The method of setting the training weight according to the reaction rate may be appropriately determined according to the embodiment. In one example, the control unit 11 may determine the training weight of each data set 4 using a function formula appropriately designed so that the more the reaction rate conforms to a predetermined condition, the larger the training weight.

[0134] Furthermore, in the third method, the control unit 11 may exclude at least a part of the second data set whose reaction speed is not evaluated as appropriate from the target of machine learning (i.e., set the training weight to 0). A data set whose training weight is 0 may not be used for machine learning. Alternatively, the control unit 11 may not exclude the second data set whose reaction speed is not evaluated as appropriate from the target of machine learning (i.e., do not set the training weight to 0).

[0135] In the first to third methods, the extraction process (arithmetic process) of the dataset to be preferentially used may be executed by any component. In one example, the extraction process may be executed by the control unit 11. That is, the control unit 11 may refer to the reaction rate of each dataset 4 and give each dataset 4 a ranking in terms of use for machine learning according to the first to third methods. In another example, even if the control unit 11 does not execute any process, the extraction process may be achieved by a mechanism of a storage area that stores the dataset 4. As a specific example, when the dataset 4 is stored in a database, the extraction process may be achieved as a database process. That is, the control unit 11 may extract each dataset 4 from the database in a state in which a ranking in terms of use for machine learning is given according to the reaction rate (for example, sorted in order of reaction rate on the database). Not using the dataset for machine learning may be achieved by not extracting it from the database. The control unit 11 may implement machine learning in which the dataset whose reaction rate is evaluated as being appropriate is used preferentially by using the dataset extracted from the database as it is for machine learning. In the above third method, the control unit 11 may set the training weight according to the order of extraction (for example, if the data sets are extracted in order of fastest reaction speed, the control unit 11 may assign a heavier training weight to the data set extracted earlier).

[0136] In addition, in the above first to third methods, whether or not to use a dataset in which the operation start time is earlier than the event start time (a dataset located to the left of the event start time in Figures 11A and 11B) for machine learning may be selected appropriately depending on the embodiment.

[0137] The control unit 11 may employ at least one of the above first to third methods, so that among the multiple data sets 4, data sets having reaction rates evaluated as more appropriate are preferentially used for machine learning. The above first to third methods may be employed in combination. In this embodiment, the control unit 11 can generate a trained control model 5 by executing the above-mentioned machine learning process. When the machine learning of the control model 5 is completed, the control unit 11 proceeds to the next step S103.

[0138] Note that the operation required for each event may be different. Therefore, the control unit 11 may generate a trained control model 5 for each event. In addition, the acquired multiple data sets 4 may include data sets related to events other than the target event to be ranked according to reaction speed. In this case, the data sets related to the other events may also be ranked according to any method. Alternatively, the data sets related to the other events may be used for machine learning without being ranked according to reaction speed.

[0139] <Step S103> Returning to Fig. 6, in step S103, the control unit 11 operates as the storage processing unit 113. That is, the control unit 11 generates information related to the trained control model 5 generated by machine learning as learning result data 125. The learning result data 125 may be appropriately configured to include information for reproducing the trained control model 5. The control unit 11 stores the generated learning result data 125 in a predetermined storage area.

[0140] The predetermined storage area may be, for example, the RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination of these. The storage medium may be, for example, a CD or a DVD. , a semiconductor memory, or the like, and the control unit 11 may store the learning result data 125 in the storage medium via the drive 16. The external storage device may be, for example, a data server such as a NAS. In this case, the control unit 11 may use the communication interface 13 to store the learning result data 125 in the data server via the network. The external storage device may be, for example, an external storage device. The external storage device may be appropriately connected to the model generation device 1. For example, the model generation device 1 may further include an external interface, and may be connected to the external storage device via this external interface.

[0141] When the storage of the machine learning results is completed, the control unit 11 ends the processing procedure of the model generating device 1 according to this operation example.

[0142] The generated learning result data 125 may be provided to the control device 2 at any timing and by any method. For example, the control unit 11 may transfer the learning result data 125 to the control device 2 as part of the process of step S103 or separately from the process of step S103. The control device 2 may acquire the learning result data 125 by receiving this transfer. For example, the control device 2 may acquire the learning result data 125 by accessing the model generation device 1 or a data server via a network using the communication interface 23. For example, the control device 2 may acquire the learning result data 125 via the storage medium 92. For example, the learning result data 125 may be incorporated in advance into the control device 2.

[0143] Furthermore, the control unit 11 may update or newly generate the learning result data 125 by repeatedly executing the processes of steps S101 to S103 on a regular or irregular basis. During this repetition, at least a part of the dataset 4 used for machine learning may be changed, modified, added, deleted, or the like as appropriate. Then, the control unit 11 may update the learning result data 125 held by the control device 2 by providing the updated or newly generated learning result data 125 to the control device 2 by any method.

[0144] [Control device] FIG. 12 is a flowchart showing an example of a processing procedure for automatic control of a target moving body M using a trained control model 5 by the control device 2 according to this embodiment. The following processing procedure is an example of a control method executed by a computer. However, the following processing procedure of the control device 2 is merely an example, and each step may be modified as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0145] <Step S201> In step S201, the control unit 21 operates as the acquisition unit 211. That is, the control unit 21 acquires target data 221 indicating the environment in which the target moving object M moves. In one example, the target data 221 may include sensor data obtained by a sensor S. In addition, the target data 221 may include any information that may be involved in control, such as a set speed, a speed limit, map information, navigation information, etc. The control unit 21 may acquire various information by any method. After acquiring the target data 221, the control unit 21 proceeds to the next step S202.

[0146] <Step S202> In step S202, the control unit 21 operates as the derivation unit 212. That is, the control unit 21 derives a control command from the acquired target data 221 by using the trained control model 5.

[0147] In addition, the control unit 21 may receive the learning result at any timing before executing step S202. The control unit 21 may set the trained control model 5 to a usable state (i.e., a state in which calculation processing can be executed) by referring to the data 125. Furthermore, if a trained control model 5 is generated for each event, the control unit 21 may identify an event that the target moving body M has encountered or is likely to encounter. The occurrence of an event or its probability can be detected by the sensor S. The control unit 21 may select a trained control model 5 corresponding to the identified event from among the multiple trained control models 5 held. The control unit 21 may then derive a control command from the acquired target data 221 using the selected trained control model 5.

[0148] The arithmetic processing of the trained control model 5 may be appropriately determined depending on the type, configuration, structure, etc. of the control model 5. In one example, when the control model 5 is configured by a neural network, the control unit 21 inputs the target data 221 to the trained control model 5 and executes a forward propagation arithmetic processing of the trained control model 5. As a result of executing this arithmetic processing, the control unit 21 can acquire an output value corresponding to the result of deriving the control command from the trained control model 5. When the output of the control model 5 is configured to indirectly indicate the control command, the control unit 21 may derive the control command by executing a predetermined arithmetic processing on the output of the control model 5. When the derivation of the control command is completed, the control unit 21 proceeds to the next step S203.

[0149] <Step S203> In step S203, the control unit 21 operates as the operation control unit 213. That is, the control unit 21 controls the operation of the target moving object M according to the result of deriving the control command. The control unit 21 may control the operation of the target moving object M directly or indirectly.

[0150] When the control of the target moving body M is completed, the control unit 21 ends the processing procedure of the control device 2 according to this operation example. The control unit 21 may repeatedly execute the series of information processing from step S201 to step S203. The timing of repetition may be appropriately determined according to the embodiment. In one example, the control unit 21 may repeatedly execute the series of information processing from step S201 to step S203 for a predetermined period (for example, from when the power source of the moving body M is started to when it is stopped). This allows the control device 2 to continuously perform automatic control of the moving body M.

[0151] [Features] In this embodiment, in the process of step S102, a data set with a more appropriate reaction speed is preferentially used for machine learning to generate a trained control model 5. Since the ability of a trained model generated by machine learning depends on the data set used for the machine learning, according to this embodiment, it is expected to obtain a trained control model 5 that has acquired the ability to control a moving object at an appropriate reaction speed. Furthermore, by using such a trained control model 5 in the process of step S202, it is expected that the target moving object M can be controlled at an appropriate reaction speed.

[0152] [4 Variations] Although the embodiment of the present disclosure has been described in detail above, the above description is merely an example of the present disclosure in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present disclosure. For example, the following modifications are possible.

[0153] 13 is a schematic diagram showing an example of another example of a situation to which the present disclosure is applied. In addition to the model generating device 1 and the control device 2, the system according to the modified example includes a data collecting device 3. The data collecting device 3 is one or more computers configured to collect a data set 4. In this modified example, the data collecting device 3 collects a combination of training data 41 and ground truth data 45. Collecting the multiple data sets 4 may include preferentially collecting data sets that are evaluated as more appropriate because the reaction speed of the control command event indicated by the correct answer data 45 meets a predetermined condition. Then, the data collection device 3 outputs the collected multiple data sets 4 for use in machine learning.

[0154] In the example of FIG. 13, it is assumed that the data collection device 3 is mounted on a moving object MZ and collects a data set 4 in response to an operation of the moving object MZ by a subject. In this example, the data collection device 3 may generate training data 41 from sensor data and other information (e.g., set speed, speed limit, map information, navigation information, etc.) obtained from the sensor S. The data collection device 3 may also generate correct answer data 45 from the result of the subject's operation during the period in which the information constituting the training data 41 is acquired. Then, the data collection device 3 may acquire the data set 4 by associating the generated training data 41 and correct answer data 45. The data collection device 3 may be configured to execute a process of collecting the data set 4 in response to a command from an external computer such as the model generation device 1. Data exchange between the devices 1 to 3 may be performed by any method.

[0155] The form of the data collection device 3 need not be limited to the example shown in FIG. 13. In another example, the data collection device 3 may not be mounted on the moving body MZ, but may be arranged separately from the moving body MZ. In another example, the data collection device 3 may be configured integrally with the control device 2 or the model generating device 1. That is, the control device 2 may also function as the data collection device 3 (in this case, the moving body MZ is the moving body M). Or, the model generating device 1 may also function as the data collection device 3.

[0156] [Hardware configuration] Fig. 14 is a schematic diagram showing an example of a hardware configuration of the data collection device 3 according to this modification. As shown in Fig. 14, the data collection device 3 according to this modification is a computer in which a control unit 31, a storage unit 32, a communication interface 33, an input device 34, an output device 35, a drive 36, and an external interface 37 are electrically connected.

[0157] The control unit 31 to the external interface 37 and the storage medium 93 of the data collection device 3 may be configured similarly to the control unit 21 to the external interface 27 and the storage medium 92 of the control device 2. The control unit 31 (CPU) is an example of a processor resource of the data collection device 3, and the storage unit 32 (and RAM, ROM) is an example of a memory resource of the data collection device 3. In this modification, the storage unit 32 stores various information such as a data collection program 83, a data set 4, etc.

[0158] The data collection program 83 is a program for causing the data collection device 3 to execute information processing (FIG. 16 described below) related to the collection of the data set 4. The data collection program 83 includes a series of instructions for the information processing. The data set 4 may be accumulated as a result of execution of the data collection program 83. At least one of the data collection program 83 and the data set 4 may be stored in a storage medium 93 instead of or together with the storage unit 32. The data collection device 3 may acquire the data collection program 83 from the storage medium 93.

[0159] The data collection device 3 may perform data communication with another computer (e.g., the model generating device 1) via the communication interface 33. An operator (e.g., a subject) can operate the data collection device 3 by using the input device 34 and the output device 35. The input device 34 and the output device 35 may be integrally configured with, for example, a touch panel display. The data collection device 3 may perform data communication with another computer (e.g., the model generating device 1) via the external interface 37. The data collection device 3 may be connected to the sensor S. However, the method of connecting the sensor S is not limited to this example. In another example, the data collection device 3 may be connected to the sensor S via the communication interface 23.

[0160] In addition, regarding the specific hardware configuration of the data collection device 3, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 31 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, FPGA, DSP, ECU, GPU, etc. At least one of the communication interface 33, the input device 34, the output device 35, the drive 36, and the external interface 37 may be omitted. The data collection device 3 may be configured with multiple computers. In this case, the hardware configuration of each computer may or may not be the same. The data collection device 3 may be a computer designed specifically for the service provided, as well as a general-purpose server device, a general-purpose computer, a mobile phone including a smartphone, a tablet PC (Personal Computer), etc. In the case where the moving object MZ is a vehicle, The data collection device 3 may be an in-vehicle device.

[0161] [Software configuration] Fig. 15 shows an example of a software configuration of the data collection device 3 according to this embodiment. The control unit 31 of the data collection device 3 loads the data collection program 83 stored in the storage unit 32 into the RAM, and executes instructions included in the data collection program 83 by the CPU. As a result, as shown in Fig. 15, the data collection device 3 according to this embodiment operates as a computer including a collection unit 311 and an output unit 312 as software modules.

[0162] The collection unit 311 is configured to collect a plurality of data sets 4, each of which is composed of a combination of training data 41 and correct answer data 45. Collecting the plurality of data sets 4 in the collection unit 311 may include preferentially collecting a data set that is evaluated as more appropriate because the reaction speed of the control command to an event indicated by the correct answer data 45 meets a predetermined condition. The output unit 312 is configured to output the collected plurality of data sets 4 for use in machine learning.

[0163] In this modification, each software module of the data collecting device 3 is realized by the control unit 31 (CPU) in the same manner as the model generating device 1 and the control device 2. That is, an example in which each software module of the data collecting device 3 is realized by a general-purpose CPU is described. However, some or all of the software modules of the data collecting device 3 may be realized by one or more dedicated processors. Each of the above modules may be realized as a hardware module. With regard to the software configuration of the data collecting device 3, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0164] [Example] 16 is a flowchart showing an example of a processing procedure for collecting a data set 4 by the data collection device 3 according to this modified example. The following processing procedure is an example of a data collection method executed by a computer. However, the following processing procedure is merely an example, and each step may be modified as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0165] (Step S301) In step S301, the control unit 31 operates as a collection unit 311 and collects a plurality of data sets 4 each of which is composed of a combination of training data 41 and correct answer data 45. In this case, in one example, the control unit 31 may collect the data set 4 regardless of whether the reaction speed is appropriate or not. In another example, the control unit 31 may preferentially collect a data set that is evaluated as being appropriate because the reaction speed to the event of the control command indicated by the correct answer data 45 meets a predetermined condition.

[0166] Collecting the data set 4 may be a new acquisition of the data set or a selection from an already acquired data set. Acquiring the data set may be a new creation of the data set or a data set obtained from another computer. Preferential collection may be performed at least in a step of newly acquiring the data set and / or a step of selecting from an already acquired data set. Note that the step of newly acquiring the data set may correspond to a first entry step of saving (storing) the data.

[0167] The preferential collection may be configured by increasing the amount of datasets in which the reaction rates are evaluated as appropriate and decreasing the amount of datasets in which the reaction rates are not evaluated as appropriate. In one example, decreasing the amount of datasets in which the reaction rates are not evaluated as appropriate may include not collecting datasets related to at least some of the reaction rates that are not evaluated as appropriate.

[0168] In the case where the device for acquiring the data set (first device) and the device for storing the acquired data set (second device) are provided separately, the priority collection may be performed in one of three stages: (1) Whether or not to continue to store the target data set in the storage of the first device (2) Whether the first device transmits (transfers) the target data set to the second device (3) Whether or not the target data set will be retained in the storage of the second device. An example of the first device is a terminal device (user terminal, in-vehicle device, etc.), and an example of the second device is a server device. The data collection device 3 may be either the first device or the second device. In the example of FIG. 13, the data collection device 3 is assumed to be the first device.

[0169] In one example, the data collection device 3 may be the first device or the second device, and may perform preferential collection at the above stage (1) or (3). In this case, preferential collection of a dataset with a more appropriate reaction rate may be configured by maintaining a dataset with a reaction rate evaluated as appropriate under a predetermined condition, and deleting a dataset with a reaction rate evaluated as inappropriate under a predetermined condition, among the datasets temporarily stored in the storage area (RAM, storage unit 32, storage medium 93, etc.) of the data collection device 3. In this way, by not maintaining a dataset with a reaction rate evaluated as inappropriate, the storage area of ​​the data collection device 3 can be efficiently used for collection of an appropriate dataset.

[0170] In another example, the data collection device 3 may be the first device, and may perform preferential collection at the above stage (2). In this case, the second device is, for example, an external storage device such as an external server. In one example, the external server may be the model generation device 1 or a network server (NAS, etc.). The transmission process is performed in step S302, which will be described later. Therefore, when this form is adopted, in this step S301, the control unit 31 may evaluate the reaction rate of the obtained data set to determine whether or not to transmit. In other words, preferential collection may include evaluating the reaction rate.

[0171] Each data set 4 may be acquired by any method. As described above, the data set 4 may be acquired through the operation of the moving body MZ by the subject. The operation of the moving body MZ may include a completely manual operation as well as an override operation of any automatic control. Alternatively, the data set 4 may be acquired by a method such as simulation or data augmentation. When data set 4 is acquired, the control unit 31 advances the process to the next step S302.

[0172] (Step S302) In step S302, the control unit 31 operates as the output unit 312. That is, the control unit 31 outputs a plurality of collected data sets 4 for use in machine learning.

[0173] Outputting for use in machine learning may be holding the collected multiple data sets 4 in a distinguishable (identifiable) state so that they can be used for machine learning. Therefore, outputting the collected multiple data sets 4 may consist of saving the collected multiple data sets 4 in an arbitrary storage area. The arbitrary storage area may be a RAM, a storage unit 32, an external storage device, etc. The external storage device may include an external server such as the model generating device 1 or a network server (NAS, etc.).

[0174] In one example, the control unit 31 may transmit a plurality of data sets 4 to the model generating device 1 via a network. In response to this, the model generating device 1 may execute the above steps S102 and S103 to generate a trained control model 5. When the data collecting device 3 is configured integrally with the model generating device 1, collecting the data set 4 in the above step S301 may be reflected in obtaining a data set of a specified batch size in the above step S101 or machine learning. In this case, the output process of step S302 may be included in the above step S102.

[0175] In another example, as described above, the data collection device 3 may reflect the priority of collection at the above stage (2). In this case, outputting the multiple data sets 4 may be configured by transmitting a data set whose reaction rate is evaluated to be appropriate under predetermined conditions to the second device, and omitting transmission of a data set whose reaction rate is evaluated to be inappropriate under predetermined conditions to the second device. In one example, the second device may be the model generation device 1 or an external server such as a network server. This can reduce communication costs for outputting the data set 4 by omitting communication processing for the data set whose reaction rate is evaluated to be inappropriate.

[0176] When the output of the plurality of data sets 4 is completed, the control unit 31 ends the processing procedure of the data collecting device 3 according to this operation example. The control unit 31 may repeatedly execute a series of information processing from step S301 to step S302. The timing of repetition may be appropriately determined according to the embodiment. In a typical example, the control unit 31 may start executing the processing of the above step S301 in response to a data collection command from an external computer such as the model generating device 1. Then, the control unit 31 may repeatedly execute a series of information processing from step S301 to step S302 until a collection stop command is received from the external computer. In this way, the data collecting device 3 may be configured to continuously collect the data set 4 while a data collection command is given.

[0177] (Features) In this modification, the data collection device 3 preferentially collects a data set whose reaction speed is evaluated as more appropriate in the process of step S301. This allows the data set whose reaction speed is evaluated as more appropriate to be preferentially used for machine learning in the process of step S302 and thereafter. Therefore, this modification can also be expected to obtain a trained machine learning model (control model 5) that has acquired the ability to control a moving object with an appropriate reaction speed.

[0178] [5 Supplementary Note] The processes and means described in this disclosure may be freely used as long as no technical contradictions arise. It can be implemented in combination with

[0179] Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Or, a process described as being performed by different devices may be executed by one device. In a computer system, the hardware configuration for implementing each function can be flexibly changed.

[0180] The present disclosure can also be realized by supplying a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors of the computer read and execute the program. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. Non-transitory computer-readable storage media include, for example, any type of disk, such as a magnetic disk (e.g., a floppy disk, a hard disk drive (HDD), etc.), an optical disk (e.g., a CD-ROM, a DVD disk, a Blu-ray disk, etc.), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, a semiconductor drive (e.g., a solid-state drive), and any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0181] 1...Model generation device, 11: control unit, 12: storage unit, 13: communication interface, 14...input device, 15...output device, 16...drive, 81...model generation program, 91...storage medium, 111... learning data acquisition unit, 112... learning processing unit, 113...storage processing unit, 125...learning result data, 2...Control device, 21: control unit, 22: storage unit, 23: communication interface, 24...input device, 25...output device, 26...drive, 27…External interface, 82...control program, 92...storage medium, 211: Acquisition unit; 212: Derivation unit; 213: Operation control unit; 221: Target data; 3...Data collection device, 31: control unit, 32: storage unit, 33: communication interface, 34...input device, 35...output device, 36...drive, 37…External interface, 83...Data collection program, 93...Storage medium, 311: collection unit; 312: output unit; 4…Dataset, 41...training data, 45...correct data, 5…Control model, M: moving object, S: sensor

Claims

1. 1. A computer-implemented method for generating a model, comprising: The model generation method includes: Obtaining a plurality of data sets each composed of a combination of training data indicating an environment in which a moving object moves in time series and ground truth data indicating a control command for the moving object in the environment in time series; performing machine learning of a control model using the obtained plurality of data sets; Including, The performing of the machine learning includes training the control model for each of the data sets such that a result of deriving a control command for the moving object from the training data using the control model matches the ground truth data; and Using the plurality of data sets includes preferentially using a data set that is evaluated as being appropriate because the reaction speed of the control command to the event indicated by the correct answer data meets a predetermined condition. Model generation method.

2. The more appropriate the reaction rate is, the more the data set is used preferentially. Increasing the sampling probability in the machine learning of a first data set among the plurality of data sets, in which the reaction rate is evaluated as being appropriate; and A second dataset in which the reaction rate is not evaluated as appropriate is not excluded from the machine learning target, and a sampling probability of the second dataset in the machine learning is set lower than that of the first dataset. It is composed of: The method of claim 1 .

3. The moving object includes a sensor. the training data includes sensor data obtained by the sensor; A start time of the event in the training data is identified by the sensor data. The model generating method according to claim 1 or 2.

4. The moving object is a vehicle. The model generating method according to claim 1 or 2.

5. The event includes at least one of deceleration of a preceding vehicle, a cut-in of a parallel vehicle, the occurrence of a parked vehicle, the occurrence of an obstacle, and a change in a traffic light. The method of claim 4 .

6. The control command includes at least one of acceleration, deceleration, and steering of the vehicle. The method of claim 4 .

7. 1. A computer-implemented method of data collection, comprising: The data collection method includes: Collecting a plurality of data sets each composed of a combination of training data showing a time series of an environment in which a moving object moves and ground truth data showing a time series of control commands for the moving object in the environment; outputting the collected plurality of datasets for use in machine learning; Including, Collecting the plurality of data sets includes preferentially collecting a data set that is evaluated as more appropriate because a reaction speed to an event of the control command indicated by the correct answer data satisfies a predetermined condition. Data collection methods.

8. The preferential collection of a dataset having a more appropriate reaction rate is configured by maintaining a dataset having a more appropriate reaction rate under the predetermined condition from among the datasets temporarily stored in a storage area of ​​the computer, and deleting a dataset having an inappropriate reaction rate under the predetermined condition. The data collection method according to claim 7.

9. Outputting the plurality of data sets includes: Transmitting a data set in which the reaction rate is evaluated to be appropriate under the predetermined conditions to an external server; and Omitting transmission of a data set for which the reaction rate is evaluated to be inappropriate according to the predetermined condition to the external server; Consists of: The data collection method according to claim 7.

10. On the computer, acquiring target data indicative of an environment in which a target mobile object moves; Deriving control commands from the acquired target data using the trained control model; Controlling the movement of the target moving object according to a result of deriving the control command; A control program for executing the trained control model is generated by performing machine learning using a plurality of data sets each of which is a combination of training data indicating an environment in which a training moving object moves in time series and ground truth data indicating a control command for the training moving object in the environment in time series, The performing of the machine learning includes training the control model for each of the data sets such that a result of deriving a control command for the moving object from the training data using the control model matches the ground truth data; and Using the plurality of data sets for the machine learning includes preferentially using a data set that is evaluated as more appropriate because the reaction speed of the control command to the event indicated by the correct answer data meets a predetermined condition for the machine learning. Control program.

11. The target moving object is a vehicle. The control program according to claim 10.

12. The event includes at least one of deceleration of a preceding vehicle, a cut-in of a parallel vehicle, the occurrence of a parked vehicle, the occurrence of an obstacle, and a change in a traffic light. The control program according to claim 11.

13. The control command includes at least one of acceleration, deceleration, and steering of the vehicle. The control program according to claim 11.

Images