Control device

Abstract

To improve accuracy in performing control of a mobile body that is adapted to a scene.SOLUTION: A control device acquires movement condition data of a mobile body, determines a movement mode in accordance with the acquired movement condition data, selects one trained control model among a plurality of trained control models in accordance with a result determining the movement mode, derives a control command of the mobile body by using the selected trained control model, and controls operation of the mobile body in accordance with the derived control command. Each trained control model is generated by corresponding to a movement mode of the mobile body.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a control technology for a moving object such as an autonomous vehicle. [Background technology]

[0002] Patent Document 1 proposes a system for autonomous vehicle control configured to determine vehicle commands from a route, GPS data, and sensor data using a trained neural network. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2019-533810 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the objectives of the present disclosure is to provide a technique for improving the accuracy of controlling a moving object in accordance with a scene. [Means for solving the problem]

[0005] The control device according to the first aspect of the present disclosure includes a storage unit that stores a plurality of trained control models, and a control unit. Each of the trained control models is generated corresponding to a moving mode of a moving object. The control unit is configured to execute the following: acquiring moving condition data related to a moving condition of the moving object; determining a moving mode according to the acquired moving condition data; selecting one of the trained control models according to a result of determining the moving mode; deriving a control command for the moving object using the selected trained control model; and controlling the operation of the moving object according to the derived control command. Each of the trained control models may be configured by a neural network, and deep learning may be used as a machine learning technique. Effect of the Invention

[0006] According to the present disclosure, it is possible to improve the accuracy of controlling a moving object to suit a scene. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an example of a situation to which the present disclosure is applied. [Diagram 2] FIG. 2 illustrates a schematic diagram of an example of a control model configuration according to the present disclosure. [Diagram 3] FIG. 3 illustrates an example of a hardware configuration of the control device of the present disclosure. [Figure 4] FIG. 4 shows an example of a processing procedure related to control by the control device of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] According to conventional methods such as Patent Document 1, a trained machine learning model (particularly an end-to-end model) can be used to construct an automated driving system. However, the present inventors have found that conventional methods have the following problems. For example, driving behavior can vary greatly depending on the scene, such as lane changing, lane keeping, etc. Furthermore, the occurrence probability of each scene can also vary greatly. In one example, the probability of lane changing occurring in a driving of 20 km or more is 1% or less, which is extremely lower than lane keeping. Furthermore, in lane changing scenes that occur 1% or less, the occurrence probability of avoiding a collision with a nearby vehicle or avoiding an obstacle on the road (stationary object, animal, etc.) is extremely low. Such large differences in driving behavior and occurrence probability Assume that data for learning is collected in a state where scenes are mixed, and a machine learning model is trained with the collected learning data. In this case, it may be difficult for the obtained trained machine learning model to perform vehicle control suitable for the scene with high accuracy. For example, learning of a driving mode with an extremely low probability of occurrence may be insufficient, which may lead to a decrease in the accuracy of vehicle control by the trained machine learning model. This problem may occur regardless of the type of vehicle. Moreover, such problems are not limited to situations in which a vehicle is controlled. In terms of controlling movement, the same is true for moving bodies other than vehicles. Therefore, similar problems may occur in situations in which any moving body other than a vehicle is controlled.

[0009] In contrast, the control device according to the first aspect of the present disclosure includes a storage unit that stores a plurality of trained control models, and a control unit. Each trained control model is generated corresponding to a movement mode of the moving body. The control unit is configured to acquire movement condition data related to a condition of movement of the moving body, determine the movement mode according to the acquired movement condition data, select one of the plurality of trained control models according to a result of determining the movement mode, derive a control command for the moving body using the selected trained control model, and control the operation of the moving body according to the derived control command.

[0010] In the first aspect of the present disclosure, each trained control model is prepared according to the travel mode. That is, each trained control model is prepared exclusively for the corresponding travel mode and is not responsible for travel by other travel modes, so that the machine learning of each control model can suppress the mixing of data of other travel modes (ideally, learning data is collected specially for the corresponding travel mode, and a trained model is generated from the obtained learning data). This can be expected to improve the accuracy of performing control of a moving object that matches the scene.

[0011] As another form of the control device according to the above aspect, one aspect of the present disclosure may be an information processing method or a program for implementing all or part of the above components, or may be a storage medium that stores such a program and is readable by a machine such as a computer. Here, the machine-readable storage medium is a medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action.

[0012] [1 Application example] FIG. 1 is a schematic diagram showing an example of a scene to which the present disclosure is applied. The control device 1 according to this embodiment is one or more computers configured to control the automatic movement of a target moving body M. In this embodiment, the control device 1 is mounted on the moving body M and holds a plurality of trained control models 30. Each trained control model 30 is generated corresponding to a movement mode of the moving body M. That is, each trained control model 30 has acquired the ability to derive a control command for controlling the movement of the moving body M, specialized for the corresponding movement mode.

[0013] In this embodiment, the control device 1 acquires movement condition data 120 related to the conditions of movement of the moving body M. The control device 1 determines a movement mode (running mode, flight mode, navigation mode, etc.) that matches the conditions according to the acquired movement condition data 120. The control device 1 selects one of the multiple trained control models 30 (trained control model 35) according to the result of determining the movement mode. That is, the control device 1 selects a trained control model 35 corresponding to the determined movement mode (determination result of the movement mode) from the multiple trained control models 30. The control device 1 derives a control command for the moving body M in the determined movement mode using the selected trained control model 35. The control device 1 controls the operation of the moving body M according to the derived control command. Note that a series of processes from the acquisition of the movement condition data 120 to the control of the moving body M may be executed in real time.

[0014] In this embodiment, each trained control model 30 is prepared according to the travel mode. That is, since each trained control model 30 is not responsible for travel by a travel mode other than the corresponding travel mode, the mixing of learning data of other travel modes can be suppressed in the machine learning of each control model 30. As a result, according to this embodiment, it is expected that the accuracy of performing control of the moving body M that matches the scene will be improved. In addition, there are cases where the control model 30 is required to be updated by re-learning (additional learning) for some reason, such as specializing in a specific area or adapting to a change in the environment (for example, a road is changed). In contrast, in this embodiment, when a form in which each trained control model 30 is prepared separately is adopted, it is only necessary to re-learn the control model 30 that corresponds to the reason, so that it is not necessary to update the entire control model, and therefore it is expected that the update efficiency will be improved.

[0015] (Mobile) As long as the moving body M can move automatically by mechanical control, the type of the moving body M may be appropriately selected depending on the embodiment. The moving body M may be, for example, a moving device such as a vehicle, an aircraft, a ship, or a robot device. The aircraft may be at least one of an unmanned aircraft such as a drone and a manned aircraft. In one example, as shown in FIG. 1, the moving body M may be a vehicle. Note that the type of the vehicle (number of wheels, power source, size, etc.) may be selected arbitrarily. As a typical example, the moving body M may be an automobile capable of level 2 or higher autonomous driving.

[0016] (Controlling the movement) In one example, controlling the operation of the target moving body M may be configured by directly controlling the target moving body M. In another example, the moving body M may be equipped with a dedicated control device such as a controller. In this case, controlling the operation of the target moving body M by the control device 1 may be configured by indirectly controlling the target moving body M by providing the derived result to the dedicated control device. The control device 1 may be deployed at any location. In one example, as shown in FIG. 1, the control device 1 may be mounted on the moving body M. In another example, the control device 1 may be disposed away from the moving body M and remotely control the moving body M. The control device 1 may be configured to be able to switch the moving body M from an automatic control mode to a manual control mode and from the manual control mode to the automatic control mode at any timing by any operation.

[0017] (Control Model) The control model 30 (35) is configured by a machine learning model having one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used for the calculation of the desired inference (in this case, derivation of a control command). Machine learning is to adjust (optimize) the value of the calculation parameter using learning data. The configuration and type of the machine learning model are not particularly limited and may be appropriately selected depending on the embodiment. The machine learning model may be configured by, for example, a neural network, a support vector machine, a regression model, a decision tree model, etc. The machine learning method may be appropriately selected depending on the machine learning model to be adopted (for example, backpropagation method, etc.). The machine learning may include supervised learning, unsupervised learning, and reinforcement learning. In one example, at least one of the multiple control models 30 may be configured at least partially by a neural network. The structure of the neural network may be appropriately determined depending on the embodiment. The structure of the neural network may be specified by, for example, the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationship between the nodes in each layer, etc. In one example, the neural network may have a recurrent structure. The neural network may include any layer, such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, an unpooling layer, a normalization layer, a dropout layer, or a Long short-term memory (LSTM). The neural network may have any mechanism, such as an attention mechanism. The network can be a GNN (Graph neural network), a diffusion model, or a generative model (e.g. Any model may be included, such as Generative Adversarial Network, Transformer, etc. When a neural network is used for the control model, the weight of the connection between each node included in the control model and the threshold value of each node are examples of the calculation parameters. When a machine learning model is adopted, the control model may be configured in the structure of an end-to-end model.

[0018] Each control model 30 is constructed to derive a control command according to the environment of the moving body M. The environment is an event observed by at least one of the moving body M itself and the surroundings. In one example, at least a part of the environment may be observed by one or more sensors S arranged inside or outside the moving body M. The type of the sensor S is not particularly limited as long as it can observe any environment in which the moving body M moves, and may be appropriately selected depending on the embodiment. In one example, the one or more sensors S may include an image sensor (camera), radar, LiDAR (Light Detection And Ranging), sonar (ultrasonic sensor), infrared sensor, GNSS (Global Navigation Satellite System) / GPS (Global Positioning Satellite) module, etc. The sensor S may be appropriately positioned to observe any direction such as the front, right side, left side, rear, etc. of the moving body M. When the moving body M is a vehicle, the sensor S may include at least one of an electronic inner mirror (EIM), a panoramic view monitor (PVM), and a millimeter wave radar. The electronic inner mirror may be composed of an image sensor installed at the rear of the vehicle. The panoramic view monitor may be composed of image sensors installed at the front, each side, and rear of the vehicle. The multiple millimeter wave radars may be installed on each side of the front and each side of the rear of the vehicle, respectively.

[0019] As long as the control command can be derived from the environment of the moving body M, the input / output format of each control model 30 may be appropriately selected according to the embodiment. In one example, at least one of the multiple control models 30 may be configured to derive the control command from observation data of the sensor S at one or more points in time. For example, the control model 30 may be configured to perform surrounding recognition, path planning (route / trajectory planning), and motion planning (motion / control planning) (end-to-end model). In another example, at least one of the multiple control models 30 may be configured to derive the control command from the recognition result of the surrounding environment. In this case, the control device 1 may further include an analysis model that infers the recognition result of the surrounding environment from the observation data of the sensor S. Alternatively, at least one of the multiple control models 30 may include such an analysis model. The analysis model may be appropriately configured. In one example, the analysis model may be configured by a machine learning model. In addition, other information may be added arbitrarily to the input of at least one of the multiple control models 30. At least one of the plurality of control models 30 may be configured to further receive input of optional information, such as a set speed, a speed limit, driving data, map data, navigation information (route data), and the like.

[0020] Typically, the multiple trained control models 30 may be configured separately (independently). However, the configuration of the multiple trained control models 30 may not be limited to this example. In one example of this embodiment, two or more control models among the multiple trained control models 30 may be configured at least partially integrally. For example, one model may include n output parts that derive control commands from the analysis model and the output of the analysis model arranged on the input side (n is a natural number of 2 or more). This one model may be considered as n trained control models 30. That is, in this embodiment, even if the structure of a model is integral, if the model includes n output parts configured to output control commands, holding the model may be considered as holding n trained control models 30. In this case, since the output parts are separate, the n trained control models 30 may be considered to be configured separately. In addition, a conditional model configured to output an output according to a given condition may be treated in the same manner. That is, the plurality of trained control models 30 may include a conditional model configured to output a control command according to an input condition (class / category). In this case, the condition given to the conditional model is changed and the calculation of the conditional model is repeated to obtain a control command according to the condition given to the conditional model. Thus, if one conditional model is configured to correspond to n conditions, having this one conditional model may be considered as having n trained control models 30.

[0021] (Control command) The control command relates to the operation of the moving body M. The configuration of the control command may be appropriately selected depending on the embodiment. In one example, the control command may be configured by acceleration, deceleration, steering, or a combination of these. Acceleration and deceleration may include gear changes. When at least one of acceleration, deceleration, and steering is included, the control command may be expressed by a path. Accordingly, the control model 30 may be expressed as a path planner. The control command may further include a command related to the operation of the moving body M. As an example, when the moving body M is a vehicle, the control command may include vehicle operations such as turn signals, hazards, horn, and communication processing (e.g., transmitting data to a center, making an emergency call, etc.).

[0022] Each control model 30 may be configured to directly output a control command, or may be configured to indirectly output a control command. In the latter case, the control command may be obtained by executing any information processing (interpretation processing) on ​​the output of the control model 30. The control command may be configured to directly indicate the control amount (control instruction value, control output amount) of the moving object M, such as, for example, an accelerator control amount, a brake control amount, or a steering wheel steering angle. Alternatively, the control command may be configured to indirectly indicate the control amount of the moving object M, such as, for example, a path, a state after control, or the like. In this case, the control amount of the moving object M may be obtained from the control command by executing any information processing.

[0023] In this embodiment, typically, the control device 1 may execute only the calculation process of the selected trained control model 35 to obtain a control command. In response to this, the control device 1 may stop the operation of at least a part of the sensors S provided on the moving body M that are not used in the selected trained control model 35 and are used in other trained control models. This is expected to reduce the calculation load and power consumption. However, the process of obtaining a control command is not limited to this example. In another example, the control device 1 may execute the calculation process of one or more trained control models that are likely to be used among the multiple trained control models 30, and obtain one or more control commands. In this case, the selection of the trained control model 35 may be configured by selecting a control command (a control command to be used for control) derived from the trained control model 35 from one or more control commands. This can improve responsiveness to switching of the travel mode. The one or more trained control models that are likely to be used may be selected from the multiple trained control models 30 by any method. In one example, one or more trained control models to be used to derive control commands may be selected depending on a travel scene, such as a location, a route, a type of passage (in the case of a vehicle, e.g., a highway, a general road, etc.), etc.

[0024] (Movement mode) The travel mode is defined so as to be able to prescribe a series of travel patterns in accordance with a specific purpose, such as lane change, lane keeping, etc. The purpose may be set appropriately for each travel mode depending on the type of the moving body M. In one example, when the moving body M is a vehicle, the travel mode may be lane change, lane keeping, emergency stop (EDSS: Emergency Driving Stop System), yielding to oncoming traffic, automatic parking, or any of these. The above-mentioned terms may include combinations of the above. Lane change refers to changing the lane in which the vehicle is traveling. Lane keeping refers to continuing to travel in the same lane. Emergency stop refers to pulling over to the shoulder of the road and stopping the vehicle. Yielding to merging refers to giving way to other vehicles merging into the lane in which the vehicle is traveling. Automatic parking refers to parking the vehicle by automatic control.

[0025] For example, the travel modes may include lane changing, lane keeping, and emergency stopping. , the plurality of trained control models 30 may include a first trained control model corresponding to lane change, a second trained control model corresponding to lane keeping, and a third trained control model corresponding to emergency stop. Determining the travel mode may include selecting one of the modes of lane change, lane keeping, and emergency stop. In one example of the present embodiment, in a vehicle having these three modes, it is expected that the accuracy of performing control suited to the scene can be improved. However, the types of travel modes are not limited to these, and the travel modes corresponding to each control model may be set arbitrarily. The travel mode may be interpreted according to the type of the moving body M. For example, when the moving body M is a vehicle, the travel mode may also be referred to as a traveling mode. When the moving body M is an aircraft, the travel mode may be referred to as a flight mode. When the moving body M is a ship, the travel mode may be referred to as a navigation mode.

[0026] The method of determining the travel mode from the travel condition data 120 is not particularly limited and may be appropriately selected depending on the embodiment. In one example, a computation model may be used to determine the travel mode. The computation model may be configured, for example, by a trained machine learning model, a rule-based model, or a combination of these. The machine learning model may be configured similarly to the control model. The rule-based model is configured to match a given input (travel condition data 120) with a rule, and derive a travel mode determination result depending on the result of the match (according to the matching rule). The rule may be set manually or at least partially automatically.

[0027] (Generated according to the travel mode) In this embodiment, each trained control model 30 is generated corresponding to a travel mode. The generation method may be appropriately selected according to the embodiment. As an example of the generation method, training data (input data to be given to the control model) may be collected corresponding to each travel mode. Correct answer data (teacher signal, label) may be appropriately added to the collected training data. In this way, a plurality of data sets each composed of a combination of training data and correct answer data can be obtained. Each trained control model 30 may be generated by performing machine learning using a plurality of data sets collected according to the corresponding travel mode. That is, the control model 30 may be trained so that an inference result (a derivation result of a control command) obtained by providing the training data of each data set obtained according to the corresponding travel mode to the control model 30 matches the corresponding correct answer data. Training may be adjusting the calculation parameters of the control model 30. Through this machine learning, each trained control model 30 that has acquired the ability to control the movement of the moving body M in the corresponding travel mode can be generated. In this embodiment, at least one of the plurality of trained control models 30 is generated according to a travel mode different from the travel modes of the other trained control models.

[0028] (Movement condition data) The travel condition data 120 relates to the conditions of the movement of the moving body M at the time (current time) when the travel mode is determined. The travel condition data 120 may be composed of any type of data that can be used as an explanatory variable in determining the travel mode. In one example, the travel condition data 120 may include position data (e.g., measurement data of a GNSS / GPS module) and route data. The route data may be configured to indicate a route from a current location to a destination. The route data may be appropriately acquired from a navigation device. The navigation device may be included in the control device 1, or may be provided outside the control device 1 and may be appropriately connected to the control device 1. The current location may be a location indicated by position data acquired at the current time. The position data may be composed of sensor data acquired by a positioning module such as a GNSS / GPS module. In response to this, the sensor S may include a positioning module. The travel condition data 120 may further include map data. The type of map data may be arbitrary. In one example, high-precision map data may be used as the map data. The map data may be acquired from a navigation device, or may be held in the control device 1. In addition, location data The route data and map data may be collectively referred to as navigation data.

[0029] Moreover, when the moving body M is a vehicle, the moving condition data 120 may further include at least one of driving data and reaction data. The driving data may be configured by data indicating a driving state such as a speed and a steering angle. The driving data may be appropriately acquired from an on-board sensor provided in the vehicle. When adopting a form including the driving data, the sensor S may include an on-board sensor. The reaction data is configured to indicate a reaction of an occupant OU riding in the vehicle. The reaction data may be configured by data capable of identifying a reaction of the occupant OU to the driving such as an image, a voice, and operation information via an input device. A monitoring device MD may be used to acquire the reaction data. The monitoring device MD may be, for example, an in-vehicle camera, a microphone, an input device (operation button), etc. The occupant OU may be, for example, a driver, a passenger other than the driver, etc. By adopting a form including the reaction data, the feedback of the occupant OU can be reflected in the control. As a result, a control suited to the occupant OU can be expected. Note that data other than the above (other sensor data) may also be used to determine the travel mode.

[0030] (optimization) The input of each trained control model 30 may be appropriately selected according to the embodiment. In one example, sensor data may be commonly input to each trained control model 30. However, depending on the travel mode, there may be sensor data that contributes less to the derivation of a control command (i.e., it is redundant). Therefore, in another example, the input of each trained control model 30 may be optimized according to the corresponding travel mode. Being optimized may mean that the input of at least one of the multiple control models (control models 30) is less than the input of the other control models, thereby reducing the input to the control model. Being optimized may include that the input is redundant in at least a part of the multiple control models, as long as the input is reduced.

[0031] FIG. 2 is a schematic diagram showing an example of a control model configuration according to this embodiment. In the example shown in FIG. 2, the moving body M is a vehicle. The sensor S includes a plurality of sensors arranged in the front, on each side, and behind the moving body M, and acquires surrounding information (sensor data). Each sensor is, for example, a GPS module. The travel condition data 120 includes position data, The data includes surrounding information (sensor data), driving data, navigation data (position data, map data, route data), and reaction data. The multiple trained control models 30 include a lane change model 301, a lane keeping model 302, and an emergency stop model 303. The lane change model 301 is an example of a first trained control model, the lane keeping model 302 is an example of a second trained control model, and the emergency stop model 303 is an example of a third trained control model. Any of the lane change, lane keeping, and emergency stop movement modes (driving modes) may be selected according to the movement condition data 120. Any of the lane change model 301, lane keeping model 302, and emergency stop model 303 may be selected as a model to be used for control according to the selected movement mode.

[0032] In the example of FIG. 2, driving data and navigation data are commonly input to each model (301, 302, 303). On the other hand, front, side, and rear sensor data are input to the lane change model 301 and the emergency stop model 303, whereas front and side sensor data are input to the lane keeping model 302. That is, rear sensor data is omitted from the input of the lane keeping model 302. This omission is an example of optimization. When the input of the control model 30 is optimized, a sensor may be generated that is not used by the control model 35 selected depending on the travel mode. The control device 1 may stop the operation of the sensor that is not used as appropriate. In the example of FIG. 2, while the lane keeping travel mode is selected, the control device 1 may stop the operation of the rear sensor.

[0033] By optimizing the input of each trained control model 30, it is possible to expect an improvement in the computation efficiency. For example, it is possible to expect at least one of the following: reduction in power consumption by suppressing the activation of unnecessary sensors, reduction in the sensor failure rate by making the activation time of the sensors more efficient, reduction in the computation load / computation amount by optimizing the models, thereby improving computation efficiency, and improvement in the accuracy of model computation by reducing unnecessary inputs. Note that the form of the control model configuration and optimization is not limited to the example of FIG. 2, and may be appropriately selected according to the embodiment. In another example, the multiple trained control models 30 may further include a trained control model (automatic parking model) corresponding to automatic parking. The sensor S may include a clearance sonar. In this case, the sensor data of the clearance sonar may be input to the automatic parking model, but may be omitted from the input of the other models. In response to this, the control device 1 may stop the operation of the clearance sonar while the other models are being used, and start it when the automatic parking model is being used.

[0034] (strength) In one example, determining the travel mode may include calculating an intensity. Deriving the control command may be configured by deriving the control command according to the calculated intensity. The intensity may be appropriately configured to indicate the degree to which the execution of the travel in the target travel mode is forced. Each trained control model 30 may be configured to derive a control command such that the higher the intensity, the more the travel in the travel mode is forced to be executed. For example, the higher the intensity, the smaller the limit of the execution parameter may be. As a specific example, in the case of a lane change, the higher the intensity, the higher the degree to which the lane change is forced may be increased by executing the lane change at an earlier timing, executing the lane change in a narrow space, executing the lane change with stronger acceleration and deceleration, etc. According to one example of the present embodiment, the degree to which the travel in each travel mode is executed can be controlled.

[0035] [2 Configuration example] 3 is a schematic diagram showing an example of a hardware configuration of the control device 1 according to the present embodiment. The control device 1 according to the present embodiment is a computer to which a control unit 11, a storage unit 12, an external interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0036] The control unit 11 includes a CPU (Central Processing Unit), a RAM (Random Access Memory ), ROM (Read Only Memory), etc., and is configured to execute any information processing based on the program and various data. The control unit 11 (CPU) is an example of a processor resource. The memory unit 12 may be configured with any storage device, such as a hard disk drive, a solid state drive, etc. The memory unit 12 (and RAM, ROM) is an example of a memory unit of the present disclosure. In addition, the memory unit 12 (and RAM, ROM) is an example of a memory resource. In this embodiment, the memory unit 12 stores various information such as a control program 81, learning result data 300, etc.

[0037] The control program 81 is a program for causing the control device 1 to execute information processing (FIG. 4 described later) related to the control of the moving object M. The control program 81 includes a series of instructions for the information processing. The learning result data 300 is configured to indicate information related to the trained control model 30. In one example, the learning result data 300 may be provided for each trained control model 30. In another example, one learning result data 300 may be configured to indicate two or more trained control models 30. Furthermore, as long as the trained control model 30 can be reproduced, the data configuration of the learning result data 300 may be appropriately determined according to the embodiment. In one example, the learning result data 300 may include information indicating values ​​of calculation parameters of the control model 30 adjusted by machine learning. The learning result data 300 may further include information indicating the configuration of the machine learning model (for example, the structure of a neural network, etc.).

[0038] The external interface 13 may be, for example, a Universal Serial Bus (USB) port, a dedicated port, a wireless communication port, etc., and is configured to connect to an external device by wire or wirelessly. In this embodiment, the control device 1 may be connected to the sensor S and the monitoring device MD via the external interface 13. The input device 14 is, for example, a device for inputting a mouse, a keyboard, an operator, etc. The output device 15 is, for example, a device for outputting a display, a speaker, etc. The input device 14 and the output device 15 may be integrally configured by, for example, a touch panel display, etc.

[0039] The drive 16 is a device for reading various information such as a program stored in a storage medium 91. At least one of the control program 81 and the learning result data 300 may be stored in the storage medium 91 instead of or together with the storage unit 12. The storage medium 91 is configured to accumulate various information (stored programs, etc.) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The control device 1 may acquire at least one of the control program 81 and the learning result data 300 from the storage medium 91. The storage medium 91 may be a disk-type storage medium such as a CD or DVD, or may be a non-disk-type storage medium such as a semiconductor memory (e.g., a flash memory). The type of the drive 16 may be appropriately selected according to the type of the storage medium 91.

[0040] In addition, regarding the specific hardware configuration of the control device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), an ECU, (Electronic Control Unit), GPU (Graphics Processing Unit), etc. At least one of the external interface 13, the input device 14, the output device 15, and the drive 16 may be omitted. The control device 1 may be provided with a communication interface and configured to perform data communication with an external computer. At least one of the input device 14, the output device 15, and the drive 16 may be connected via an external interface or a communication interface. The control device 1 may be a computer designed specifically for the service provided, as well as a general-purpose computer, a terminal device, etc. When the moving body M is a vehicle, the control device 1 may be an in-vehicle device.

[0041] [3 Example of operation] FIG. 4 shows an example of a processing procedure for controlling a moving body M by the control device 1 according to this embodiment. The control unit 11 of the control device 1 executes instructions included in a control program 81 by means of a CPU. As a result, the control device 1 operates as a computer capable of executing the following information processing. The following processing procedure is an example of a control method executed by a computer. However, the following processing procedure is merely an example, and each step may be modified as much as possible. Furthermore, steps of the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0042] In step S101, the control unit 11 acquires the travel condition data 120. The control unit 11 may acquire the travel condition data 120 directly or indirectly from an information source such as a sensor S. In one example, when the moving body M is a vehicle, the travel condition data 120 may include reaction data indicating the reaction of an occupant OU riding in the vehicle. The reaction data may be acquired directly or indirectly from the monitoring device MD.

[0043] In step S102, the control unit 11 determines a travel mode that matches the travel conditions according to the acquired travel condition data 120. In one example, determining the travel mode may include calculating the strength. Also, in the example of FIG. 2, determining the travel mode may include selecting one of lane change, lane keeping and emergency stop modes.

[0044] In step S103, the control unit 11 selects a trained control model 35 to be used for control from among the multiple trained control models 30 according to the result of determining the travel mode. In one example, the input of each trained control model 30 may be optimized according to the corresponding travel mode. In response to this, the control unit 11 may start the operation of a sensor to be used among the group of sensors included in the sensor S and stop the operation of a sensor not to be used according to the selection result at any timing.

[0045] In step S104, the control unit 11 derives a control command using the selected trained control model 35. The control unit 11 may appropriately acquire data to be provided to the trained control model 35 directly or indirectly from the sensor S or the like at any timing. The control unit 11 may input the acquired data to the trained control model 35 and execute arithmetic processing of the trained control model 35. In this way, the control unit 11 may acquire the derivation result of the control command from the trained control model 35. When the above-mentioned form of calculating the strength is adopted, deriving the control command may be configured by deriving the control command according to the calculated strength.

[0046] In step S105, the control unit 11, in accordance with the derived control command, controls the operation (movement) of the moving object M. When the control of the target moving object M is completed, the control unit 11 ends the processing procedure of the control device 1 according to this operation example.

[0047] The control unit 11 may repeatedly execute a series of information processing from step S101 to step S105 at any timing. In one example, the control unit 11 may repeatedly execute a series of information processing from step S101 to step S105 for a predetermined period (for example, while the power source of the moving object M is started and the automatic control mode is selected). In this way, the control device 1 may continuously execute the automatic control of the moving object M.

[0048] [Features] In this embodiment, the trained control model 35 that matches the movement conditions is selected from among the multiple trained control models 30 by the processes of steps S101 to S103. Then, the selected trained control model 35 is used to control the moving object M by the processes of steps S104 and S105. Since each trained control model 30 is not responsible for movement by a movement mode other than the corresponding movement mode, the mixing of learning data of other movement modes can be suppressed in the machine learning of each control model 30. With this specialized configuration, it can be expected that each control model 30 has acquired a high level of ability to control the movement of the moving object M in the corresponding movement mode in the machine learning. As a result, in step S105, the accuracy of performing control of the moving object M that matches the scene can be improved.

[0049] [4 Variations] Although the embodiments of the present disclosure have been described in detail above, the above description is merely an example of the present disclosure in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present disclosure. In addition, the processes and means described in the present disclosure can be freely combined and implemented as long as no technical contradiction occurs. [Explanation of symbols]

[0050] 1...Control device, 11: control unit; 12: storage unit; 120: movement condition data; 30·35…trained control model, 301...Lane change model (first trained control model), 302…Lane keeping model (second trained control model), 303…emergency stop model (third trained control model), M: moving object, OU: occupant, S: sensor, MD: monitoring device

Claims

1. a storage unit for storing a plurality of trained control models; and Control unit, A control device comprising: Each of the trained control models is generated corresponding to a moving mode of a moving object; and The control unit is acquiring movement condition data relating to conditions of movement of the moving object; determining a travel mode according to the acquired travel condition data; selecting one of the plurality of trained control models according to a result of determining the movement mode; deriving control commands for the moving object using the selected trained control model; and Controlling an operation of the moving object in accordance with the derived control command; configured to execute Control device.

2. Determining the mode of movement includes assessing an intensity; deriving the control command comprises deriving the control command in response to the calculated intensity. The control device according to claim 1 .

3. the moving object is a vehicle, the plurality of trained control models include a first trained control model corresponding to lane changing, a second trained control model corresponding to lane keeping, and a third trained control model corresponding to an emergency stop; Determining the travel mode includes selecting one of the lane change mode, the lane keeping mode, and the emergency stop mode. The control device according to claim 1 .

4. The acquired travel condition data includes reaction data indicating reactions of occupants riding in the vehicle. The control device according to claim 3.

5. The input of each of the trained control models is optimized according to the corresponding movement mode. A control device according to any one of claims 1 to 4.

Images