control device

Abstract

To improve accuracy in performing control of a mobile body that is adapted to a scene.SOLUTION: A control device acquires movement condition data of a mobile body, determines a movement mode in accordance with the acquired movement condition data, selects one trained control model among a plurality of trained control models in accordance with a result determining the movement mode, derives a control command of the mobile body by using the selected trained control model, and controls operation of the mobile body in accordance with the derived control command. Each trained control model is generated by corresponding to a movement mode of the mobile body.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to control technologies for moving bodies such as autonomous vehicles.

Background Art

[0002] Patent Document 1 proposes a system for autonomous vehicle control configured to determine vehicle commands from routes, GPS data, and sensor data using a trained neural network.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One object of the present disclosure is to provide a technique for improving the accuracy of performing control of a moving body that conforms to a scene.

Means for Solving the Problems

[0005] A control device according to a first aspect of this disclosure comprises a storage unit for storing a plurality of trained control models and a control unit. Each of the trained control models is generated in correspondence with the movement mode of a moving object. The control unit is configured to acquire movement condition data relating to the conditions of the movement of the moving object, determine the movement mode according to the acquired movement condition data, select one of the plurality of trained control models according to the result of determining the movement mode, derive a control command for the moving object using the selected trained control model, and control the operation of the moving object according to the derived control command. Each trained control model may be composed of a neural network, and deep learning may be used as the machine learning method. [Effects of the Invention]

[0006] According to this disclosure, it is possible to improve the accuracy of controlling a moving object that is appropriate for the scene. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 schematically illustrates an example of a scenario in which this disclosure applies. [Figure 2] Figure 2 schematically shows an example of the control model configuration of this disclosure. [Figure 3] Figure 3 schematically shows an example of the hardware configuration of the control device of this disclosure. [Figure 4] Figure 4 shows an example of a processing procedure related to control by the control device of this disclosure. [Modes for carrying out the invention]

[0008] According to conventional methods such as those described in Patent Document 1, an autonomous driving system can be constructed by using a trained machine learning model (especially an end-to-end model). However, the inventors of this invention have found that conventional methods have the following problems. For example, the manner of driving can vary greatly depending on the situation, such as lane changes and lane keeping. Also, the probability of each situation occurring can vary greatly. For example, the probability of a lane change occurring when driving for more than 20 km is less than 1%, which is extremely low compared to lane keeping. Furthermore, in lane change scenarios that occur less than 1%, the probability of avoiding a collision with a surrounding vehicle or avoiding an obstacle on the road (stationary object, animal, etc.) is extremely low. Such driving manners and probabilities of occurrence vary greatly. Let's consider a scenario where training data is collected in a mixed-scene environment, and a machine learning model is trained using this collected data. In this case, it may be difficult for the resulting trained machine learning model to accurately control a vehicle appropriate to the scene. For example, extremely rare driving patterns may not be adequately trained, potentially leading to a decrease in the accuracy of vehicle control by the trained machine learning model. This problem can occur regardless of the type of vehicle. Furthermore, this problem is not limited to vehicle control. The same issue applies to other moving objects besides vehicles. Therefore, similar problems can occur when controlling any moving object other than a vehicle.

[0009] In contrast, the control device according to the first aspect of the present disclosure comprises a storage unit for storing a plurality of trained control models, and a control unit. Each trained control model is generated corresponding to the movement mode of a mobile body. The control unit is configured to acquire movement condition data relating to the conditions of the movement of the mobile body, determine the movement mode according to the acquired movement condition data, select one of the plurality of trained control models according to the result of determining the movement mode, derive a control command for the mobile body using the selected trained control model, and control the operation of the mobile body according to the derived control command.

[0010] In the first aspect of this disclosure, each trained control model is prepared according to the mode of movement. That is, each trained control model is prepared exclusively for its corresponding mode of movement and does not handle movement in other modes, so that the mixing of data from other modes of movement can be suppressed in the machine learning of each control model (ideally, training data is collected exclusively for the corresponding mode of movement and the trained model is generated with the obtained training data). This can be expected to improve the accuracy of performing control of a moving object that is appropriate for the scene.

[0011] Furthermore, as another form of the control device according to the above embodiment, one aspect of this disclosure may be an information processing method that implements all or part of the above components, a program, or a machine-readable storage medium that stores such a program. Here, a machine-readable storage medium is a medium that stores information such as programs by electrical, magnetic, optical, mechanical, or chemical action.

[0012] [1. Application Examples] Figure 1 schematically shows an example of a scenario in which the present disclosure is applied. The control device 1 according to this embodiment is one or more computers configured to control the automatic movement of a target mobile object M. In this embodiment, the control device 1 is mounted on the mobile object M and holds a plurality of trained control models 30. Each trained control model 30 is generated in accordance with the movement mode of the mobile object M. That is, each trained control model 30 is specialized for its corresponding movement mode and has acquired the ability to derive control commands for controlling the movement of the mobile object M.

[0013] In this embodiment, the control device 1 acquires movement condition data 120 related to the conditions for the movement of the mobile body M. The control device 1 determines a movement mode (driving mode, flight mode, navigation mode, etc.) that fits the conditions according to the acquired movement condition data 120. The control device 1 selects one of a plurality of trained control models 30 (trained control model 35) according to the result of determining the movement mode. That is, the control device 1 selects a trained control model 35 from the plurality of trained control models 30 that corresponds to the determined movement mode (result of movement mode determination). The control device 1 uses the selected trained control model 35 to derive control commands for the mobile body M in the determined movement mode. The control device 1 controls the operation of the mobile body M according to the derived control commands. Note that the series of processes from acquiring the movement condition data 120 to controlling the mobile body M may be executed in real time.

[0014] In this embodiment, each trained control model 30 is prepared according to the mode of movement. That is, each trained control model 30 does not handle movement in modes other than its corresponding mode, so the mixing of training data from other modes of movement can be suppressed in the machine learning of each control model 30. As a result, according to this embodiment, an improvement in the accuracy of performing control of the mobile body M that is appropriate for the scene can be expected. In addition, there are cases where the control model 30 needs to be updated by retraining (additional training) for some reason, such as specializing in a specific region or adapting to environmental changes (for example, roads are changed). In contrast, in this embodiment, if each trained control model 30 is prepared separately, only the control model 30 corresponding to that reason needs to be retrained, so the entire system does not need to be updated, and an improvement in update efficiency can be expected.

[0015] (Mobile) The type of mobile body M can be appropriately selected depending on the embodiment, as long as it can be moved automatically by mechanical control. The mobile body M may be, for example, a mobile device such as a vehicle, an aircraft, a ship, or a robotic device. The aircraft may be at least one of an unmanned aircraft such as a drone or a manned aircraft. In one example, as shown in Figure 1, the mobile body M may be a vehicle. The type of vehicle (number of wheels, power source, size, etc.) may be arbitrarily selected. As a typical example, the mobile body M may be an automobile with Level 2 or higher autonomous driving capability.

[0016] (Controlling actions) In one example, controlling the movement of the target mobile object M may be done by directly controlling the target mobile object M. In another example, the mobile object M may be equipped with a dedicated control device, such as a controller. In this case, controlling the movement of the target mobile object M by the control device 1 may be done by indirectly controlling the target mobile object M by providing the dedicated control device with a derived result. The control device 1 may be deployed in any location. In one example, as shown in Figure 1, the control device 1 may be mounted on the mobile object M. In another example, the control device 1 may be positioned away from the mobile object M and remotely control the mobile object M. The control device 1 may be configured to switch between the automatic control mode and manual control mode of the mobile object M at any timing by any operation.

[0017] (Control model) The control model 30(35) is composed of a machine learning model having one or more arithmetic parameters adjustable by machine learning. The one or more arithmetic parameters are used for the arithmetic operation of the target inference (in this case, derivation of control commands). Machine learning is to adjust (optimize) the values of the arithmetic parameters using learning data. The configuration and type of the machine learning model do not have to be particularly limited, and may be appropriately selected according to the embodiment. The machine learning model may be composed of, for example, a neural network, a support vector machine, a regression model, a decision tree model, etc. The method of machine learning may be appropriately selected according to the machine learning model adopted (for example, the error backpropagation method, etc.). Machine learning may include supervised learning, unsupervised learning, and reinforcement learning. In one example, at least any one of the plurality of control models 30 may be at least partially composed of a neural network. The structure of the neural network may be appropriately determined according to the embodiment. The structure of the neural network may be specified by, for example, the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationship between the nodes in each layer, etc. In one example, the neural network may have a recursive structure. Also, the neural network may include any layer such as a fully connected layer, a convolutional layer, a pooling layer, a transposed convolutional layer, an unpooling layer, a normalization layer, a dropout layer, LSTM (Long short-term memory), etc. The neural network may have any mechanism such as an Attention mechanism. Neural network may include any model such as GNN (Graph neural network), diffusion model, generative model (e.g for example, Generative Adversarial Network, Transformer, etc.). When using a neural network as the control model, the connection weights between each node included in the control model and the threshold value of each node are an example of the arithmetic parameters. When adopting a machine learning model, the control model may be configured with an end-to-end model structure.

[0018] Each control model 30 is constructed to derive control commands according to the environment of the mobile body M. The environment is the events observed in at least one of the mobile body M itself and its surroundings. In one example, at least a portion of the environment may be observed by one or more sensors S located inside or outside the mobile body M. The type of sensor S is not particularly limited as long as it can observe any environment in which the mobile body M is moving, and may be appropriately selected depending on the embodiment. In one example, one or more sensors S may include an image sensor (camera), radar, LiDAR (Light Detection and Ranging), sonar (ultrasonic sensor), infrared sensor, GNSS (Global Navigation Satellite System) / GPS (Global Positioning Satellite) module, etc. The sensors may be appropriately positioned to observe any direction of the moving object M, such as the front, right, left, or rear. If the moving object M is a vehicle, the sensors S may include at least one of an electronic rearview mirror (EIM), a panoramic view monitor (PVM), and millimeter-wave radar. The electronic rearview mirror may consist of an image sensor installed at the rear of the vehicle. The panoramic view monitor may consist of image sensors installed at the front, each side, and the rear of the vehicle. Multiple millimeter-wave radars may be installed at each side of the front and each side of the rear of the vehicle.

[0019] If it is possible to derive a control command from the environment of the mobile body M, the input / output format of each control model 30 may be appropriately selected according to the embodiment. In one example, at least any one of the plurality of control models 30 may be configured to derive a control command from the observation data of the sensor S at one or more time points. For example, the control model 30 may be configured to execute surrounding recognition, path planning (route / trajectory planning), and motion planning (operation / control planning) (end-to-end model). In another example, at least any one of the plurality of control models 30 may be configured to derive a control command from the recognition result of the surrounding environment. In this case, the control device 1 may further include an analysis model that infers the recognition result of the surrounding environment from the observation data of the sensor S. Alternatively, at least any one of the plurality of control models 30 may include such an analysis model. The analysis model may be configured as appropriate. In one example, the analysis model may be configured by a machine learning model. In addition, any other information may be arbitrarily added to the input of at least any one of the plurality of control models 30. At least any one of the plurality of control models 30 may be configured to further receive an input of arbitrary information such as, for example, a set speed, a limit speed, running data, map data, navigation information (route data), etc.

[0020] Typically, multiple trained control models 30 may be configured separately (independently). However, the configuration of multiple trained control models 30 is not limited to this example. In one example of this embodiment, two or more control models among the multiple trained control models 30 may be configured at least partially as a single unit. For example, one model may have an analysis model placed on the input side and n output parts that derive control commands from the output of the analysis model (n is a natural number of 2 or more). This one model may be considered as n trained control models 30. That is, in this embodiment, even if the structure of the model is integrated, if the model has n output parts configured to output control commands, then holding the model may be considered as holding n trained control models 30. In this case, since the output parts are separate, the n trained control models 30 may be considered as being configured separately. Conditional models configured to produce an output according to given conditions may be treated similarly. In other words, the multiple trained control models 30 may include a conditional model configured to output a control command according to the input condition (class / category). In this case, by changing the conditions given to the conditional model and repeating the calculation of the conditional model, each given Multiple control commands can be derived depending on the conditions. Therefore, if one conditional model is configured to correspond to n conditions, then holding this one conditional model can be considered equivalent to holding n trained control models 30.

[0021] (Control command) The control commands relate to the operation of the mobile body M. The configuration of the control commands may be appropriately selected depending on the embodiment. In one example, the control commands may consist of acceleration, deceleration, steering, or a combination thereof. Acceleration and deceleration may include gear changes. If at least one of acceleration, deceleration, and steering is included, the control commands may be expressed as paths. Accordingly, the control model 30 may be expressed as a path planner. The control commands may also further include commands relating to the operation of the mobile body M. For example, if the mobile body M is a vehicle, the control commands may include vehicle operations such as turn signals, hazard lights, horn, and communication processing (e.g., sending data to a center, making an emergency call, etc.).

[0022] Each control model 30 may be configured to output a control command directly, or it may be configured to output a control command indirectly. In the latter case, a control command may be obtained by performing arbitrary information processing (interpretation processing) on ​​the output of the control model 30. The control command may be configured to directly indicate a control amount (control instruction value, control output amount) of the moving body M, such as an accelerator control amount, a brake control amount, or a steering angle. Alternatively, the control command may be configured to indirectly indicate a control amount of the moving body M, such as a path or a state after control. In this case, the control amount of the moving body M may be obtained from the control command by performing arbitrary information processing.

[0023] In this embodiment, typically, the control device 1 may execute only the computational processing for the selected trained control model 35 and obtain control commands. Accordingly, the control device 1 may stop the operation of at least some of the sensors S provided on the mobile body M that are not used by the selected trained control model 35 but are used by other trained control models. This is expected to reduce computational load and power consumption. However, the process of obtaining control commands is not limited to this example. In another example, the control device 1 may execute the computational processing for one or more trained control models 30 that are likely to be used and obtain one or more control commands. In this case, selecting a trained control model 35 may be done by selecting one or more control commands derived from the trained control model 35 (control commands used for control). This is possible to improve responsiveness to switching of movement modes. The one or more trained control models that are likely to be used may be selected from the multiple trained control models 30 in any way. For example, one or more trained control models may be selected to derive control commands depending on the travel scenario, such as location, route, and type of road (for vehicles, e.g., highway, general road).

[0024] (Movement mode) A movement mode can be defined to specify a series of movement patterns that align with a particular purpose, such as lane changes or lane keeping. The purpose may be set appropriately for each movement mode depending on the type of moving object M. For example, if the moving object M is a vehicle, the movement modes may include lane changes, lane keeping, emergency stopping (EDSS: Emergency Driving Stop System), merging yield, automatic parking, or these. Combinations are allowed. Lane change is changing the lane you are driving in. Lane keeping is continuing to drive in the same lane. Emergency stop is pulling over to the shoulder and coming to a stop. Merging yield is giving way to another vehicle merging into your lane. Automatic parking is parking using automatic control.

[0025] For example, the movement modes may include lane changes, lane keeping, and emergency stops. The multiple trained control models 30 may include a first trained control model for lane changes, a second trained control model for lane keeping, and a third trained control model for emergency stops. Determining the mode of movement may include selecting one of the modes: lane change, lane keeping, or emergency stop. In one example of this embodiment, an improved accuracy in performing control appropriate to the scene can be expected in a vehicle equipped with these three modes. However, the types of modes of movement are not limited to these, and the modes of movement corresponding to each control model may be set arbitrarily. The term "mode of movement" may be interpreted differently depending on the type of moving body M. For example, if the moving body M is a vehicle, the mode of movement may also be called the driving mode. If the moving body M is an aircraft, the mode of movement may be called the flight mode. If the moving body M is a ship, the mode of movement may be called the navigation mode.

[0026] The method for determining the movement mode from the movement condition data 120 is not particularly limited and may be appropriately selected depending on the embodiment. In one example, a computational model may be used to determine the movement mode. The computational model may consist of, for example, a trained machine learning model, a rule-based model, or a combination thereof. The machine learning model may be configured in the same way as the control model described above. The rule-based model is configured to match the given input (movement condition data 120) against rules and derive a movement mode determination result according to the matching result (according to the matching rule). The rules may be set manually or at least partially automatically.

[0027] (Generated according to the movement mode) In this embodiment, each trained control model 30 is generated in accordance with the movement mode. The generation method may be appropriately selected depending on the embodiment. As an example of the generation method, training data (input data given to the control model) may be collected in accordance with each movement mode. Ground truth data (teacher signals, labels) may be appropriately attached to the collected training data. This makes it possible to obtain multiple datasets, each composed of combinations of training data and ground truth data. Each trained control model 30 may be generated by performing machine learning using the multiple datasets collected in accordance with the corresponding movement mode. That is, the control model 30 may be trained so that the inference result (derivation result of control command) obtained by providing the training data of each dataset obtained in accordance with the corresponding movement mode fits the corresponding ground truth data. Training may involve adjusting the computational parameters of the control model 30. Through this machine learning, each trained control model 30 can be generated that has acquired the ability to control the movement of the moving object M in accordance with the corresponding movement mode. In this embodiment, at least one of the multiple trained control models 30 is generated in accordance with a movement mode different from that of the other trained control models.

[0028] (Movement condition data) The movement condition data 120 relates to the conditions of movement of the moving object M at the time (present moment) when determining the movement mode. The movement condition data 120 may consist of any type of data that can be used as explanatory variables in determining the movement mode. In one example, the movement condition data 120 may include position data (e.g., measurement data from a GNSS / GPS module) and route data. The route data may be configured to show the route from the current location to the destination. The route data may be acquired from a navigation device as appropriate. The navigation device may be included in the control device 1, or it may be provided outside the control device 1 and connected to the control device 1 as appropriate. The current location may be the location indicated by the position data obtained at the present moment. The position data may consist of sensor data obtained from a positioning module such as a GNSS / GPS module. Accordingly, the sensor S may include a positioning module. The movement condition data 120 may further include map data. The type of map data may be arbitrary. In one example, high-precision map data may be used for the map data. The map data may be acquired from a navigation device, or it may be held in the control device 1. Location data Route data and map data can be collectively referred to as navigation data.

[0029] Furthermore, if the moving object M is a vehicle, the movement condition data 120 may further include at least one of driving data and reaction data. Driving data may consist of data indicating the driving state, such as speed and steering angle. Driving data may be acquired as appropriate from on-board sensors installed on the vehicle. When adopting a form that includes driving data, the sensor S may include on-board sensors. The reaction data is configured to show the reactions of the occupants OU riding in the vehicle. The reaction data may consist of data that can identify the occupants OU's reactions to driving, such as images, voice, and operation information via an input device. A monitoring device MD may be used to acquire the reaction data. The monitoring device MD may be, for example, an in-vehicle camera, a microphone, an input device (operation button), etc. The occupants OU may be, for example, a driver, a passenger other than the driver, etc. By adopting a form that includes reaction data, the feedback of the occupants OU can be reflected in the control. This makes it possible to expect control that is adapted to the occupants OU. Furthermore, other data (other sensor data) may also be used to determine the movement mode.

[0030] (optimization) The inputs to each trained control model 30 may be appropriately selected depending on the embodiment. In one example, all sensor data may be input to each trained control model 30 in common. However, depending on the movement mode, there may be sensor data that does not contribute much to the derivation of control commands (i.e., it is redundant). Therefore, in another example, the inputs to each trained control model 30 may be optimized according to the corresponding movement mode. Optimization means that the input to at least one of the multiple control models (control models 30) is less than the input to the other control models, thereby reducing the inputs to the control models. Optimization may include redundant inputs in at least some of the multiple control models, provided that the inputs are reduced.

[0031] Figure 2 schematically shows an example of a control model configuration according to this embodiment. In the example in Figure 2, the moving body M is a vehicle. Sensor S includes a plurality of sensors arranged in front, on each side, and at the rear, respectively, and acquires surrounding information (sensor data). Each sensor is, for example, a GPS module These include a radar, image sensor (camera), LiDAR, etc. The movement condition data 120 is location data, The data includes surrounding information (sensor data), driving data, navigation data (location data, map data, route data), and response data. Multiple trained control models 30 include a lane change model 301, a lane keeping model 302, and an emergency stop model 303. The lane change model 301 is an example of a first trained control model, the lane keeping model 302 is an example of a second trained control model, and the emergency stop model 303 is an example of a third trained control model. One of the movement modes (driving modes) of lane change, lane keeping, and emergency stop may be selected according to the movement condition data 120. Depending on the selected movement mode, one of the lane change model 301, the lane keeping model 302, and the emergency stop model 303 may be selected as the model to be used for control.

[0032] In the example shown in Figure 2, each model (301, 302, 303) receives common input for both driving data and navigation data. On the other hand, while the lane change model 301 and the emergency stop model 303 receive input for front, side, and rear sensor data, the lane keeping model 302 receives input for front and side sensor data. In other words, rear sensor data is omitted from the input of the lane keeping model 302. This omission is an example of optimization. When the input of the control model 30 is optimized, some sensors may not be used depending on the control model 35 selected according to the mode of movement. The control device 1 may appropriately stop the operation of unused sensors. In the example shown in Figure 2, while the lane keeping mode of movement is selected, the control device 1 may stop the operation of the rear sensors.

[0033] By optimizing the inputs to each trained control model 30, improvements in computational efficiency can be expected. For example, at least one of the following can be expected: reduced power consumption by suppressing the activation of unnecessary sensors, reduced sensor failure rate by optimizing sensor activation time, reduced computational load / amount of computation by optimizing the model, resulting in improved computational efficiency, and improved accuracy of model calculations by reducing unnecessary inputs. Note that the control model configuration and optimization form are not limited to the example in Figure 2 and may be appropriately selected depending on the embodiment. In another example, the multiple trained control models 30 may further include a trained control model corresponding to automatic parking (automatic parking model). Sensor S may include a clearance sonar. In this case, the sensor data of the clearance sonar may be input to the automatic parking model, while it may be omitted from the inputs of other models. Accordingly, the control device 1 may stop the operation of the clearance sonar while other models are being used and start it when the automatic parking model is being used.

[0034] (strength) In one example, determining the movement mode may include calculating the intensity. Deriving a control command may consist of deriving a control command according to the calculated intensity. The intensity may be appropriately configured to indicate the degree to which movement in the target movement mode is enforced. Each trained control model 30 may be configured to derive a control command that enforces movement in that movement mode as the intensity increases. For example, the higher the intensity, the smaller the constraint on the execution parameters. As a specific example, in the case of a lane change, the higher the intensity, the greater the degree to which the lane change is enforced, such as by executing it at an earlier timing, executing it in a narrow space, or executing it with stronger acceleration and deceleration. According to this example of the embodiment, the degree to which movement in each movement mode is performed can be controlled.

[0035] [2 Example Configurations] Figure 3 schematically shows an example of the hardware configuration of the control device 1 according to this embodiment. The control device 1 according to this embodiment is a computer in which a control unit 11, a storage unit 12, an external interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0036] The control unit 11 includes a CPU (Central Processing Unit) and RAM (Random Access Memory). The memory unit 12 includes a RAM (Read Only Memory), ROM (Read Only Memory), etc., and is configured to perform arbitrary information processing based on the program and various data. The control unit 11 (CPU) is an example of a processor resource. The memory unit 12 may be composed of any storage device such as a hard disk drive or a solid-state drive. The memory unit 12 (and RAM, ROM) is an example of a memory unit in this disclosure. The memory unit 12 (and RAM, ROM) is also an example of a memory resource. In this embodiment, the memory unit 12 stores various information such as a control program 81 and learning result data 300.

[0037] The control program 81 is a program that causes the control device 1 to execute information processing related to the control of the mobile body M (Figure 4, described later). The control program 81 includes a series of instructions for said information processing. The learning result data 300 is configured to show information about the trained control model 30. In one example, the learning result data 300 may be provided for each trained control model 30. In another example, one set of learning result data 300 may be configured to show two or more trained control models 30. Furthermore, if the trained control model 30 can be reproduced, the data structure of the learning result data 300 may be appropriately determined according to the embodiment. In one example, the learning result data 300 may include information showing the values ​​of the computational parameters of the control model 30 adjusted by machine learning. The learning result data 300 may further include information showing the configuration of the machine learning model (e.g., the structure of the neural network).

[0038] The external interface 13 may be, for example, a USB (Universal Serial Bus) port, a dedicated port, a wireless communication port, etc., and is configured to connect to an external device by wire or wireless connection. In this embodiment, the control device 1 may be connected to the sensor S and the monitoring device MD via the external interface 13. The input device 14 is, for example, a device for inputting data such as a mouse, keyboard, or control. The output device 15 is, for example, a device for outputting data such as a display or speaker. The input device 14 and the output device 15 may be integrated together by, for example, a touch panel display.

[0039] The drive 16 is a device for reading various information, such as programs, stored in the storage medium 91. At least one of the control program 81 and the learning result data 300 may be stored in the storage medium 91 instead of or together with the storage unit 12. The storage medium 91 is configured to store various information (stored programs, etc.) by electrical, magnetic, optical, mechanical, or chemical means so that a machine such as a computer can read the information. The control device 1 may acquire at least one of the control program 81 and the learning result data 300 from the storage medium 91. The storage medium 91 may be a disk-type storage medium such as a CD or DVD, or a non-disk-type storage medium such as semiconductor memory (e.g., flash memory). The type of drive 16 may be appropriately selected according to the type of storage medium 91.

[0040] Regarding the specific hardware configuration of the control device 1, components can be omitted, replaced, and added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. Hardware processors include microprocessors, FPGAs (field-programmable gate arrays), DSPs (digital signal processors), and ECUs. It consists of components such as an Electronic Control Unit (ECC) and a Graphics Processing Unit (GPU). At least one of the external interface 13, input device 14, output device 15, and drive 16 may be omitted. The control device 1 may be equipped with a communication interface and configured to perform data communication with an external computer. At least one of the input device 14, output device 15, and drive 16 may be connected via an external interface or a communication interface. The control device 1 may be a computer designed specifically for the services provided, a general-purpose computer, a terminal device, etc. If the mobile body M is a vehicle, the control device 1 may be an in-vehicle device.

[0041] [3 Examples of operation] Figure 4 shows an example of a processing procedure for controlling the mobile body M by the control device 1 according to this embodiment. The control unit 11 of the control device 1 executes instructions included in the control program 81 using the CPU. As a result, the control device 1 operates as a computer capable of performing the following information processing. The following processing procedure is an example of a control method executed by the computer. However, the following processing procedure is merely an example, and each step may be modified as much as possible. Furthermore, steps in the following processing procedure can be omitted, replaced, and added as appropriate, depending on the embodiment.

[0042] In step S101, the control unit 11 acquires movement condition data 120. The control unit 11 may acquire the movement condition data 120 directly or indirectly from an information source such as a sensor S. For example, if the moving object M is a vehicle, the movement condition data 120 may include reaction data indicating the reactions of the occupants OU riding in the vehicle. The reaction data may be acquired directly or indirectly from a monitoring device MD.

[0043] In step S102, the control unit 11 determines a movement mode that matches the movement conditions according to the acquired movement condition data 120. In one example, determining the movement mode may include calculating the intensity. Also, in the example shown in Figure 2 above, determining the movement mode is This may include selecting one of the following modes: lane change, lane keeping, or emergency stop.

[0044] In step S103, the control unit 11 selects a trained control model 35 to be used for control from among a plurality of trained control models 30, according to the result of determining the movement mode. In one example, the inputs of each trained control model 30 may be optimized according to the corresponding movement mode. Accordingly, the control unit 11 may, at any time, activate the operation of the sensors to be used and deactivate the operation of the sensors that are not being used, according to the selection result.

[0045] In step S104, the control unit 11 derives a control command using the selected trained control model 35. The control unit 11 may, at any time, appropriately acquire data to be given to the trained control model 35, either directly or indirectly, from sensors S, etc. The control unit 11 may input the acquired data into the trained control model 35 and execute calculation processing of the trained control model 35. As a result, the control unit 11 may obtain the result of the control command derivation from the trained control model 35. When the above method of calculating intensity is adopted, the derivation of the control command may be performed by deriving the control command according to the calculated intensity.

[0046] In step S105, the control unit 11 controls the movement (movement) of the mobile body M according to the derived control command. Once the control of the target mobile body M is complete, the control unit 11 terminates the processing procedure of the control device 1 related to this example of operation.

[0047] The control unit 11 may repeatedly execute the series of information processing steps S101 to S105 at any time. For example, the control unit 11 may repeatedly execute the series of information processing steps S101 to S105 for a predetermined period of time (for example, while the power source of the mobile body M is started and the automatic control mode is selected). This allows the control device 1 to continuously perform automatic control of the mobile body M.

[0048] [Features] In this embodiment, steps S101 to S103 select a trained control model 35 from among multiple trained control models 30 that is suitable for the movement conditions. Then, steps S104 and S105 select the trained control model 35 and use it to control the mobile object M. Since each trained control model 30 is not responsible for movement in any movement mode other than its corresponding movement mode, the mixing of training data for other movement modes can be suppressed in the machine learning of each control model 30. With this specialized configuration, it can be expected that each control model 30 will acquire a high level of ability to control the movement of the mobile object M in its corresponding movement mode through machine learning. As a result, in step S105, the accuracy of performing control of the mobile object M that is suitable for the scene can be improved.

[0049] [4. Variant] While embodiments of this disclosure have been described in detail above, the above description is merely illustrative in all respects of this disclosure. Needless to say, various improvements or modifications can be made without departing from the scope of this disclosure. Furthermore, the processes and means described in this disclosure can be freely combined and implemented as long as no technical inconsistencies arise. [Explanation of symbols]

[0050] 1...Control device, 11...Control unit, 12...Storage unit, 120...Movement condition data, 30.35…Trained control models, 301...Lane change model (first trained control model), 302... Lane keeping model (2nd trained control model), 303…Emergency Stop Model (Third Trained Control Model), M...Mobile unit, OU...Occupant, S...Sensor, MD...Monitoring device

Claims

1. A memory unit for storing multiple trained control models, and Control unit, A control device comprising, Each of the aforementioned trained control models is generated in accordance with the movement mode of the mobile body, and The control unit, To acquire movement condition data relating to the conditions under which the moving object moves, Based on the acquired movement condition data, determine the movement mode. To calculate the intensity that indicates the degree to which movement in the determined movement mode is enforced, In accordance with the result of determining the aforementioned movement mode, one of the multiple trained control models is selected. Using the selected trained control model, derive control commands for the moving body according to the calculated intensity, and Controlling the movement of the moving body according to the derived control command, Configured to perform, Control device.

2. The aforementioned moving object is a vehicle, The aforementioned multiple trained control models include a first trained control model for lane changes, a second trained control model for lane keeping, and a third trained control model for emergency stops. Determining the mode of movement includes selecting one of the following modes: lane change, lane keeping, and emergency stop. The control device according to claim 1.

3. The acquired movement condition data includes reaction data showing the reactions of the occupants riding in the vehicle. The control device according to claim 2.

4. The inputs to each of the aforementioned trained control models are optimized according to the corresponding movement mode. ru, The control device according to any one of claims 1 to 3.

Images