Control device

Abstract

The invention provides a control device capable of improving execution precision of control of a moving body suitable for a scene. A control device according to one aspect of the present disclosure acquires movement condition data of a moving body, determines a movement pattern on the basis of the acquired movement condition data, and selects one of a plurality of trained control models on the basis of the determination result of the movement pattern. A control command for the moving body is derived using the selected trained control model, and an operation of the moving body is controlled in accordance with the derived control command. Each of the trained control models is generated in accordance with a movement pattern of the moving body.

Description

Technical Field

[0001] The present disclosure relates to control technology for mobile objects such as autonomous vehicles. Background Art

[0002] Patent Document 1 proposes an autonomous vehicle control system that uses a trained neural network and determines vehicle commands based on a route, GPS data, and sensor data.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application No. 2019-533810 Summary of the invention

[0006] Problems to be solved by the invention

[0007] One of the objects of the present disclosure is to provide a technology for improving the execution accuracy of control of a moving object suitable for a scene.

[0008] Technical solutions to solve problems

[0009] The control device of the first scheme of the present disclosure comprises a storage unit for storing a plurality of trained control models, and a control unit. Each of the trained control models is generated corresponding to a movement mode of a mobile body. The control unit is configured to perform the following processing: obtaining movement condition data related to the movement conditions of the mobile body; determining the movement mode based on the obtained movement condition data; selecting one of the plurality of trained control models based on the determination result of the movement mode; deriving control instructions for the mobile body using the selected trained control model; and controlling the movement of the mobile body according to the derived control instructions. Each trained control model can be composed of a neural network, and deep learning can be used in the machine learning method.

[0010] Effects of the Invention

[0011] According to the present disclosure, it is possible to improve the execution accuracy of the control of the moving object suitable for the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 An example of a scenario in which the present disclosure is applied is schematically shown.

[0013] Figure 2 An example of the control model configuration of the present disclosure is schematically shown.

[0014] Figure 3 An example of the hardware configuration of the control device of the present disclosure is schematically shown.

[0015] Figure 4 An example of a processing procedure related to control performed by the control device of the present disclosure is shown.

[0016] Description of Reference Numerals

[0017] 1…control device, 11…control unit, 12…storage unit, 120…movement condition data, 30, 35…trained control model, 301…lane change model (first trained control model), 302…lane keeping model (second trained control model), 303…emergency parking model (third trained control model), M…moving body, OU…occupant, S…sensor, MD…monitoring device. DETAILED DESCRIPTION

[0018] According to the previous methods such as Patent Document 1, an automatic driving system can be constructed by using a trained machine learning model (especially an end-to-end model). However, the inventors of the present application have found that the following problems exist in the previous methods. For example, the driving mode may be very different depending on the scenarios such as lane change and lane keeping. In addition, the probability of occurrence of each scenario may also be very different. In one example, the probability of lane change is sometimes less than 1% when driving more than 20 km, which is extremely low compared to lane keeping. Moreover, in the scenario of lane change with a probability of less than 1%, the probability of avoiding collision with surrounding vehicles or avoiding obstacles on the road (stationary objects, animals, etc.) is extremely low. Imagine collecting learning data in a state where such driving modes and scenarios with very different probabilities of occurrence are mixed, and using the collected learning data to train the machine learning model. In this case, in the obtained trained machine learning model, it may be difficult to perform vehicle control suitable for the scenario with high accuracy. For example, the learning of driving modes with extremely low probability of occurrence may be insufficient, resulting in a decrease in the accuracy of vehicle control based on the trained machine learning model. In addition, this problem may occur regardless of the type of vehicle. In addition, "such a problem occurs" is not limited to the scene of controlling the vehicle. In terms of controlling movement, the same is true for moving objects other than vehicles. Therefore, the same problem may also occur in the scene of controlling all moving objects other than vehicles.

[0019] In contrast, the control device of the first scheme of the present disclosure includes a storage unit storing a plurality of trained control models, and a control unit. Each trained control model is generated corresponding to a movement mode of a mobile body. The control unit is configured to perform the following processing: obtaining movement condition data related to movement conditions of the mobile body; determining the movement mode based on the obtained movement condition data; selecting one of the plurality of trained control models based on the determination result of the movement mode; deriving a control instruction of the mobile body using the selected trained control model; and controlling the movement of the mobile body according to the derived control instruction.

[0020] In the first scheme of the present disclosure, each trained control model is prepared according to the mobile mode. That is, each trained control model is prepared "exclusively for the corresponding mobile mode" and is not responsible for movement in other mobile modes. Therefore, in the machine learning of each control model, the mixed existence of data of other mobile modes can be suppressed (ideally, learning data is collected by specializing in the corresponding mobile mode, and the trained model is generated using the obtained learning data). As a result, it can be expected that the execution accuracy of the control of the mobile body suitable for the scene will be improved.

[0021] In addition, as another form of the control device of the above scheme, one aspect of the present disclosure may also be an information processing method that realizes all or part of the above components, or a program, or a storage medium readable by a device such as a computer storing such a program. Here, a storage medium readable by a device refers to a medium that stores information such as a program through electrical, magnetic, optical, mechanical or chemical action.

[0022] [1 Application example]

[0023] Figure 1 An example of a scenario in which the present disclosure is applied is schematically shown. The control device 1 of the present embodiment is one or more computers configured to control the automatic movement of a moving body M as an object. In the present embodiment, the control device 1 is mounted on the moving body M and maintains a plurality of trained control models 30. Each trained control model 30 is generated corresponding to the movement mode of the moving body M. That is, each trained control model 30 is specialized for the corresponding movement mode and acquires the ability to derive control instructions for controlling the movement of the moving body M.

[0024] In the present embodiment, the control device 1 obtains movement condition data 120 related to the movement conditions of the mobile body M. The control device 1 determines the movement mode (driving mode, flight mode, navigation mode, etc.) that is suitable for the conditions based on the obtained movement condition data 120. The control device 1 selects one of the multiple trained control models 30 (trained control model 35) based on the determination result of the movement mode. That is, the control device 1 selects the trained control model 35 corresponding to the determined movement mode (determination result of the movement mode) from the multiple trained control models 30. The control device 1 uses the selected trained control model 35 to derive the control instructions of the mobile body M in the determined movement mode. The control device 1 controls the action of the mobile body M according to the derived control instructions. In addition, a series of processes from "obtaining the movement condition data 120" to "controlling the mobile body M" can be performed in real time.

[0025] In the present embodiment, each trained control model 30 is prepared according to the movement mode. That is, each trained control model 30 is not responsible for movement in other movement modes other than the corresponding movement mode, and therefore, in the machine learning of each control model 30, the mixed existence of learning data of other movement modes can be suppressed. Therefore, according to the present embodiment, it can be expected that the execution accuracy of the control of the mobile body M suitable for the scene will be improved. In addition, sometimes it is required to update through relearning (additional learning) of the control model 30 due to certain reasons such as "suitability for specialization to a specific region, changes in the environment (for example, the road is changed)". In this regard, in the present embodiment, when a method of preparing each trained control model 30 independently is adopted, only the control model 30 corresponding to the reason needs to be relearned, so it is not necessary to update the entire body, and it can be expected that the update will be more efficient.

[0026] (Moving body)

[0027] The type of the mobile body M can be appropriately selected according to the embodiment as long as it can be automatically moved by machine control. The mobile body M can be, for example, a vehicle, an aircraft, a ship, a robot device, or other movable device. The aircraft can be at least one of an unmanned aircraft such as an unmanned aircraft and a manned aircraft. In one example, Figure 1 Thus, the moving body M may be a vehicle. In addition, the type of vehicle (number of wheels, power source, size, etc.) may be selected arbitrarily. As a typical example, the moving body M may be an automobile having a level 2 or higher autonomous driving capability.

[0028] (Control the action)

[0029] In one example, "controlling the movement of the moving body M as an object" includes "directly controlling the moving body M as an object". In another example, the moving body M may have a dedicated control device such as a controller. In this case, "the control device 1 controls the movement of the moving body M as an object" may include "indirectly controlling the moving body M as an object by providing the derived result to the dedicated control device". The control device 1 may be installed in any place. In one example, as Figure 1 As shown, the control device 1 may be mounted on the mobile body M. In another example, the control device 1 may be configured separately from the mobile body M to remotely control the mobile body M. The control device 1 may be configured to be able to perform "switching from the automatic control mode of the mobile body M to the manual control mode" and "switching from the manual control mode to the automatic control mode" at any timing through any operation.

[0030] (Control Model)

[0031] The control model 30 (35) is composed of a machine learning model having one or more operation parameters that can be adjusted by machine learning. One or more operation parameters are used for the operation of inference as the purpose (in this application, the derivation of control instructions). Machine learning is the use of learning data to adjust (optimize) the value of the operation parameter. The composition and type of the machine learning model are not particularly limited, and can be appropriately selected according to the implementation method. The machine learning model can be composed of, for example, a neural network, a support vector machine, a regression model, a decision tree model, etc. The machine learning method can be appropriately selected according to the machine learning model adopted (for example, error back propagation method, etc.). Machine learning can include teacher learning, unsupervised learning, and reinforcement learning. In one example, at least any one of the multiple control models 30 can be at least partially composed of a neural network. The structure of the neural network can be appropriately determined according to the implementation method. The structure of the neural network can be determined, for example, based on the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) contained in each layer, the connection relationship between the nodes of each layer, etc. In one example, the neural network can have a recursive structure. In addition, the neural network may have any layer such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, a depooling layer, a normalization layer, a random dropout layer, an LSTM (Long short-term memory), etc. The neural network may have any mechanism such as an Attention mechanism. The neural network may include any model such as a GNN (Graph neural network), a diffusion model, a generative model (for example, a Generative Adversarial Network, a Transformer, etc.). When a neural network is used for a control model, the weights of the connections between the nodes contained in the control model and the thresholds of each node are examples of operating parameters. In addition, when a machine learning model is used, the control model may be composed of an end-to-end model structure.

[0032] Each control model 30 is constructed in such a way that a control command is derived according to the environment of the mobile body M. The environment is a phenomenon observed by at least one of the mobile body M itself and the surroundings. In one example, at least a part of the environment can be observed by one or more sensors S arranged inside or outside the mobile body M. The sensor S can be any environment in which the mobile body M moves, and its type is not particularly limited, and can be appropriately selected according to the implementation. In one example, one or more sensors S may include an image sensor (camera), a radar, a LiDAR (Light Detection And Ranging), a sonar (ultrasonic sensor), an infrared sensor, a GNSS (Global Navigation Satellite System) / GPS (Global Positioning Satellite) module, etc. The sensor S can be appropriately configured in such a way that it observes any direction such as the front, right side, left side, and rear of the mobile body M. In the case where the mobile body M is a vehicle, the sensor S may include at least one of an electronic rearview mirror (EIM), a panoramic monitoring device (PVM), and a millimeter wave radar. The electronic rearview mirror may include an image sensor arranged at the rear of the vehicle. The panoramic monitoring device may include image sensors disposed in front, on each side, and on the rear of the vehicle. A plurality of millimeter wave radars may be disposed on each side of the front and each side of the rear of the vehicle, respectively.

[0033] As long as the control instructions can be derived according to the environment of the mobile body M, the form of the input and output of each control model 30 can be appropriately selected according to the implementation method. In one example, at least one of the multiple control models 30 can be configured to derive control instructions based on the observation data of the sensor S at more than one time point. For example, the control model 30 can be configured to perform peripheral recognition, path planning (path / track planning), and motion planning (action / control planning) (end-to-end model). In another example, at least one of the multiple control models 30 can be configured to derive control instructions based on the recognition results of the surrounding environment. In this case, the control device 1 may also have an analytical model that infers the recognition results of the surrounding environment based on the observation data of the sensor S. Alternatively, at least one of the multiple control models 30 may include such an analytical model. The analytical model can be appropriately configured. In one example, the analytical model can be composed of a machine learning model. In addition, other information can be arbitrarily added to the input of at least one of the multiple control models 30. At least any one of the plurality of control models 30 may be configured to further receive input of arbitrary information such as a set speed, a speed limit, driving data, map data, or navigation information (route data), for example.

[0034] In addition, typically, the plurality of trained control models 30 are separately (independently) constructed. However, the construction of the plurality of trained control models 30 may not be limited to such an example. In one example of the present embodiment, two or more of the plurality of trained control models 30 may also be at least partially integrally constructed. For example, one model may include the above-mentioned analytical model configured on the input side and n output parts (n is a natural number greater than 2) that derive control instructions respectively according to the output of the analytical model. This one model may be regarded as n trained control models 30. That is, in the present embodiment, even if the structure of the model is integrated, in the case where the model also has n output parts that are respectively configured to output control instructions, "maintaining the model" may be regarded as "maintaining n trained control models 30". In this case, since the output parts are each independent, the n trained control models 30 may be regarded as being independently constructed. In addition, the same treatment may be applied to conditional models that are configured to output in accordance with the given conditions. That is, the plurality of trained control models 30 may include conditional models that are configured to output control instructions corresponding to the input conditions (levels / categories). In this case, by changing the conditions assigned to the conditional model and repeatedly performing the operation of the conditional model, multiple control instructions corresponding to the assigned conditions can be derived. Therefore, in the case where one conditional model corresponds to n conditions, "maintaining this one conditional model" can be regarded as "maintaining n trained control models 30".

[0035] (Control Instructions)

[0036] The control instructions are related to the action of the mobile body M. The composition of the control instructions can be appropriately selected according to the implementation method. In one example, the control instructions can be composed of acceleration, deceleration, steering or a combination thereof. Acceleration and deceleration may include gear changes. In the case of including at least one of acceleration, deceleration and steering, the control instructions can be represented by a path. Accordingly, the control model 30 can be represented as a path planner. In addition, the control instructions may also include instructions related to the operation of the mobile body M. As an example, when the mobile body M is a vehicle, the control instructions may include vehicle operations such as direction indication, hazard warning, honking, communication processing (for example, "sending data to the center", "sending an emergency call", etc.).

[0037] Each control model 30 may be configured to directly output a control instruction or to indirectly output a control instruction. In the latter case, the control instruction may be obtained by performing any information processing (interpretation processing) on ​​the output of the control model 30. The control instruction may be configured to directly represent the control quantity (control instruction value, control output quantity) of the mobile body M, such as the accelerator control quantity, the brake control quantity, the steering wheel steering angle, etc. Alternatively, the control instruction may be configured to indirectly represent the control quantity of the mobile body M, such as the path, the state after control, etc. In this case, the control quantity of the mobile body M may be obtained from the control instruction by performing any information processing.

[0038] In addition, in this embodiment, typically, the control device 1 can only perform the calculation processing of the selected trained control model 35 to obtain the control instruction. Accordingly, the control device 1 can stop the operation of at least a part of the sensors S provided in the mobile body M that are not used in the selected trained control model 35 but are used in other trained control models. Thus, it can be expected that the calculation load and the amount of power consumption will be reduced. However, the process of obtaining the control instruction is not limited to such an example. In another example, the control device 1 can perform the calculation processing of more than one trained control model that has a probability of being used among multiple trained control models 30 to obtain more than one control instruction. In this case, "selecting the trained control model 35" may include "selecting the control instruction (control instruction for control) derived from the trained control model 35 from more than one control instruction". Thus, the rapid responsiveness to the switching of the mobile mode can be improved. In addition, more than one trained control model that has a probability of being used can be selected from multiple trained control models 30 by any method. In one example, one or more trained control models for deriving control instructions may be selected based on the movement scenario such as the location, route, and type of passage (for example, a highway or a general road in the case of a vehicle).

[0039] (Mobile Mode)

[0040] The movement mode is defined as a series of movement methods that can be specified according to specific purposes such as lane change, lane keeping (maintaining), etc. The "purpose" can be appropriately set for each movement mode according to the type of the moving body M. In one example, when the moving body M is a vehicle, the movement mode may include lane change, lane keeping, emergency stop (EDSS: Emergency Driving Stop System), merging and giving way, automatic parking, or a combination thereof. Lane change is changing the lane of travel. Lane keeping is continuing to travel in the same lane. Emergency stop is retreating to the shoulder of the road and parking. Giving way to merging is giving way to other vehicles merging into the lane of the vehicle. Automatic parking is parking through automatic control.

[0041] For example, the mobile mode may include lane change, lane keeping, and emergency stop. Accordingly, the plurality of trained control models 30 may include a first trained control model corresponding to lane change, a second trained control model corresponding to lane keeping, and a third trained control model corresponding to emergency stop. "Determining the mobile mode" may include "selecting one of the modes of lane change, lane keeping, and emergency stop". In one example of the present embodiment, in a vehicle having these three modes, it is possible to expect an improvement in the execution accuracy of the control suitable (matching) for the scene. However, the types of mobile modes may not be limited to these, and the mobile modes corresponding to each control model may be set arbitrarily. In addition, the mobile mode may be replaced with other wordings according to the type of the mobile body M. For example, when the mobile body M is a vehicle, the mobile mode may also be referred to as a driving mode. When the mobile body M is a flying body, the mobile mode may be referred to as a flight mode. When the mobile body M is a ship, the mobile mode may be referred to as a navigation mode.

[0042] In addition, the method of determining the movement mode based on the movement condition data 120 may not be particularly limited and may be appropriately selected according to the implementation method. In one example, a computational model may be used to determine the movement mode. The computational model may be composed of, for example, a trained machine learning model, a rule base model, or a combination thereof. The machine learning model may be constructed in the same manner as the above-mentioned control model. The rule base model is constructed to compare the provided input (movement condition data 120) with the rules, and derive the result of the movement mode determination based on the comparison result (according to the appropriate (matching) rules). The rules may be set manually or at least partially automatically.

[0043] (Generated according to the movement mode)

[0044] In the present embodiment, each trained control model 30 is generated corresponding to the movement mode. The generation method can be appropriately selected according to the embodiment. As an example of the generation method, training data (input data given to the control model) can be collected corresponding to each movement mode. Correct data (teacher signal, label) can be appropriately assigned to the collected training data. Thus, a plurality of data groups each consisting of a combination of training data and correct data can be obtained. Each trained control model 30 can be generated by implementing machine learning using a plurality of data groups collected corresponding to the corresponding movement mode. That is, the control model 30 can be trained in such a way that the inference result (derivation result of the control instruction) obtained by giving the training data of each data group obtained corresponding to the corresponding movement mode to the control model 30 becomes a result matching the corresponding correct data. "Training" can be adjusting the operation parameters of the control model 30. Through this machine learning, each trained control model 30 that has acquired the ability to control the movement of the mobile body M in the corresponding movement mode can be generated. In the present embodiment, at least one of the plurality of trained control models 30 is generated corresponding to a movement mode different from the movement mode of other trained control models.

[0045] (Moving condition data)

[0046] The movement condition data 120 is related to the movement condition of the moving body M at the time point (current time point) of determining the movement mode. The movement condition data 120 can be composed of any type of data that can be used as an explanatory variable in the determination of the movement mode. In one example, the movement condition data 120 may include position data (for example, measurement data of a GNSS / GPS module) and route data. The route data may be constructed to represent a route from the current position to the destination. The route data may be appropriately obtained from a navigation device. The navigation device may be included in the control device 1, or may be arranged outside the control device 1 and appropriately connected to the control device 1. The current position may be a position represented by the position data obtained at the current time point. The position data may be composed of sensor data obtained by a positioning module such as a GNSS / GPS module. Accordingly, the sensor S may include a positioning module. The movement condition data 120 may also include map data. The type of map data may be arbitrary. In one example, high-precision map data may be used for the map data. The map data may be obtained from the navigation device, or may be maintained in the control device 1. Furthermore, position data, route data, and map data may be collectively referred to as navigation data.

[0047] In addition, when the mobile body M is a vehicle, the mobile condition data 120 may also include at least one of driving data and reaction data. Driving data may be composed of data indicating the driving state, such as speed and steering angle. Driving data can be appropriately obtained from a vehicle-mounted sensor installed in the vehicle. In the case of adopting a method including driving data, the sensor S may include a vehicle-mounted sensor. The reaction data is composed of data indicating the reaction of the occupant OU riding the vehicle. The reaction data may be composed of data that can determine the reaction of the occupant OU to driving, such as images, sounds, and operation information input via an input device. For obtaining the reaction data, a monitoring device MD can be used. The monitoring device MD may be, for example, an in-vehicle camera, a microphone, an input device (operation button), etc. The occupant OU may be, for example, a driver, a fellow passenger other than the driver, etc. By adopting a method including reaction data, the feedback of the occupant OU can be reflected in the control. As a result, control suitable for the occupant OU can be expected. In addition, other data (other sensor data) other than the above may also be used for determining the movement mode.

[0048] (Optimization)

[0049] The input of each trained control model 30 can be appropriately selected according to the implementation method. In one example, all sensor data can be commonly input to each trained control model 30. However, depending on the movement mode, there may be sensor data that has a low contribution to the derivation of control instructions (that is, there is redundancy). Therefore, in another example, the input of each trained control model 30 can be optimized according to the corresponding movement mode. "Optimization" can be "making the input of at least any one of the multiple control models (control models 30) less than the input of other control models, thereby achieving a reduction in the input to the control model". "Optimization" may include "if a reduction in input can be achieved, then there is redundant input in at least a part of the multiple control models."

[0050] Figure 2 An example of the control model configuration of this embodiment is schematically shown. Figure 2In one example, the moving body M is a vehicle. The sensor S includes a plurality of sensors respectively arranged in the front, each side and the rear, and obtains surrounding information (sensor data). Each sensor is, for example, a GPS module, an image sensor (camera), a LiDAR, etc. The moving condition data 120 includes position data, surrounding information (sensor data), driving data, navigation data (position data, map data, route data), and reaction data. The plurality of trained control models 30 include a lane change model 301, a lane keeping model 302, and an emergency stop model 303. The lane change model 301 is an example of the first trained control model, the lane keeping model 302 is an example of the second trained control model, and the emergency stop model 303 is an example of the third trained control model. Any one of the moving modes (driving modes) of lane change, lane keeping and emergency stop can be selected according to the moving condition data 120. According to the selected moving mode, one of the lane change model 301, the lane keeping model 302, and the emergency stop model 303 can be selected as the model for control.

[0051] exist Figure 2 In one example, driving data and navigation data are commonly input to each model (301, 302, 303). On the other hand, the front, side and rear sensor data are input to the lane change model 301 and the emergency stop model 303, and the front and side sensor data are input to the lane keeping model 302. That is, the rear sensor data is omitted from the input of the lane keeping model 302. This omission is an example of optimization. When the input of the control model 30 is optimized, sensors that are not used by the selected control model 35 may be generated depending on the movement pattern. The control device 1 can appropriately stop the operation of unused sensors. In Figure 2 In one example, while the lane keeping movement mode is selected, the control device 1 may stop the operation of the rear sensor.

[0052] By optimizing the input of each trained control model 30, it is expected that the operation efficiency will be improved. For example, it is expected that at least one of the following will be achieved: a reduction in power consumption by suppressing the activation of unnecessary sensors, a reduction in sensor failure rate by efficiencies in the activation time of sensors, a reduction in the operation load / amount of operations by optimizing the model, and an improvement in the operation efficiency thereby achieved, and an improvement in the accuracy of model operations by reducing unnecessary inputs. In addition, the control model configuration and optimization method may not be limited to Figure 2In another example, the plurality of trained control models 30 may further include a trained control model corresponding to automatic parking (automatic parking model). The sensor S may include a clearance sonar. In this case, the sensor data of the clearance sonar may be input to the automatic parking model, and the sensor data of the clearance sonar may be omitted from the input of other models. Accordingly, the control device 1 may stop the operation of the clearance sonar during the use of other models and start it when the automatic parking model is used.

[0053] (strength)

[0054] In one example, "determining the movement mode" may include "calculating the intensity". "Exporting control instructions" may include "exporting control instructions based on the calculated intensity". "Intensity" may be appropriately constructed in a manner that represents the degree of enforcement of movement in the movement mode of the object. Each trained control model 30 may be constructed to export control instructions in a manner that the higher the intensity, the more enforced the movement in the movement mode. For example, the higher the intensity, the smaller the restriction on the execution parameter. As a specific example, in the case of lane change, the higher the intensity, the more "early timing execution", "execution in a narrow space", "enforced acceleration and deceleration (acceleration, deceleration)", etc., so that the degree of enforcement of the lane change is increased. According to one example of the present embodiment, the execution degree of movement in each movement mode can be controlled.

[0055] [2 Configuration example]

[0056] Figure 3 An example of the hardware configuration of the control device 1 of the present embodiment is schematically shown. The control device 1 of the present embodiment is a computer in which a control unit 11, a storage unit 12, an external interface (I / F) 13, an input device 14, an output device 15, and a driver 16 are electrically connected.

[0057] The control unit 11 is configured to include a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc., and performs arbitrary information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 can be composed of any storage device such as a hard disk drive, a solid state drive, etc. The storage unit 12 (and RAM, ROM) is an example of a storage unit of the present disclosure. In addition, the storage unit 12 (and RAM, ROM) is an example of a memory resource. In the present embodiment, the storage unit 12 stores various information such as the control program 81 and the learning result data 300.

[0058] The control program 81 is used to cause the control device 1 to execute information processing related to the control of the moving body M (described later). Figure 4 ) program. The control program 81 includes a series of commands for processing the information. The learning result data 300 is configured to represent information related to the trained control model 30. In one example, the learning result data 300 can be provided for each trained control model 30. In another example, one piece of learning result data 300 can be configured to represent more than two trained control models 30. In addition, as long as the trained control model 30 can be regenerated, the data structure of the learning result data 300 can be appropriately determined according to the implementation method. In one example, the learning result data 300 may include information representing the values ​​of the operation parameters of the control model 30 adjusted by machine learning. The learning result data 300 may also include information representing the structure of the machine learning model (for example, the structure of the neural network, etc.).

[0059] The external interface 13 may be, for example, a USB (Universal Serial Bus) port, a dedicated port, a wireless communication port, etc., and is configured to be connected to an external device by wire or wireless. In this embodiment, the control device 1 may be connected to the sensor S and the monitoring device MD via the external interface 13. The input device 14 may be, for example, a device for inputting, such as a mouse, a keyboard, or an operating member. The output device 15 may be, for example, a device for outputting, such as a display, a speaker, etc. The input device 14 and the output device 15 may be, for example, integrally formed by a touch panel display, etc.

[0060] The drive 16 is a device for reading various information such as programs stored in the storage medium 91. At least one of the control program 81 and the learning result data 300 can be stored in the storage medium 91 instead of being stored in the storage unit 12, or stored in the storage unit 12 and stored in the storage medium 91. The storage medium 91 is configured to store various information (stored programs, etc.) through electrical, magnetic, optical, mechanical or chemical effects in a manner that enables a computer or other device to read the information. The control device 1 can obtain at least one of the control program 81 and the learning result data 300 from the storage medium 91. In addition, the storage medium 91 can be a disk-type storage medium such as a CD, DVD, or a storage medium other than a disk-type such as a semiconductor memory (for example, a flash memory). The type of the drive 16 can be appropriately selected according to the type of the storage medium 91.

[0061] In addition, regarding the specific hardware configuration of the control device 1, the components can be appropriately omitted, replaced, and added according to the implementation method. For example, the control unit 11 may include a plurality of hardware processors. The hardware processor may be composed of a microprocessor, an FPGA (field-programmable gate array), a DSP (digital signal processor), an ECU (Electronic Control Unit), a GPU (Graphics Processing Unit), etc. At least one of the external interface 13, the input device 14, the output device 15, and the driver 16 may also be omitted. The control device 1 may also be configured to have a communication interface to perform data communication with an external computer. At least one of the input device 14, the output device 15, and the driver 16 may also be connected via an external interface or a communication interface. In addition to a computer designed specifically for the service provided, the control device 1 may also be a general-purpose computer, a terminal device, etc. In the case where the mobile body M is a vehicle, the control device 1 may be a vehicle-mounted device.

[0062] [3 Action Examples]

[0063] Figure 4 An example of processing steps related to the control of the mobile body M performed by the control device 1 of the present embodiment is shown. The control unit 11 of the control device 1 executes the commands included in the control program 81 through the CPU. As a result, the control device 1 acts as a computer capable of performing the following information processing. The following processing steps are an example of a control method performed by a computer. However, the following processing steps are only an example, and each step can be changed within a possible range. In addition, regarding the following processing steps, the steps can be appropriately omitted, replaced, and added according to the embodiment.

[0064] In step S101, the control unit 11 obtains the movement condition data 120. The control unit 11 may obtain the movement condition data 120 directly or indirectly from an information source such as a sensor S. In one example, when the moving body M is a vehicle, the movement condition data 120 may include reaction data indicating the reaction of the occupant OU riding in the vehicle. The reaction data may be obtained directly or indirectly from the monitoring device MD.

[0065] In step S102, the control unit 11 determines a movement mode suitable for the movement condition based on the obtained movement condition data 120. In one example, the process of determining the movement mode may include a process of calculating the intensity. Figure 2 In one example, the process of determining the movement mode may include a process of selecting one of lane change, lane keeping, and emergency stop.

[0066] In step S103, the control unit 11 selects a trained control model 35 for control from a plurality of trained control models 30 according to the result of the movement pattern determination. In one example, the input of each trained control model 30 can be optimized according to the corresponding movement pattern. Accordingly, the control unit 11 can start the operation of the sensor to be used in the sensor group included in the sensor S and stop the operation of the unused sensor according to the selection result at an arbitrary timing.

[0067] In step S104, the control unit 11 uses the selected trained control model 35 to derive control instructions. The control unit 11 can obtain data provided to the trained control model 35 directly or indirectly from the sensor S or the like at any timing. The control unit 11 can input the obtained data into the trained control model 35 and perform calculation processing of the trained control model 35. Thus, the control unit 11 can obtain the derivation result of the control instruction from the trained control model 35. In the case of adopting the method of calculating the above-mentioned intensity, the process of deriving the control instruction may include the process of deriving the control instruction according to the calculated intensity.

[0068] In step S105, the control unit 11 controls the action (movement) of the moving object M according to the derived control command. When the control of the moving object M is completed, the control unit 11 ends the processing procedure of the control device 1 in this operation example.

[0069] The control unit 11 can repeatedly perform a series of information processing from step S101 to step S105 at any timing. In one example, the control unit 11 can repeatedly perform a series of information processing from step S101 to step S105 during a predetermined period (for example, during a period when the power source of the mobile body M is started and the automatic control mode is selected). In this way, the control device 1 can continuously perform automatic control of the mobile body M.

[0070] [feature]

[0071] In this embodiment, a trained control model 35 suitable for the moving condition is selected from a plurality of trained control models 30 through the processing of steps S101 to S103. Then, the selected trained control model 35 is used for the control of the moving body M through the processing of steps S104 and S105. Since each trained control model 30 is not responsible for the movement in other moving modes other than the corresponding moving mode, the mixed existence of the learning data of other moving modes can be suppressed in the machine learning of each control model 30. Through this specialized structure, it can be expected that each control model 30 can obtain the ability to control the movement of the moving body M in the corresponding moving mode at a high level in the machine learning. As a result, in step S105, the execution accuracy of the control of the moving body M suitable for the scene can be improved.

[0072] [4 Modifications]

[0073] The embodiments of the present disclosure have been described in detail above, but the aforementioned description is merely an illustration of the present disclosure in all aspects. Various improvements or modifications can be made without departing from the scope of the present disclosure. In addition, the processing and means described in the present disclosure can be freely combined and implemented as long as no technical contradictions are generated.

Claims

1. A control device comprising: A storage unit for storing a plurality of trained control models; and Control Department, Each of the trained control models is generated corresponding to the movement pattern of the moving object, and, The control unit is configured to execute the following processing: acquiring movement condition data related to movement conditions of the moving body; determining a movement mode according to the obtained movement condition data; selecting one of the plurality of trained control models according to the determination result of the movement mode; Using the selected trained control model to derive control instructions for the moving object; as well as The movement of the moving object is controlled according to the derived control command.

2. The control device according to claim 1, The process of determining the movement mode includes a process of calculating the intensity, The process of deriving the control command includes a process of deriving the control command based on the calculated strength.

3. The control device according to claim 1, The mobile object is a vehicle, The plurality of trained control models include a first trained control model corresponding to lane change, a second trained control model corresponding to lane keeping, and a third trained control model corresponding to emergency stop. The process of determining the movement mode includes a process of selecting one of the lane change, the lane keeping, and the emergency stop.

4. The control device according to claim 3, The acquired movement condition data includes response data indicating a response of an occupant riding in the vehicle.

5. The control device according to any one of claims 1 to 4, The input of each of the trained control models is optimized according to the corresponding movement pattern.

Images