Control device

Abstract

To provide an automatic control technique that can reduce frequencies at which a use intervenes.SOLUTION: A control device relating to one aspect in the disclosure selects one or more control models out of a plurality of control models, using a selection model; derives a control command for a moving body, using the selected one or more control models; unless intervention operation is performed, controls motion of the moving body, in accordance with the derived control command and when the intervention operation is performed, discards the derived control command or overlaps with the derived control command, and controls motion of the moving body, in accordance with operation of intervention by a user. The selection model is configured to select one or more control models so as to avoid occurrence of intervention from a history of n intervention performed by the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a control device. [Background technology]

[0002] Patent Document 1 proposes a system for autonomous vehicle control configured to determine vehicle commands from a route, GPS data, and sensor data using a trained neural network. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2019-533810 Summary of the Invention [Problem to be solved by the invention]

[0004] One objective of the present disclosure is to provide an automatic control technique that can reduce the frequency of user intervention. [Means for solving the problem]

[0005] The control device according to the first aspect of the present disclosure includes a storage unit that stores a plurality of control models and a selection model, and a control unit. Each of the control models is configured to derive a control command to automatically control the movement of a moving object. The control unit is configured to execute the following: selecting one or more control models from the plurality of control models by the selection model; deriving a control command for the moving object using the one or more selected control models; controlling the operation of the moving object according to the derived control command when there is no user intervention operation; and controlling the operation of the moving object according to the user intervention operation by discarding the derived control command or overlapping the derived control command when there is user intervention operation. The selection model is configured to select the one or more control models from the user intervention history so as to avoid the occurrence of the user intervention. At least one of the plurality of control models and the selection model may use a trained machine learning model. A neural network may be used as the machine learning model. Effect of the Invention

[0006] According to the present disclosure, it is expected that the frequency of user intervention will be reduced. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an example of a situation to which the present disclosure is applied. [Diagram 2] FIG. 2 illustrates an example of a data configuration of the intervention history of the present disclosure. [Diagram 3] FIG. 3 illustrates an example of a hardware configuration of the control device of the present disclosure. [Figure 4] FIG. 4 shows an example of a processing procedure related to control by the control device of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Conventionally, rule-based autonomous driving systems are known. According to the method of Patent Document 1 and the like, an autonomous driving system can be constructed by using a trained machine learning model. However, the control of autonomous driving by a control model (rule-based model or machine learning model) does not necessarily suit the user. When the control of autonomous driving by a control model does not suit the user, an intervention by the user occurs, and autonomous driving is not performed. Manual driving is performed without implementing the control model. If the automatic driving based on the control model does not suit the user to a great extent, the automatic driving system is hardly used, and there is a risk that the user will frequently intervene. This problem can occur regardless of the type of vehicle. Furthermore, such problems do not occur only in situations where a vehicle is controlled. In terms of controlling movement, the same is true for moving bodies other than vehicles. Therefore, similar problems can occur in situations where any moving body other than a vehicle is controlled.

[0009] In contrast, the control device according to the first aspect of the present disclosure includes a storage unit that stores a plurality of control models and a selection model, and a control unit. Each control model is configured to derive a control command to automatically control the movement of the moving object. The control unit is configured to execute the following: selecting one or more control models from the plurality of control models by the selection model; deriving a control command for the moving object using the selected one or more control models; controlling the operation of the moving object according to the derived control command when there is no user intervention operation; and controlling the operation of the moving object according to the user intervention operation by discarding the derived control command or overlapping it with the derived control command when there is user intervention operation. The selection model is configured to select one or more control models from the user intervention history so as to avoid occurrence of user intervention.

[0010] In a first aspect of the present disclosure, the selection model is constructed to select a control model to avoid occurrence of user intervention based on an intervention history indicating a record of past intervention operations by a user. In a simple example, the selection model may be constructed to select a different control model in a situation where an intervention occurred in the past, without using the control model used in that situation for a new opportunity. In this way, the selection model constructed from the intervention history is used to adjust the control model used for automatic control, so that the derived control command can be adapted to the user. As a result, according to the present disclosure, it is expected that the frequency of user intervention will be reduced.

[0011] As another form of the control device according to the above aspect, one aspect of the present disclosure may be an information processing method or a program for implementing all or part of the above components, or may be a storage medium that stores such a program and is readable by a machine such as a computer. Here, the machine-readable storage medium is a medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action.

[0012] [1 Application example] 1 is a schematic diagram showing an example of a situation in which the present disclosure is applied. The control device 1 according to this embodiment is one or more computers configured to control the automatic movement of a target moving object M. In this embodiment, the control device 1 is mounted on the moving object M and holds a selection model 20 and a plurality of control models 30. Each control model 30 is configured to derive a control command to perform automatic control of the movement of the moving object M.

[0013] In this embodiment, the control device 1 selects one or more control models 35 from the multiple control models 30 by the selection model 20, and derives a control command 50 for the moving object M using the selected one or more control models 35. If the derived control command 50 is appropriate, automatic control by the control command 50 is likely to be accepted, and user intervention operation 55 is unlikely to occur, but if the derived control command 50 is inappropriate, it is assumed that user intervention operation 55 is likely to occur. For example, if the speed of the moving object M in automatic control by the control model 35 is appropriate for the user, the possibility of intervention of acceleration or deceleration occurring is low, but if the speed of the moving object M is slow or fast, it is assumed that the possibility of intervention of acceleration or deceleration occurring is high.

[0014] The control device 1 controls the movement of the moving object M according to the derived control command 50 when there is no user intervention operation 55. On the other hand, the control device 1 controls the movement of the moving object M according to the derived control command 50 when there is no user intervention operation 55 If there is an intervention operation 55, the control device 1 discards or overlaps the derived control command 50 to control the operation of the moving object M according to the user's intervention operation 55. When the intervention operation 55 occurs, the control device 1 records information indicating the intervention as an intervention history 60. The more user interventions occur, the more the intervention history 60 accumulates. The selection model 20 is configured to select one or more control models 35 from the user intervention history 60 so as to avoid occurrence of user intervention.

[0015] It is estimated that if the same automatic control as in the scene where the intervention occurred in the past is repeatedly performed in the same or similar scene where the intervention occurred, there is a high possibility that the user's intervention will occur again. Therefore, in this embodiment, the selection model 20 is constructed so as to select the control model 35 so as to avoid the occurrence of the user's intervention from the record of the user's past intervention operation (intervention history 60). That is, in this embodiment, after deploying a plurality of control models 30, the intervention history 60 is used to optimize the control model 35 used for the automatic control (i.e., selected by the selection model 20) from the tendency of the user's past intervention. This allows the control device 1 to avoid using a control model among the plurality of control models 30 that is estimated not to be suitable for the user in the target scene from the tendency of the past intervention, and to actively use a control model that is likely to be suitable for the user. As a result, according to this embodiment, it is possible to expect a reduction in the frequency of the user's intervention. In addition, by reducing the frequency of the user's operation, it is possible to expect a longer life for the operating tool (for example, a steering wheel, an accelerator pedal, a brake pedal, etc. when the moving body M is a vehicle) to suppress wear of the operating tool of the moving body M, and to expect a longer life for the operating tool.

[0016] (Mobile) If the moving body M can move automatically by mechanical control, the type of the moving body M may be appropriately selected according to the embodiment. The moving body M may be, for example, a moving device such as a vehicle, an aircraft, a ship, or a robot device. The aircraft may be at least one of an unmanned aircraft such as a drone and a manned aircraft. In one example, as shown in FIG. 1, the moving body M may be a vehicle. In this case, it is expected that the frequency of intervention by the user can be reduced when the vehicle is driven automatically. In addition, when the moving body M is a vehicle, the type of the vehicle (number of wheels, power source, size, etc.) may be selected arbitrarily. As a typical example, the moving body M may be an automobile having an automatic driving capability of level 2 or higher.

[0017] (Controlling the movement) In one example, controlling the operation of the target moving body M may be configured by directly controlling the target moving body M. In another example, the moving body M may be equipped with a dedicated control device such as a controller. In this case, controlling the operation of the target moving body M by the control device 1 may be configured by indirectly controlling the target moving body M by providing the derivation result to the dedicated control device. Note that the control device 1 may be deployed at any location. In one example, as shown in FIG. 1, the control device 1 may be mounted on the moving body M. In another example, the control device 1 may be disposed away from the moving body M and remotely control the moving body M.

[0018] (Control Model) Each control model 30 is constructed to derive a control command according to the environment of the moving body M. The environment is an event observed by at least one of the moving body M itself and the surroundings. In one example, at least a part of the environment may be observed by one or more sensors S arranged inside or outside the moving body M. The type of the sensor S is not particularly limited as long as it can observe any environment in which the moving body M moves, and may be appropriately selected according to the embodiment. In one example, the one or more sensors S may include a camera (image sensor), a radar, a LiDAR (Light Detection And Ranging), a sonar (ultrasonic sensor), an infrared sensor, a GNSS (Global Navigation Satellite System) / GPS (Global Positioning Satellite) module, etc.

[0019] As long as the control command can be derived from the environment of the moving body M, the input / output format of each control model 30 may be appropriately selected according to the embodiment. In one example, at least one of the control models 30 may be configured to derive the control command from observation data of a sensor at one or more points in time. In another example, at least one of the control models may be configured to derive the control command from a recognition result of the surrounding environment. In this case, the control device 1 may further include an analysis model that infers the recognition result of the surrounding environment from the observation data of the sensor. Alternatively, at least one of the control models may include an analysis model. The analysis model may be configured arbitrarily. In one example, the analysis model may be configured by a machine learning model. Other information may be added arbitrarily to the input of at least one of the control models. At least one of the control models may be configured to further accept input of arbitrary information such as a set speed, a speed limit, a position, map information, and navigation information.

[0020] Each control model 30 may be configured by at least one of a trained machine learning model and a rule-based model. The rule-based model is configured to match a given input (e.g., information indicating the environment such as observation data and a recognition result of the surrounding environment) with a rule, and derive a control command according to the result of the match (according to the applicable rule). The rule may be set manually or at least partially automatically. The machine learning model is configured to have one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used for the calculation of the desired inference (in this disclosure, derivation of the control command). Machine learning is to adjust (optimize) the value of the calculation parameter using learning data. The machine learning model may be configured by, for example, a neural network, a support vector machine, a regression model, a decision tree model, etc. The machine learning method may be appropriately selected according to the machine learning model to be adopted (e.g., backpropagation method, etc.).

[0021] As an example, at least one of the control models may be configured by a neural network. The structure of the neural network may be appropriately determined depending on the embodiment, and may be specified, for example, by the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationship between the nodes in each layer, etc. In one example, the neural network may have a recursive structure. In addition, the neural network may include any layer, such as a fully connected layer, a convolution layer, a pooling layer, a deconvolution layer, an unpooling layer, a normalization layer, a dropout layer, or a Long short-term memory (LSTM). The neural network may have any mechanism, such as an Attention mechanism. The neural network The network may include any model such as a graph neural network (GNN), a diffusion model, a generative model (e.g., Generative Adversarial Network, Transformer, etc.). When a neural network is used for the control model, the weight of the connection between each node included in the control model and the threshold value of each node are examples of the calculation parameters. Note that when a machine learning model is adopted, the control model may be configured with an end-to-end model structure.

[0022] At least one of the control models 30 is configured to derive a different control command for the other control models under the same environment or the same movement mode. This ensures that automatic control suited to the user can be executed by using another control model when automatic control by any of the control models 30 does not suit the user (intervention occurs). In other words, it is possible to ensure the possibility that a control model 35 suited to the user can be found by selecting the selection model 20.

[0023] In one example, the structures of the trained machine learning model and the rule-based model may be different. Therefore, the automatic control characteristics of the trained machine learning model and the rule-based model may differ from each other. Therefore, the multiple control models 30 may include one or more trained machine learning models and one or more rule-based models. This ensures variation in the automatic control characteristics, and increases the probability that a control model 30 that suits the user exists. As a result, As a result, it is expected that the frequency of user intervention will be reduced.

[0024] In another example, the characteristics of the automatic control of the trained machine learning model may depend on the learning conditions. Therefore, the characteristics of the machine learning model trained under different learning conditions (e.g., different training samples used for machine learning, changing the sampling probability, etc.) may differ. In addition, if the structure of the machine learning model is different, the inference result of the machine learning model may also differ. Therefore, the multiple control models 30 may include multiple trained machine learning models each having different learning conditions and / or structures. This can be expected to achieve the same effects as described above. When the multiple control models 30 include multiple rule-based models, the rules of each rule-based model may be different.

[0025] In addition, the control required for each scene may differ. Therefore, each control model 30 may be prepared for each scene. In one example, when the moving object M is a vehicle, a plurality of control models 30 that differ for each scene, such as lane changing, lane keeping, emergency stopping (EDSS: Emergency Driving Stop System), etc., may be prepared. 0 may be prepared. In this case, the multiple control models 30 may be configured to include at least two or more control models that can be used for automatic control in at least any of the same or overlapping scenes among the scenes that are targets of automatic control. That is, the control device 1 may hold two or more control models for the same or overlapping scenes. As an example, when the moving body M is a vehicle, the multiple control models 30 may include a first control model and a second control model for the same lane change. In this case, the first control model and the second control model may be configured to derive different control commands so that the control contents such as the timing, speed, and steering angle of the lane change are different from each other even in the same lane change scene.

[0026] (Control command) The control command (control command 50) relates to the operation of the moving body M. The configuration of the control command may be appropriately selected depending on the embodiment. In one example, the control command may be configured by acceleration, deceleration, steering, or a combination of these. Acceleration and deceleration may include gear changes. When at least one of acceleration, deceleration, and steering is included, the control command may be expressed by a path. Accordingly, each control model 30 may be expressed as a path planner. The control command may further include a command related to the operation of the moving body M. As an example, when the moving body M is a vehicle, the control command may include vehicle operations such as turn signals, hazards, horn, and communication processing (e.g., transmitting data to a center, making an emergency call, etc.).

[0027] In one example, each control model 30 may be configured to directly output a control command. In another example, each control model 30 may be configured to indirectly output a control command, and the control command may be obtained by performing any information processing (interpretation processing) on ​​the output of each control model 30. The control command may be configured to directly indicate a control amount (control instruction value, control output amount) of the moving object M, such as an accelerator control amount, a brake control amount, or a steering wheel steering angle. Alternatively, the control command may be configured to indirectly indicate a control amount of the moving object M, such as a path, a state after control, or the like. In this case, the control amount of the moving object M may be obtained from the control command by performing any information processing.

[0028] (Select one or more control models) Selecting one or more control models 35 may be configured by selecting one control model 35, or selecting two or more control models 35. Accordingly, deriving the control command 50 for the moving object M may be configured by deriving the control command 50 from the selected one control model 35, or by integrating the control commands obtained from each of the selected two or more control models 35 to derive the control command 50. The integration may be performed by any calculation such as summation, averaging, weighted averaging, etc. In this way, the control command 50 can be appropriately obtained.

[0029] (Discard or overlap) Overriding may be an immediate switch from automatic control to manual control, i.e. from control according to control commands 50 derived from one or more control models 35 to control according to user intervention operations 55. On the other hand, overlapping may be a gradual switch from automatic control to manual control (user control).

[0030] (Selection Model) The selection model 20 is configured to select a control model 35 that is estimated from the intervention history 60 to have a low probability of occurrence of an intervention or that an intervention will not occur. In one example, the selection rules of the selection model 20 may be adjusted so as not to use the control model used at that time in an environment that is the same as or close to the environment of the conditions under which an intervention occurred in the past (i.e., the range of the environment in which the intervention occurred in the past). Whether the conditions are the same or close may be inferred from environmental information such as observation data obtained by the sensor S.

[0031] In one example, the measure for defining the selection rule may be to select one appropriate control model (e.g., randomly, in a predetermined order, etc.) while avoiding the selection model in which the intervention occurred. Simply, the selection model 20 may be configured to select a control model other than the control model used at the time of the intervention in the past within the range of the environment in which the intervention occurred. Preferably, the selection model 20 may be configured to select one control model capable of deriving a control command that is the same as or close to the control command of the intervention operation by the user given at that time.

[0032] In another example, the measure for defining the selection rule may be to arbitrarily select two or more control models and derive an integration ratio of the two or more selected control models. The selection model 20 may be configured to select two or more control models within the scope of an environment in which an intervention has occurred in the past. The two or more control models may be configured of the control model used at that time and one or more other control models, or two or more other control models other than the control model used at that time. The integration ratio of the two or more control models may be calculated as appropriate so as to conform to a control command by the user intervention at that time.

[0033] As long as the intervention history 60 can be reflected in the selection of the control model 35, the configuration of the selection model 20 may be appropriately determined according to the embodiment. In one example, the selection model 20 may be configured by at least one of a rule-based model and a trained machine learning model. When the rule-based model is adopted, the selection model 20 may reflect the information of the intervention history 60 as it is in the selection rule. In one example, the selection model 20 may be configured by the information of the intervention history 60 and a criterion for setting the selection rule from the intervention history 60. The selection rule may be set manually or at least partially automatically. When the machine learning model is adopted, the training of the selection model 20 may be performed using information indicating the environment in which an intervention occurred in the past, which is indicated by the intervention history 60, as a training sample (input data), and using the control model (and the integration ratio) selected according to the above selection rule as a label (teacher signal, correct answer data). Thereby, a trained machine learning model (selection model 20) in which the selection rule according to the above measure is reflected can be generated.

[0034] The multiple control models 30 may include a trained machine learning model and a rule-based model. Accordingly, when the intervention history 60 includes a record (information) indicating that an intervention by a user occurred while the automatic control of the moving object M was being performed using the trained machine learning model, selecting one or more control models 35 may include selecting a rule-based model under conditions indicated by the record. The conditions indicated by the record may be the same as or a nearby range of the environment in which the intervention occurred. The nearby range may be arbitrarily defined. This allows the control model to be used in an environment in which the trained machine learning model is not suitable for the user. You can switch the control model from a trained machine learning model to a rule-based model.

[0035] (Intervention History) The intervention history 60 is composed of information on interventions by a user that occurred in the past while automatic control of the moving object M was being performed using at least one of the multiple control models 30 (records of interventions that occurred in the past). The data format of the intervention history 60 may be appropriately determined depending on the embodiment. The intervention history 60 may be held in any database format.

[0036] The items of information to be saved as the intervention history 60 are not particularly limited and may be appropriately selected depending on the embodiment, so long as they are capable of forming the selection model 20. In one example, the intervention history 60 may include the conditions of the environment in which the intervention occurred (location, route, etc.), and information for identifying the control model used in the environment with those conditions (e.g., an identifier, etc.).

[0037] Further, the intervention history 60 may further include information usable for improving the control model 30 (information for formulating rules or information for generating a data set to be used for machine learning) in addition to information used for forming the selection model 20. In one example, the intervention history 60 may further include an intervention operation by a user or a control command by the operation. When the control model 30 is configured as a machine learning model, machine learning (relearning, additional learning, etc.) for updating the control model 30 may be performed using information indicating the conditions of the environment in which the intervention occurred, which is included in the intervention history 60, as a training sample, and using the intervention operation by the user or the control command by the operation as a label. The machine learning for updating may be performed by the control device 1, or may be performed by a computer other than the control device 1 (for example, an external server).

[0038] FIG. 2 is a schematic diagram showing an example of a data configuration of the intervention history 60 according to the present embodiment. In the example of FIG. 2, the records of the intervention history 60 include a time stamp, identification information of the control model used (model used), the environmental conditions in which the intervention occurred, and information indicating the operation of the intervention. The time stamp indicates the date and time when the intervention occurred. The date and time indicated by the time stamp may be used to delete old records, identify records to be reflected in the selection model 20 (when the intervention history within a certain period is reflected in the selection model), and the like. The records of the intervention history 60 may be generated for each intervention operation. The unit of the intervention operation indicated by the record may be determined arbitrarily. In one example, one record (a sample of the intervention history) may be generated for each intervention operation. In another example, one record may be generated in response to multiple interventions.

[0039] (User) The user may refer to a specific user or an unspecified user. In one example, while the target user is using the control device 1, the intervention history 60 of only the target user may be reflected in the selection rule of the control model 35. In another example, the intervention history 60 of any user including users other than the target user may be reflected in the selection rule of the control model 35. Note that when the moving body M is a vehicle, the user is typically a driver.

[0040] [2 Configuration example] 3 is a schematic diagram showing an example of a hardware configuration of the control device 1 according to the present embodiment. The control device 1 according to the present embodiment is a computer to which a control unit 11, a storage unit 12, an external interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0041] The control unit 11 includes a CPU (Central Processing Unit), a RAM (Random Access Memory The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 may be configured with, for example, a hard disk drive, a solid state drive, etc. The storage unit 12 (as well as RAM and ROM) are memory resources. In this embodiment, the storage unit 12 stores various types of information such as a control program 81, control model data 300, selection model data 200, intervention history data 600, and the like.

[0042] The control program 81 is a program for causing the control device 1 to execute information processing (FIG. 4 described later) related to the control of the moving body M. The control program 81 includes a series of instructions for the information processing. The control model data 300 is configured to indicate information related to the control model 30. The selection model data 200 is configured to indicate information related to the selection model 20. In one example, at least one of the control model data 300 and the selection model data 200 may include information indicating values ​​of calculation parameters adjusted by machine learning. At least one of the control model data 300 and the selection model data 200 may further include information indicating the configuration of the machine learning model (e.g., the structure of a neural network, etc.). In another example, at least one of the control model data 300 and the selection model data 200 may include information indicating a rule-based model (rule). The intervention history data 600 may be appropriately configured to include information indicating the intervention history 60.

[0043] The external interface 13 may be, for example, a Universal Serial Bus (USB) port, a dedicated port, a wireless communication port, or the like, and is configured to connect to an external device in a wired or wireless manner. In this embodiment, the control device 1 may be connected to the sensor S via the external interface 13. The input device 14 is, for example, a device for inputting such as a mouse or a keyboard. The output device 15 is, for example, a device for outputting such as a display or a speaker. The input device 14 and the output device 15 may be integrally configured, for example, by a touch panel display or the like.

[0044] The drive 16 is a device for reading various information such as programs stored in the storage medium 91. At least one of the control program 81, the control model data 300, the selection model data 200, and the intervention history data 600 may be stored in the storage medium 91 instead of the storage unit 12 or together with the storage unit 12. The storage medium 91 is configured to accumulate various information (stored programs, etc.) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The control device 1 may acquire at least one of the control program 81, the control model data 300, the selection model data 200, and the intervention history data 600 from the storage medium 91. The storage medium 91 may be a disk-type storage medium such as a CD or a DVD, or may be a storage medium other than a disk-type storage medium such as a semiconductor memory (for example, a flash memory). The type of the drive 16 may be appropriately selected according to the type of the storage medium 91.

[0045] In addition, regarding the specific hardware configuration of the control device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), an ECU, (Electronic Control Unit), GPU (Graphics Processing Unit), etc. At least one of the external interface 13, the input device 14, the output device 15, and the drive 16 may be omitted. The control device 1 may be a computer designed specifically for the service to be provided, a general-purpose computer, a terminal device, or the like.

[0046] [3 Example of operation] FIG. 4 shows an example of a processing procedure for controlling a moving body M by the control device 1 according to this embodiment. The following processing procedure is an example of a control method executed by a computer. However, the following processing procedure is merely an example, and each step may be modified as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0047] In step S101, the control unit 11 operates as the acquisition unit 111 and acquires the observation data 125 of the sensor S. The control unit 11 may directly or indirectly acquire the observation data 125 from the sensor S. In step S102, the control unit 11 operates as the selection unit 112 and selects one or more control models 35 from the multiple control models 30 by the selection model 20. In one example, when the selection model 20 includes a rule-based model, the control unit 11 may derive a selection result of the control model 35 from an environment indicated by at least a part of the observation data 125 according to the rule. In another example, when the selection model 20 includes a trained machine learning model, the control unit 11 may input at least a part of the observation data 125 to the trained machine learning model and execute a calculation process of the trained machine learning model to acquire a selection result of the control model 35. In step S103, the control unit 11 operates as the derivation unit 113 and derives a control command 50 for the moving object M from at least a portion of the acquired observation data 125 using the selected one or more control models 35. As with the selection model 20, in one example, when the control model 35 includes a rule-based model, the control unit 11 may derive the control command 50 according to the rule. In another example, when the control model 35 includes a trained machine learning model, the control unit 11 may derive the control command 50 by executing a calculation process of the trained machine learning model.

[0048] In step S102, the control unit 11 may select one control model 35 or two or more control models 35. In response to this, in step S103, the control unit 11 may derive the control command 50 from the selected one control model 35. Alternatively, the control unit 11 may derive the control command 50 by integrating the control commands obtained from the two or more selected control models 35. In addition, the order of steps S102 and S103 may not be limited to this example. In another example, step S103 may be executed before step S102. In this case, the control unit 11 may derive control commands from all control models 30 that may be selected, regardless of whether they are selected or not. Then, the control unit 11 may extract the control command of the control model 35 selected in step S102 from the derived control commands.

[0049] In step S104, the control unit 11 operates as the intervention receiving unit 114 and determines whether or not there is an intervention operation 55 by the user. The control unit 11 may receive an intervention operation 55 by the user at any time while controlling the operation of the moving object M. If an intervention operation 55 is received (i.e., there is an intervention operation 55), the control unit 11 advances the process to step S106. On the other hand, if an intervention operation 55 is not received (there is no intervention operation 55), the control unit 11 advances the process to step S105.

[0050] In step S105, the control unit 11 operates as the operation control unit 116, and controls the operation of the moving object M in accordance with the derived control command 50. When the control of the operation is completed, the control unit 11 advances the process to the next step S108.

[0051] In step S106, the control unit 11 operates as the operation control unit 116, discards the derived control command 50 or overlaps the derived control command 50, and controls the operation of the moving body M according to the user's intervention operation 55. In step S107, the control unit 11 operates as the history generation unit 115, generates information indicating the user's intervention operation 55, and stores the generated information in a predetermined storage area as the intervention history 60. The predetermined storage area (storage destination) may be selected arbitrarily. In one example, the predetermined storage area may be a RAM, the storage unit 12, the storage medium 91, or the like. When the control device 1 is configured to be communicable, the predetermined storage area may be an external computer. Note that the processing timing of step S107 is not limited to such an example and may be changed as appropriate. When the recording of the intervention history 60 (updating the intervention history data 600) is completed, the control unit 11 advances the processing to the next step S108.

[0052] In step S108, the control unit 11 judges whether or not to end the control of the moving object M. The criteria for the judgment may be set arbitrarily. In one example, while the moving object M is activated, the control unit 11 may judge not to end the control of the moving object M, whereas in response to an arbitrary end instruction (e.g., an end operation by the user via the input device 14), the control unit 11 may judge to end the control of the moving object M. If it is judged not to end the control, the control unit 11 returns to step S101 and executes the process again from step S101. On the other hand, if it is judged to end the control, the control unit 11 advances the process to step S109.

[0053] In step S109, the control unit 11 operates as the model update unit 117 and updates the selection model 20 with the generated intervention history 60. In one example, the control unit 11 may update the selection model 20 according to the above-mentioned measure. As a result, the selection model 20 is configured to select one or more control models 35 from the user intervention history 60 so as to avoid the occurrence of user intervention. In the example of FIG. 4, step S109 is executed every time control ends, and the selection model 20 is updated. However, the processing timing of step S109 is not limited to such an example, and may be appropriately determined according to the embodiment. In another example, the control unit 11 may update the selection model 20 by executing step S109 at regular intervals (for example, once a month, etc.). When the update of the selection model 20 is completed, the control unit 11 ends the processing procedure of the control device 1 according to this operation example.

[0054] [Features] In this embodiment, the selection model 20 is constructed by the processes of steps S107 and S109 so as to select a control model 35 from the record of the user's past intervention operations to avoid the occurrence of user intervention. By using this selection model 20 in step S102, the control device 1 can avoid using a control model among the multiple control models 30 that is presumed to be inappropriate for the user in the target scene based on the tendency of past interventions, and can actively use a control model that is likely to be appropriate for the user. As a result, according to this embodiment, a reduction in the frequency of user intervention can be expected.

[0055] [4 Variations] Although the embodiment of the present disclosure has been described in detail above, the above description is merely an example of the present disclosure in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present disclosure. The processes and means described in the present disclosure can be freely combined and implemented as long as no technical contradiction occurs. [Explanation of symbols]

[0056] 1...Control device, 11: control unit, 12: storage unit, 20: Selection model, 30-35: Control model, 50...Control command, 55...Operation, 60...Intervention history M: moving object, S: sensor

Claims

1. A storage unit that stores a plurality of control models and selection models; and Control unit, A control device comprising: Each of the control models is configured to derive a control command for automatically controlling a movement of a moving object; The control unit is selecting one or more control models from the plurality of control models according to the selection model; deriving control commands for the moving object using the one or more selected control models; Controlling the movement of the moving object according to the derived control command in the absence of user intervention; and When an operation of intervention by the user is performed, the derived control command is discarded or overlapped with the derived control command, and an operation of the moving object is controlled in accordance with the operation of intervention by the user. and The selection model is configured to select the one or more control models so as to avoid occurrence of an intervention by the user based on the intervention history of the user. Control device.

2. the plurality of control models include a trained machine learning model and a rule-based model; The control device according to claim 1 .

3. selecting the one or more control models includes, when the intervention history includes a record indicating that an intervention by the user occurred while performing automatic control of the vehicle using the trained machine learning model, selecting the rule-based model under a condition indicated by the record. The control device according to claim 2.

4. Selecting the one or more control models comprises: Selecting one control model, or selecting two or more control models; and Deriving a control command for the moving object includes: Obtaining the control instructions from the one control model; or deriving the control command by integrating control commands obtained from the two or more control models; Consists of: The control device according to claim 1 .

5. The moving object is a vehicle. A control device according to any one of claims 1 to 4.

Images