Control device

Abstract

Provided is a control device that suppresses an increase in unsuitability of automatic control caused by model changes. A control device according to one aspect of the present disclosure generates a first path and a second path using a first control model and a second control model, evaluates the deviation between the generated first path and second path using a suitability reference, and generates a final path from the first path and second path by reflecting the evaluation result. The movement of the moving body is controlled in accordance with the generated final path. The suitability reference is adjusted according to the user feedback such that the first route is prioritized as the user does not perform the interventional operation, and the second route is prioritized as the user performs the interventional operation.

Description

Technical Field

[0001] The present disclosure relates to control technology for mobile objects such as autonomous vehicles. Background Art

[0002] Patent Document 1 proposes an autonomous vehicle control system that uses a trained neural network and determines vehicle commands based on a route, GPS data, and sensor data.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application No. 2019-533810 Summary of the invention

[0006] Problems to be solved by the invention

[0007] One of the objects of the present disclosure is to provide a control technology for suppressing an increase in inappropriateness (mismatch) of automatic control caused by a model change.

[0008] Technical solutions to solve problems

[0009] The control device of the first scheme of the present disclosure includes a storage unit storing a first control model and a second control model, and a control unit. The first control model is a control model newly equipped with respect to the second control model. The control unit is configured to perform the following processing: generating a first path by using the first control model; generating a second path by using the second control model; evaluating the divergence between the generated first path and the second path using a suitability (matching) benchmark; reflecting the result obtained by evaluating the divergence and generating a final path from the first path and the second path; and controlling the movement of the moving body according to the generated final path. The suitability benchmark is adjusted by the feedback of the user in such a way that the less the user performs intervention operation, the more the first path is preferentially generated to generate the final path, and the more the user performs intervention operation, the more the second path is preferentially generated to generate the final path. In addition, at least one of the first control model and the second control model can be composed of a neural network, and the machine learning method can use deep learning.

[0010] Effects of the Invention

[0011] According to the present disclosure, it is expected that incompatibility (mismatch) in automatic control caused by model change can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 An example of a scenario in which the present disclosure is applied is schematically shown.

[0013] Figure 2 An example of the data structure of the intervention history of the present disclosure is schematically shown.

[0014] Figure 3 An example of the hardware configuration of the control device of the present disclosure is schematically shown.

[0015] Figure 4 An example of a processing procedure related to control performed by the control device of the present disclosure is shown.

[0016] Description of Reference Numerals

[0017] 1…control device, 11…control unit, 12…storage unit, 20…suitability standard, 30…first control model, 35…second control model, 40…first path, 45…second path, 50…final path, 55…(intervention) operation, 60…intervention history, M…moving body, S…sensor. DETAILED DESCRIPTION

[0018] In the past, there are known autonomous driving systems based on rule bases. In addition, according to the methods of Patent Document 1, it is possible to construct an autonomous driving system by using a trained machine learning model. However, the inventors of the present application have found that there are the following problems in these previous methods. For example, it is assumed that a scenario of model changes such as "changing from a rule base model to a machine learning model" and "updating a machine learning model" is performed. Through this model change, the content of the control instruction (path) under the same environment may change (for example, "the timing of executing a lane change changes", etc.). Depending on the situation, the instruction content may change significantly due to the model change. For example, in the case of major model changes such as "changing the structure of the model" (for example, "changing from a rule base model to a machine learning model", "adopting a machine learning model with a different structure", etc.), "additional learning or relearning for a large amount of new training data", the possibility of a significant change in the instruction content is high. In the case where the instruction content has changed significantly, the control of the autonomous driving based on the new control model after the change may not be suitable for the user (matching the user) compared to the original control model before the change. In one example, the user may feel uncomfortable with the automatic control based on the new control model because he is accustomed to the automatic control based on the original control model (that is, it is no longer suitable for the user). That is to say, as the model changes, it is possible that the automatic driving based on the control model is no longer suitable for the user (although it is suitable under the original control model, it is not suitable under the new control model). In the case where the control of the automatic driving based on the control model is not suitable for the user, the user's intervention operation will occur instead of implementing the automatic driving, and manual driving will be implemented. The increase in the unsuitability of automatic control caused by the model change may lead to frequent user intervention operations. In addition, this problem may occur regardless of the type of vehicle. In addition, the occurrence of such a problem is not limited to the scenario of controlling the vehicle. In terms of controlling movement, the same is also true for moving objects other than vehicles. Therefore, the same problem may also occur in the scenario of controlling all moving objects other than vehicles.

[0019] In contrast, the control device of the first scheme of the present disclosure includes a storage unit for storing a first control model and a second control model, and a control unit. The first control model is a control model newly equipped relative to the second control model. The control unit is configured to perform the following processing: generating a first path by using the first control model; generating a second path by using the second control model; evaluating the divergence between the generated first path and the second path using a suitability benchmark; reflecting the result obtained by evaluating the divergence and generating a final path from the first path and the second path; and controlling the movement of the moving body according to the generated final path. The suitability benchmark is adjusted by user feedback in such a manner that the less the user performs intervention operation, the more the first path is prioritized in generating the final path, and the more the user performs intervention operation, the more the second path is prioritized in generating the final path.

[0020] In the first scheme of the present disclosure, both a new control model (first control model) and an original control model (second control model) are maintained. The suitability criterion is adjusted according to user feedback in the manner of "for automatic control based on the first control model, the more the user intervenes, the more priority is given to the second control model, and the less the user intervenes, the more priority is given to the first control model." Thus, in the case of user intervention, the usage ratio of the second control model is increased, and in the case of no user intervention, the usage ratio of the first control model is increased. As a result, the usage scenario of the first control model is limited to (restricted to) a range of "no incompatibility (mismatch)", so that it can be expected to suppress the "increase in incompatibility (mismatch) of automatic control caused by model changes."

[0021] In addition, as another form of the control device of the above scheme, one aspect of the present disclosure may also be an information processing method that realizes all or part of the above components, or a program, or a storage medium readable by a device such as a computer storing such a program. Here, a storage medium readable by a device refers to a medium that stores information such as a program through electrical, magnetic, optical, mechanical or chemical action.

[0022] [1 Application example]

[0023] Figure 1 An example of a scenario to which the present disclosure is applied is schematically shown. The control device 1 of the present embodiment is one or more computers configured to control the automatic movement of a moving body M as an object. In the present embodiment, the control device 1 is mounted on the moving body M and holds a first control model 30 and a second control model 35. Each control model (30, 35) is configured to derive a path for executing automatic control of the movement of the moving body M. The first control model 30 is a control model newly equipped with respect to the second control model 35.

[0024] In the present embodiment, the control device 1 generates the first path 40 by using the first control model 30. In addition, the control device 1 generates the second path 45 by using the second control model 35. The control device 1 evaluates the divergence between the generated first path 40 and the second path 45 using the suitability criterion 20. The control device 1 reflects the result of the divergence evaluation and generates the final path 50 from the first path 40 and the second path 45.

[0025] In the absence of the user's intervention operation 55, the control device 1 controls the motion (movement) of the moving body M according to the generated final path 50. On the other hand, in the presence of the user's intervention operation 55, the control device 1 invalidates the generated final path 50 or overwrites the final path 50, and controls the motion of the moving body M according to the user's intervention operation 55. The suitability criterion 20 is adjusted based on the user's feedback in such a way that the first path 40 is prioritized in generating the final path 50 when the user does not perform the intervention operation 55, and the second path 45 is prioritized in generating the final path 50 when the user performs the intervention operation 55.

[0026] In the present embodiment, the control device 1 maintains both a new control model (the first control model 30) and an original control model (the second control model 35). The first control model 30 is a newer control model than the second control model 35. Therefore, basically, the first control model 30 is used in preference to the second control model 35. That is, in the generation of the final path 50, the first path 40 is given priority. However, control based on the new control model (the first control model 30) is not necessarily suitable for users who are accustomed to control based on the original (original) control model (the second control model 35). In particular, in the case where the divergence between the first path 40 and the second path 45 is large, there is a high possibility that control based on the first path 40 is not suitable for users who are accustomed to the second path 45 (feeling a sense of disharmony). In the case of inappropriateness, user intervention operation 55 may occur.

[0027] In the present embodiment, the user's intervention operation 55 is used as an index to evaluate whether the control based on the new control model (the first control model 30) is suitable for the user. In the case of intervention, it is evaluated that the control based on the new control model is not suitable for the user, and therefore, the suitability criterion 20 is adjusted in a manner to increase the use ratio of the original control model (the second control model 35) (that is, give priority to the second path 45). On the other hand, in the case of no intervention, it is evaluated that the control based on the new control model is suitable for the user, and therefore, the suitability criterion 20 is adjusted in a manner to increase the use ratio of the new control model (the first control model 30) (that is, give priority to the first path 40). In this way, the use range of the first control model 30 can be limited to (limited to) a range where "no incompatibility occurs", and as a result, it can be expected to suppress the "increase in incompatibility of automatic control caused by model change".

[0028] (Moving body)

[0029] The type of the mobile body M can be appropriately selected according to the embodiment as long as it can be automatically moved by machine control. The mobile body M can be, for example, a vehicle, an aircraft, a ship, a robot device, or other movable device. The aircraft can be at least one of an unmanned aircraft such as an unmanned aircraft and a manned aircraft. In one example, Figure 1 Thus, the moving body M may be a vehicle. In addition, the type of vehicle (number of wheels, power source, size, etc.) may be selected arbitrarily. As a typical example, the moving body M may be an automobile having a level 2 or higher autonomous driving capability.

[0030] (Control the action)

[0031] In one example, "controlling the action (movement) of the moving body M as an object" includes "directly controlling the moving body M as an object". In another example, the moving body M may have a dedicated control device such as a controller. In this case, "the control device 1 controls the action of the moving body M as an object" may include "indirectly controlling the moving body M as an object by providing the derived result to the dedicated control device". The control device 1 may be installed in any place. In one example, as Figure 1 As shown, the control device 1 may be mounted on the mobile body M. In another example, the control device 1 may be configured separately from the mobile body M to remotely control the mobile body M. The control device 1 may be configured to be able to perform "switching from the automatic control mode of the mobile body M to the manual control mode" and "switching from the manual control mode to the automatic control mode" at any timing through any operation.

[0032] (path)

[0033] Regarding the path (output of the control model), as long as it can control the movement of the mobile body M, its method (form) is not particularly limited and can be appropriately determined according to the implementation method. In one example, the path (40, 45, 50) can be composed of one or more control instructions. The control instruction can be configured to represent the control amount of the mobile body M. In another example, the path (40, 45, 50) can be configured to represent the future movement path of the mobile body M, and is used to derive one or more control instructions. Accordingly, one or more control instructions can be determined using any method based on the final path 50. The control model (30, 35) can also be called a path planner.

[0034] The control instruction is related to the action of the mobile body M. The composition of the control instruction can be appropriately selected according to the implementation method. In one example, the control instruction can be composed of acceleration, deceleration, steering or a combination thereof. Acceleration and deceleration may include gear changes. The control instruction can be composed, for example, to represent the control quantity (control instruction value, control output quantity) of the mobile body M such as the accelerator control quantity, the brake control quantity, and the steering wheel steering angle. In addition, the control instruction may also include instructions related to the operation of the mobile body M. As an example, when the mobile body M is a vehicle, the control instruction may include vehicle operations such as direction indication, hazard warning, honking, communication processing (for example, "sending data to the center", "sending an emergency call", etc.).

[0035] (Void or overwrite)

[0036] Overriding (ignoring) may be an immediate switch from automatic control to manual control, that is, an immediate switch from automatic control based on final path 50 to manual control according to user intervention 55. On the other hand, overwriting (overlapping) may be a gradual switch from automatic control to manual control (user control).

[0037] (Control Model)

[0038] The control model (30, 35) is constructed in such a way as to derive a path according to the environment of the mobile body M. The environment is a phenomenon observed in at least one of the mobile body M itself and the surroundings. In one example, at least a part of the environment can be observed by one or more sensors S arranged inside or outside the mobile body M. The sensor S can be any environment in which the mobile body M moves, and its type is not particularly limited and can be appropriately selected according to the implementation method. In one example, the one or more sensors S may include a camera (image sensor), a radar, a LiDAR (Light Detection And Ranging), a sonar (ultrasonic sensor), an infrared sensor, a GNSS (Global Navigation Satellite System) / GPS (Global Positioning Satellite) module, etc.

[0039] As long as the path can be derived from the environment of the mobile body M, the form of the input and output of the control model (30, 35) can be appropriately selected according to the implementation. In one example, at least one of the control models (30, 35) can be configured to derive the path based on the observation data (sensor data) of the sensor S at more than one time point. In another example, at least one of the control models (30, 35) can be configured to derive the path based on the recognition result of the surrounding environment. In this case, the control device 1 can also include an analytical model that infers the recognition result of the surrounding environment based on the observation data of the sensor S. Or, at least one of the control models (30, 35) can include such an analytical model. The analytical model can be arbitrarily configured by a trained machine learning model or the like. Other information can be arbitrarily added to the input of at least one of the control models (30, 35). At least one of the control models (30, 35), for example, can be configured to also accept the input of arbitrary information such as a set speed, a speed limit, map data, navigation information (route data), etc. In the case where the mobile body M is a vehicle, the arbitrary information can include driving data. The arbitrary information can be appropriately obtained from a navigation device, a vehicle-mounted sensor, or the like. In addition, the control model (30, 35) can be configured as a direct output path or an indirect output path. In the latter case, the path can be obtained by performing arbitrary information processing (interpretation processing) on ​​the output of the control model (30, 35).

[0040] The control model (30, 35) can be composed of at least one of a trained machine learning model and a rule base model. The rule base model is constructed to compare the input provided (for example, information representing the environment such as observation data, recognition results of the surrounding environment, etc.) with the rules, and derive the path according to the comparison results (according to the appropriate rules). The rules can be set manually or at least partially automatically. The machine learning model is constructed to have one or more operation parameters that can be adjusted using machine learning. One or more operation parameters are used for the operation of inference as the purpose (in the present disclosure, the derivation of the path). Machine learning is the use of learning data to adjust (optimize) the value of the operation parameter. The machine learning model can be composed of, for example, a neural network, a support vector machine, a regression model, a decision tree model, etc. The machine learning method can be appropriately selected according to the machine learning model adopted (for example, error back propagation method, etc.). Machine learning can include teacher learning, teacher-free learning, and reinforcement learning.

[0041] In one example, at least one of the control models (30, 35) may be at least partially composed of a neural network. The structure of the neural network may be appropriately determined according to the implementation method. The structure of the neural network may be determined, for example, by the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) contained in each layer, the connection relationship between the nodes of each layer, etc. In one example, the neural network may have a recursive structure. In addition, the neural network may include, for example, any layers such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, a depooling layer, a normalization layer, a random inactivation layer, and an LSTM (Long short-term memory). The neural network may have any mechanism such as an Attention mechanism. The neural network may include any model such as a GNN (Graph neural network), a diffusion model, a generative model (for example, a Generative Adversarial Network, a Transformer, etc.). When a neural network is used for a control model, the weights of the connections between the nodes contained in the control model and the thresholds of each node are examples of operation parameters. In addition, when a machine learning model is used, an example of a control model can be constructed by an end-to-end model.

[0042] In addition, the first control model 30 and the second control model 35 are constructed for the same purpose. That is, the first control model 30 and the second control model 35 are constructed in a manner that can complete the same type of movement in the same or repeated environment. The control required for each scene may be different, so the first control model 30 and the second control model 35 can be prepared for each scene. In one example, when the moving body M is a vehicle, the first control model 30 and the second control model 35 can be prepared for each scene, such as lane change, lane keeping, emergency stop (EDSS: Emergency Driving Stop System), merging and giving way, automatic parking, etc.

[0043] (New Equipment)

[0044] The first control model 30 can be prepared as a new control model relative to the second control model 35. That is, the first control model 30 can be a new control model after the change, and the second control model 35 can be the original control model before the change. In one example, the change of the control model can be a replacement of the system. Accordingly, the change of the control model can include, for example, "replacing from a rule base model to a trained machine learning model", "replacing from a trained machine learning model to a rule base model", "changing the structure of the machine learning model", etc. In another example, the change of the model can be a version upgrade. Accordingly, the change of the model can include, for example, "equipping a new rule base model with modified rules", "equipping a new trained machine learning model that has been updated through additional learning or relearning", etc. In addition, each control model (30, 35) (model data) can be appropriately provided from an external server, etc.

[0045] In one example, the first control model 30 may be composed of a trained machine learning model. The second control model 35 may be composed of a rule base model. The structures of the trained machine learning model and the rule base model may be different. Therefore, the characteristics of the automatic control of the trained machine learning model and the rule base model may be different from each other. As a result, a situation may occur in which "when updating the automatic control system from the rule base model to the trained machine learning model, the control based on the trained machine learning model is not suitable for users who are accustomed to the control based on the rule base model." According to an example of the present embodiment, even when such an automatic control system is updated, through the role of the mediation process based on the above-mentioned suitability benchmark 20, it is expected to suppress the "increase in the inappropriateness of the automatic control caused by the model change."

[0046] In another example, the first control model 30 may be composed of a trained machine learning model after the update. On the other hand, the second control model 35 may be composed of a trained machine learning model before the update. "Update" may include at least one of the replacement and version upgrade of the above-mentioned system. "Update" may include any processing that can change the output of the control model. "Update" may include, for example, "correcting the value of a parameter through additional learning or relearning", "changing the structure of the model", etc. According to an example of the present embodiment, it can be expected to suppress the "inappropriate increase in automatic control caused by model update".

[0047] In addition, the number of control models maintained by the control device 1 may not necessarily be limited to two. The number of the first control models 30 may not be limited to one. The number of the second control models 35 may not be limited to one. "Using the first control model 30" may include "using more than one first control model 30". "Using the second control model 35" may include "using more than one second control model 35". In another example, the control device 1 may maintain more than three control models. It may be that, in a manner corresponding to the first control model 30 and the second control model 35, the order of the control model with more priority when there is more intervention and the control model with more priority when there is less intervention is specified among the more than three control models. The control device 1 may also be configured to perform mediation processing using the suitability criterion 20 among the more than three control models to generate the final path 50. That is, "generating the final path 50 from the first path 40 and the second path 45" may include "generating the final path 50 from more than three paths". In addition, typically, the first control model 30 is a control model of the next generation (next generation) of the second control model 35. However, the relationship between the first control model 30 and the second control model 35 is not limited to such an example. The generation difference between the first control model 30 and the second control model 35 may be two or more.

[0048] (Suitability Criteria)

[0049] The suitability criterion 20 is defined in a manner to evaluate whether the automatic control based on the first control model 30 is suitable for the user in response to the divergence generated between the first control model 30 and the second control model 35. As long as such suitability can be evaluated, the suitability criterion 20 can be appropriately set. Simply put, the suitability criterion 20 can be composed of thresholds for physical indicators. In one example, the suitability criterion 20 can be composed of thresholds for speed, acceleration, timing of changes, or a combination thereof for movement (straight ahead, right turn, left turn, etc.). In this case, "adjusting the suitability criterion 20" may include "updating (i.e., correcting) thresholds". The suitability criterion 20 can be defined in a manner to evaluate the suitability of the user based on the recognition of at least one of the moving scene (lane change, etc.) and the moving environment (e.g., location, route, surrounding environment, etc.).

[0050] (Generation of the final path)

[0051] “Generating the final path 50 from the first path 40 and the second path 45” may include: (I) selecting one of the first path 40 and the second path 45 as the final path 50 based on the evaluation result, or (II) generating the final path 50 by integrating the first path 40 and the second path 45. The final path 50 is a finally derived path used for controlling the moving body M.

[0052] In the case of adopting (I), "giving priority to a path" may be "selecting the path of the one with priority". As a simple example, assume a case where the suitability criterion 20 is provided as a threshold value for the speed of movement. In this case, the control device 1 may select the second path 45 as the final path 50 when the deviation between the first path 40 and the second path 45 regarding the speed exceeds the threshold value of the suitability criterion 20, and may select the first path 40 as the final path 50 in other cases. The threshold value of the suitability criterion 20 may be adjusted to a larger value as the less intervention operation 55 occurs, thereby making it easier for the first path 40 to be selected as the final path 50. On the other hand, the threshold value of the suitability criterion 20 may be adjusted to a smaller value as the more intervention operation 55 occurs, thereby making it easier for the second path 45 to be selected as the final path 50.

[0053] In the case of adopting (II), "giving priority to the path" may be "increasing the ratio of the prioritized path during integration". The suitability criterion 20 may also include the integrated ratio together with the above-mentioned threshold or instead of the threshold. "Adjusting the suitability criterion 20" may include "adjusting the value of the integrated ratio". The less the intervention operation 55 occurs, the higher the ratio of the first path 40 is, and the more the intervention operation 55 occurs, the higher the ratio of the second path 45 is. "Using the suitability criterion 20 to evaluate the deviation" may include "determining the integration ratio of each path (40, 45) according to the suitability criterion 20". The integration ratio may be determined according to the degree of deviation between the first path 40 and the second path 45, or may be determined regardless of the degree of deviation. Integration can be implemented by any operation such as total (sum), average, weighted average, etc. In one example, in the above situation, the control device 1 may generate the final path 50 by determining the ratio of each path (40, 45) according to the suitability criterion 20 and integrating the first path 40 and the second path 45 at the determined ratio. In another example, in the case where the suitability criterion 20 includes the above threshold, in the above situation, the control device 1 may integrate the first path 40 and the second path 45 to generate the final path 50 when the speed difference between the first path 40 and the second path 45 exceeds the threshold of the suitability criterion 20. In other cases, the first path 40 may be selected as the final path 50.

[0054] In addition, "using the suitability benchmark 20 to evaluate the deviation" and "generating the final path 50 based on the evaluation result" can be performed by at least one of the rule base and the calculation model (machine learning model) library. When the rule base is used, the control device 1 can use the suitability benchmark 20 to evaluate the deviation between the first path 40 and the second path 45, and perform the above-mentioned processing (I) or (II) according to the evaluation result of the deviation. When the calculation model library is used, the control device 1 can evaluate the deviation and the above-mentioned processing (I) or (II) as a series of processing of the calculation model. In this case, the suitability benchmark 20 can be loaded (included) in the calculation model.

[0055] In addition, in the present embodiment, when the user's intervention operation 55 occurs, the suitability criterion 20 is adjusted in a manner to increase the usage ratio of the second control model 35 (the second path 45). "Increasing the usage ratio" in the above-mentioned case (I) is to make it easier to select the second path 45 (for example, to reduce the threshold), and in the above-mentioned case (II) is to increase the integration ratio of the second path 45. In one example, after increasing the usage ratio of the second control model 35 in this way, the suitability criterion 20 may be adjusted in a manner to gradually increase the usage ratio of the first control model 30 within a range where the user's intervention does not occur. In this way, the user's sense of disharmony with the control based on the new control model (the first control model 30) may be gradually eliminated, so that the user can enjoy the benefits of the new control model.

[0056] In addition, sometimes, even after the priority of the second path 45 is increased due to the occurrence of the user's intervention operation 55, the user's intervention operation 55 continues to occur. In this case, in one example, the control device 1 can continue to increase the priority of the second path 45. In another example, the control device 1 can also use the calculation model to generate the final path 50 from the first path 40 and the second path 45. The control device 1 can also adjust the value of the parameter of the calculation model (for example, the integration ratio of the first path 40 and the second path 45) in a manner to obtain an output suitable for the path based on the intervention operation 55. In addition, even if the second path 45 is given the maximum priority, the automatic control suitable for the user is not performed. It is possible that the second control model 35 is not suitable for the user in the first control model 30 and the second control model 35 are not suitable for the user. As a result, it may be difficult to generate a path suitable for the user in the first control model 30 and the second control model 35. Therefore, in one example, when such a situation occurs, the control device 1 can send information indicating the environmental conditions during the intervention and the intervention operation (or path) to the computer (e.g., an external server) that provides the control model, thereby urging further updating of the control model.

[0057] (user)

[0058] The user may refer to a specific user or an unspecified user. In one example, the suitability benchmark 20 may be distinguished for each user. In this case, the suitability benchmark 20 for the target user may only reflect the feedback of the target user. In another example, the suitability benchmark 20 may not be distinguished. In this case, the feedback of any user including other users other than the target user may be reflected in the suitability benchmark 20, and the obtained suitability benchmark 20 may be applied when the target user uses the moving body M. In addition, when the moving body M is a vehicle, typically, the user is the driver.

[0059] (feedback)

[0060] In one example, the user's feedback (the presence or absence of the intervention operation 55) can be immediately reflected in the suitability benchmark 20. In another example, the user's feedback can be stored as the intervention history record 60, and the obtained intervention history record 60 can be reflected in the suitability benchmark 20 afterwards. The user's feedback can be reflected in the adjustment of the suitability benchmark 20 (for example, the update of the above-mentioned threshold) based on the recognition of at least one of the scene of movement and the environment of movement. In this case, the conditions (scene, environment, etc.) for the occurrence of intervention can be appropriately determined.

[0061] In the case where user feedback is stored as intervention history record 60, the items of information stored as the object of intervention history record 60 are not particularly limited as long as the suitability benchmark 20 can be adjusted, and can be appropriately selected according to the implementation method. In one example, in the case of identifying the conditions for intervention and adjusting the suitability benchmark 20, intervention history record 60 may include information indicating the conditions for intervention. In addition, in the case where a trained machine learning model is used for a control model (30, 35) or there is a possibility of this, intervention history record 60 may include information for generating learning data that can be used in the training of the machine learning model. In one example, in order to determine the model to be improved, intervention history record 60 may include information (for example, an identifier, etc.) for identifying the control model used when the intervention occurred. In addition, intervention history record 60 may also include the user's intervention operation 55 or a path based on the operation 55. The information indicating the conditions of the environment in which the intervention occurs contained in intervention history record 60 can be used as training data (input data), and the user's intervention operation 55 or a path based on the operation 55 can be used as correct answer data (teacher signal, label). In this case, the control model composed of the machine learning model can be updated (relearned, additionally learned, etc.) based on the obtained intervention history 60. The machine learning for updating can be executed in the control device 1, or can be executed by a computer outside the control device 1 (for example, an external server). In addition, the data format of the intervention history 60 can be appropriately determined according to the implementation method. The intervention history 60 can be maintained in any database format.

[0062] Figure 2 An example of the data structure of the intervention history record 60 of this embodiment is schematically shown. Figure 2In the example, the record of intervention history record 60 includes a timestamp, identification information of the control model used (usage model), the conditions of the environment in which the intervention occurred, and information about the intervention operation. The timestamp indicates the date and time when the intervention occurred. The date and time indicated by the timestamp can be used for deleting old records, determining the records reflected in the suitability benchmark 20 (when the intervention history records within a certain period are reflected in the suitability benchmark 20), etc. In addition, the record of intervention history record 60 can be generated for each intervention operation. The unit of intervention operation represented by the record can be determined arbitrarily. In one example, one record (a sample of intervention history record) can be generated for each intervention operation. In another example, one record can also be generated based on multiple interventions.

[0063] [2 Configuration example]

[0064] Figure 3 An example of the hardware configuration of the control device 1 of the present embodiment is schematically shown. The control device 1 of the present embodiment is a computer in which a control unit 11, a storage unit 12, an external interface (I / F) 13, an input device 14, an output device 15, and a driver 16 are electrically connected.

[0065] The control unit 11 is configured to include a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc., and performs arbitrary information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 can be composed of, for example, a hard disk drive, a solid state drive, etc. The storage unit 12 (as well as RAM, ROM) is an example of a memory resource and is an example of a storage unit of the present disclosure. In the present embodiment, the storage unit 12 stores various information such as the control program 81, the first model data 300, the second model data 350, the suitability reference information 200, and the intervention history data 600.

[0066] The control program 81 is used to cause the control device 1 to execute information processing related to the control of the moving body M (described later). Figure 4) program. The control program 81 includes a series of commands for the information processing. The first model data 300 is configured to represent information related to the first control model 30. The second model data 350 is configured to represent information related to the second control model 35. In one example, when a machine learning model is adopted for the control model, the model data (300, 350) may include information representing the value of the operation parameter adjusted by machine learning. Depending on the circumstances, the model data (300, 350) may also include information representing the composition of the machine learning model (for example, the structure of the neural network, etc.). In another example, when a rule base model is adopted for the control model, the model data (300, 350) may include information representing the rule base model (rules). The suitability benchmark information 200 can be appropriately configured in a manner to represent the suitability benchmark 20. The intervention history data 600 can be appropriately configured in a manner to represent the intervention history 60.

[0067] The external interface 13 may be, for example, a USB (Universal Serial Bus) port, a dedicated port, a wireless communication port, etc., and is configured to be connected to an external device by wire or wirelessly. In the present embodiment, the control device 1 may be connected to the sensor S via the external interface 13. The input device 14 may be, for example, a device for inputting, such as a mouse, a keyboard, or an operating member. The output device 15 may be, for example, a device for outputting, such as a display, a speaker, etc. The input device 14 and the output device 15 may be integrally formed by, for example, a touch panel display, etc.

[0068] The drive 16 is a device for reading various information such as programs stored in the storage medium 91. At least one of the control program 81, the first model data 300, the second model data 350, the suitability reference information 200, and the intervention history data 600 may be stored in the storage medium 91 instead of being stored in the storage unit 12, or may be stored in the storage unit 12 and stored in the storage medium 91. The storage medium 91 is configured to store various information (stored programs, etc.) by electrical, magnetic, optical, mechanical or chemical action so that a computer or other device can read the information. The control device 1 can obtain at least one of the control program 81, the first model data 300, the second model data 350, the suitability reference information 200, and the intervention history data 600 from the storage medium 91. In addition, the storage medium 91 may be a disk-type storage medium such as a CD or a DVD, or may be a storage medium other than a disk-type storage medium such as a semiconductor memory (for example, a flash memory). The type of the drive 16 can be appropriately selected according to the type of the storage medium 91.

[0069] In addition, regarding the specific hardware configuration of the control device 1, the components can be appropriately omitted, replaced, and added according to the implementation method. For example, the control unit 11 may include a plurality of hardware processors. The hardware processor may be composed of a microprocessor, an FPGA (field-programmable gate array), a DSP (digital signal processor), an ECU (Electronic Control Unit), a GPU (Graphics Processing Unit), etc. At least one of the external interface 13, the input device 14, the output device 15, and the driver 16 may also be omitted. The control device 1 may also be configured to have a communication interface to perform data communication with an external computer. At least one of the input device 14, the output device 15, and the driver 16 may also be connected via an external interface or a communication interface. In addition to a computer designed specifically for the service provided, the control device 1 may also be a general-purpose computer, a terminal device, etc. In the case where the mobile body M is a vehicle, the control device 1 may be a vehicle-mounted device.

[0070] [3 Action Examples]

[0071] Figure 4 An example of processing steps related to the control of the mobile body M performed by the control device 1 of the present embodiment is shown. The control unit 11 of the control device 1 executes the commands included in the control program 81 through the CPU. As a result, the control device 1 acts as a computer capable of performing the following information processing. The following processing steps are an example of a control method performed by a computer. However, the following processing steps are only an example, and each step can be changed within a possible range. In addition, regarding the following processing steps, the steps can be appropriately omitted, replaced, and added according to the embodiment.

[0072] In step S101, the control unit 11 obtains the observation data of the sensor S. The control unit 11 can obtain the observation data from the sensor S directly or indirectly. In step S102, the control unit 11 uses the first control model 30 and the second control model 35 to generate the first path 40 and the second path 45. The operation of each control model (30, 35) can be performed appropriately. In one example, when a trained machine learning model is used for at least one of the control models (30, 35), the control unit 11 can obtain the path derivation result by inputting at least a part of the observation data into the trained machine learning model and performing the operation processing of the trained machine learning model. In another example, when a rule base model is used for at least one of the control models (30, 35), the control unit 11 can derive the path according to the rule based on the environment represented by at least a part of the observation data.

[0073] In step S103, the control unit 11 evaluates the divergence between the generated first path 40 and the second path 45 using the suitability reference 20. In step S104, the control unit 11 reflects the result of the divergence evaluation and generates a final path 50 from (based on) the first path 40 and the second path 45. In one example, the control unit 11 may select one of the first path 40 and the second path 45 as the final path 50 based on the evaluation result. In another example, the control unit 11 may generate the final path 50 by integrating the first path 40 and the second path 45. The processing of step S103 and step S104 may be performed independently, or may be performed as a series of processing by calculating the model.

[0074] In step S105, the control unit 11 determines whether the user's intervention operation 55 is present. The control unit 11 can accept the user's intervention operation 55 at any time during the period of controlling the movement of the mobile body M. When the intervention operation 55 is not accepted (intervention operation 55 does not exist), the control unit 11 advances the process to step S106. On the other hand, when the intervention operation 55 is accepted (intervention operation 55 exists), the control unit 11 advances the process to step S107.

[0075] In step S106, the control unit 11 controls the motion (movement) of the moving body M according to the generated final path 50. When the motion control is completed, the control unit 11 advances the process to the next step S108. In step S107, the control unit 11 cancels the generated final path 50 or overwrites (overlaps) the final path 50, and controls the motion (movement) of the moving body M according to the user's intervention operation 55. When the motion control is completed, the control unit 11 advances the process to the next step S108.

[0076] In step S108, the control unit 11 adjusts the suitability criterion 20 based on the user's feedback. The control unit 11 adjusts the suitability criterion 20 (for example, increasing the threshold value, increasing the integration ratio of the first path 40) in such a manner that the less the user performs the intervention operation 55, the more the first path 40 is prioritized in generating the final path 50. On the other hand, the control unit 11 adjusts the suitability criterion 20 (for example, decreasing the threshold value, increasing the integration ratio of the second path 45) in such a manner that the first path 40 is prioritized in generating the final path 50, the more the user performs the intervention operation 55.

[0077] In step S109, the control unit 11 determines whether to terminate the control of the moving body M. The criterion for determination can be set arbitrarily. In one example, the control unit 11 may determine that the control of the moving body M is not terminated during the start-up of the moving body M, whereas the control unit 11 determines that the control of the moving body M is terminated based on an arbitrary termination instruction (for example, a termination operation by a user via the input device 14). If it is determined that the control is not terminated, the control unit 11 returns to step S101 and executes the process again from step S101. On the other hand, if it is determined that the control is terminated, the processing steps of the control device 1 of this action example are terminated.

[0078] In addition, the processing order of each step may not be limited to Figure 4 In another example, the processing of step S108 may not be performed in each control cycle. In another example, the processing of step S108 may be performed at predetermined intervals (for example, the passage of a predetermined number of times / predetermined time, ignition on / off (ON / OFF)). Moreover, in another example, the processing of step S108 may be performed independently of the processing of steps S101 to S107. In the case of performing the processing of step S108 (adjustment of the suitability benchmark 20) ​​afterwards, the control unit 11 may generate information representing the user's intervention operation 55 and save the generated information as an intervention history record 60 in a predetermined storage area. In one example, the predetermined storage area may be a RAM, a storage unit 12, a storage medium 91, etc. In the case where the control device 1 is configured to be able to communicate, the predetermined storage area may also be an external computer. The control unit 11 may use the obtained intervention history record 60 to adjust the suitability benchmark 20 at any timing.

[0079] [feature]

[0080] In the present embodiment, through the processing of step S108, the suitability criterion 20 is adjusted based on user feedback in such a manner that the more the intervention operation 55 is performed, the more priority is given to the second control model 35, and the less the intervention operation 55 is performed, the more priority is given to the first control model 30. Thus, in the processing of step S104, the more intervention occurs, the more priority is given to the second path 45, and the less intervention occurs, the more priority is given to the first path 40 to generate the final path 50. As a result, the "scenario (the usage scenario of the first control model 30) in which the first control model 30 (first path 40) is used to perform the automatic control under the processing of step S106" can be limited to (limited to) the range of "no incompatibility". Therefore, according to the present embodiment, it is expected that the "increase in incompatibility (mismatch) of automatic control caused by model change" can be suppressed.

[0081] [4 Modifications]

[0082] The embodiments of the present disclosure have been described in detail above, but the foregoing description is merely an illustration of the present disclosure in all respects. Various improvements or modifications can certainly be made without departing from the scope of the present disclosure. The processes and means described in the present disclosure can be freely combined for implementation as long as no technical contradictions arise.

Claims

1. A control device comprising a storage unit storing a first control model and a second control model, and a control unit, The first control model is a control model newly provided with respect to the second control model. The control unit is configured to execute the following processing: generating a first path by using the first control model; generating a second path by using the second control model; evaluating the divergence between the generated first path and the generated second path using a suitability criterion; reflecting the result obtained by evaluating the divergence and generating a final path from the first path and the second path; as well as Controlling the movement of the moving object according to the generated final path, The suitability criterion is adjusted based on feedback from the user such that the first pathway is prioritized in generating the final pathway as the user performs less interventional operation, and the second pathway is prioritized in generating the final pathway as the user performs more interventional operation.

2. The control device according to claim 1, The first control model is composed of a trained machine learning model. The second control model is composed of a rule base model.

3. A control device according to claim 1. The first control model is composed of an updated trained machine learning model, The second control model is composed of a machine learning model that has been trained before updating.

4. The control device according to claim 1, The process of generating a final path from the first path and the second path includes: A process of selecting one of the first path and the second path as the final path, or a process of generating the final path by integrating the first path and the second path.

5. The control device according to any one of claims 1 to 4, The moving object is a vehicle.

Images