Control device

Abstract

To suppress an increase in incompatibility of automatic control caused by a model change.SOLUTION: A control device generates a first path and a second path according to a first control model and a second control model, evaluates a divergence between the generated first path and the generated second path by using compatibility criteria, generates a final path from the first path and the second path by reflecting an evaluation result, and controls movement of a mobile body according to the generated final path. The compatibility criteria are adjusted according to user feedback so that the first path is given increased priority, the less a user performs an operation of intervention, and the second path is given increased priority, the more the user performs the operation of intervention.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a control technology for a moving object such as an autonomous vehicle. [Background technology]

[0002] Patent Document 1 proposes a system for autonomous vehicle control configured to determine vehicle commands from a route, GPS data, and sensor data using a trained neural network. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2019-533810 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the objectives of the present disclosure is to provide a control technique for suppressing an increase in mismatch of automatic control caused by model changes. [Means for solving the problem]

[0005] The control device according to the first aspect of the present disclosure includes a storage unit that stores a first control model and a second control model, and a control unit. The first control model is newly provided for the second control model. The control unit is configured to execute the following: generating a first path by using the first control model; generating a second path by using the second control model; evaluating a deviation between the generated first path and the generated second path with a compatibility criterion; generating a final path from the first path and the second path by reflecting the result of evaluating the deviation; and controlling the movement of a moving object according to the generated final path. The compatibility criterion is adjusted by feedback from a user so that the less the user's intervention operation is performed, the more the first path is prioritized in generating the final path, and the more the user's intervention operation is performed, the more the second path is prioritized in generating the final path. At least one of the first control model and the second control model may be configured by a neural network, and deep learning may be used as a machine learning method. Effect of the Invention

[0006] According to the present disclosure, it is expected that incompatibility of automatic control caused by model changes can be reduced. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an example of a situation to which the present disclosure is applied. [Diagram 2] FIG. 2 illustrates an example of a data configuration of the intervention history of the present disclosure. [Diagram 3] FIG. 3 illustrates an example of a hardware configuration of the control device of the present disclosure. [Figure 4] FIG. 4 shows an example of a processing procedure related to control by the control device of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Conventionally, rule-based automated driving systems have been known. According to the method of Patent Document 1 and the like, an automated driving system can be constructed by using a trained machine learning model. However, the present inventor has found that these conventional methods have the following problems. For example, consider a situation in which a model change is made, such as changing from a rule-based model to a machine learning model or updating the machine learning model. This model change results in: The contents of the control instruction (path) in the same environment may change (for example, the timing of executing a lane change may change). In some cases, the contents of the instruction may change significantly due to a model change. For example, when a major model change is made, such as changing the structure of the model (for example, changing from a rule-based model to a machine learning model, adopting a machine learning model with a different structure, etc.), additional learning or re-learning of a large amount of new training data, etc., the contents of the instruction are likely to change significantly. When the contents of the instruction change significantly, the control of the automatic driving by the new control model after the change may not necessarily suit the user, compared to the original control model before the change. In one example, the user may feel uncomfortable with the automatic control by the new control model (i.e., it may not suit the user) because the user is accustomed to the automatic control by the original control model. In other words, the automatic driving by the control model may not suit the user due to the model change (it was suitable in the original control model, but not in the new control model). When the control of the automatic driving by the control model does not suit the user, an intervention operation by the user occurs, and the automatic driving is not performed, and manual driving is performed. The increase in incompatibility of the automatic control due to the model change may lead to frequent intervention operations by the user. This problem can occur regardless of the type of vehicle. Furthermore, this problem does not occur only when controlling a vehicle. The same applies to moving objects other than vehicles in terms of controlling the movement. Therefore, the same problem can occur when controlling any moving object other than a vehicle.

[0009] In contrast, the control device according to the first embodiment of the present disclosure includes a storage unit that stores a first control model and a second control model, and a control unit. The first control model is newly provided for the second control model. The control unit is configured to execute the following: generating a first path by using the first control model; generating a second path by using the second control model; evaluating a deviation between the generated first path and the generated second path with a compatibility criterion; generating a final path from the first path and the second path by reflecting the result of evaluating the deviation; and controlling the movement of the moving object according to the generated final path. The compatibility criterion is adjusted by user feedback so that the less the user's intervention operation is performed, the more the final path is generated with priority given to the first path, and the more the user's intervention operation is performed, the more the final path is generated with priority given to the second path.

[0010] In the first aspect of the present disclosure, both the new control model (first control model) and the original control model (second control model) are maintained. The compatibility criteria are adjusted based on user feedback so that the more the user intervenes, the more the second control model is prioritized over the automatic control by the first control model, and the less the user intervenes, the more the first control model is prioritized. In this way, when the user intervenes, the proportion of the second control model used is increased, and when the user does not intervene, the proportion of the first control model used is increased. As a result, by narrowing down the usage scenarios of the first control model to a range where no incompatibility occurs, it is expected that the increase in incompatibility of automatic control caused by model changes can be suppressed.

[0011] As another form of the control device according to the above aspect, one aspect of the present disclosure may be an information processing method or a program for implementing all or part of the above components, or may be a storage medium that stores such a program and is readable by a machine such as a computer. Here, the machine-readable storage medium is a medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action.

[0012] [1 Application example] FIG. 1 is a schematic diagram showing an example of a situation in which the present disclosure is applied. The control device 1 according to this embodiment is one or more computers configured to control the automatic movement of a target moving object M. In this embodiment, the control device 1 is mounted on the moving object M and holds a first control model 30 and a second control model 35. Each control model (30, 35) is used for automatically controlling the movement of the moving object M. The first control model 30 is newly deployed with respect to the second control model 35.

[0013] In this embodiment, the control device 1 generates a first path 40 by using a first control model 30. The control device 1 also generates a second path 45 by using a second control model 35. The control device 1 evaluates the deviation between the generated first path 40 and second path 45 using a compatibility criterion 20. The control device 1 generates a final path 50 from the first path 40 and the second path 45, reflecting the result of evaluating the deviation.

[0014] When there is no user intervention operation 55, the control device 1 controls the operation (movement) of the moving body M according to the generated final path 50. On the other hand, when there is user intervention operation 55, the control device 1 discards the generated final path 50 or overlaps it with the final path 50, and controls the operation of the moving body M according to the user intervention operation 55. The suitability criteria 20 are adjusted based on user feedback so that the less the user intervention operation 55 is performed, the more the final path 50 is generated with priority given to the first path 40, and the more the user intervention operation 55 is performed, the more the final path 50 is generated with priority given to the second path 45.

[0015] In this embodiment, the control device 1 holds both the new control model (first control model 30) and the original control model (second control model 35). The first control model 30 is newer than the second control model 35. Therefore, basically, the first control model 30 is used preferentially over the second control model 35. That is, in generating the final path 50, the first path 40 is prioritized. However, the control by the new control model (first control model 30) is not necessarily suitable for a user who is accustomed to the control by the original control model (second control model 35). In particular, when the deviation between the first path 40 and the second path 45 is large, the control by the first path 40 is highly likely to be incompatible (feel uncomfortable) with a user who is accustomed to the second path 45. When it is not compatible, an operation 55 of intervention by the user may occur.

[0016] In this embodiment, the user's intervention operation 55 is used as an index to evaluate whether or not the control by the new control model (first control model 30) is suitable for the user. When intervention occurs, the control by the new control model is evaluated as not suitable for the user, so the suitability criteria 20 are adjusted to increase the usage ratio of the original control model (second control model 35) (i.e., prioritize the second path 45). On the other hand, when intervention does not occur, the control by the new control model is evaluated as suitable for the user, so the suitability criteria 20 are adjusted to increase the usage ratio of the new control model (first control model 30) (i.e., prioritize the first path 40). This makes it possible to narrow the usage range of the first control model 30 to a range where no incompatibility occurs, and as a result, it is expected that the increase in incompatibility of automatic control caused by model change can be suppressed.

[0017] (Mobile) As long as the moving body M can move automatically by mechanical control, the type of the moving body M may be appropriately selected depending on the embodiment. The moving body M may be, for example, a moving device such as a vehicle, an aircraft, a ship, or a robot device. The aircraft may be at least one of an unmanned aircraft such as a drone and a manned aircraft. In one example, as shown in FIG. 1, the moving body M may be a vehicle. Note that the type of the vehicle (number of wheels, power source, size, etc.) may be selected arbitrarily. As a typical example, the moving body M may be an automobile capable of level 2 or higher autonomous driving.

[0018] (Controlling the movement) In one example, controlling the movement (movement) of the target moving object M may be configured by directly controlling the target moving object M. In another example, the moving object M may be equipped with a dedicated control device such as a controller. In this case, the control device 1 may directly control the movement (movement) of the target moving object M. may be configured by indirectly controlling the target moving body M by providing the derived result to the dedicated control device. The control device 1 may be deployed at any location. In one example, as shown in FIG. 1, the control device 1 may be mounted on the moving body M. In another example, the control device 1 may be disposed away from the moving body M and remotely control the moving body M. The control device 1 may be configured to be able to switch the moving body M from an automatic control mode to a manual control mode and from the manual control mode to the automatic control mode at any timing by any operation.

[0019] (path) The form of the path (output of the control model) is not particularly limited as long as it is capable of controlling the operation of the moving body M, and may be appropriately determined depending on the embodiment. In one example, the path (40, 45, 50) may be configured with one or more control commands. The control commands may be configured to indicate a control amount of the moving body M. In another example, the path (40, 45, 50) may be configured to indicate a future movement path of the moving body M and may be used to derive one or more control commands. In response, the one or more control commands may be determined in any manner from the final path 50. The control model (30, 35) may be referred to as a path planner.

[0020] The control command relates to the operation of the moving body M. The configuration of the control command may be appropriately selected depending on the embodiment. In one example, the control command may be configured by acceleration, deceleration, steering, or a combination of these. Acceleration and deceleration may include gear changes. The control command may be configured to indicate control amounts (control instruction values, control output amounts) of the moving body M, such as accelerator control amount, brake control amount, and steering wheel steering angle. The control command may further include commands related to the operation of the moving body M. As an example, if the moving body M is a vehicle, the control command may include vehicle operations such as turn signals, hazard lights, horn, and communication processing (e.g., sending data to a center, making an emergency call, etc.).

[0021] (Discard or overlap) Abandoning (ignoring) may be an immediate switch from automatic control to manual control, i.e., from automatic control by final path 50 to manual control following user intervention operation 55. On the other hand, overlapping may be a gradual switch from automatic control to manual control (user control).

[0022] (Control Model) The control model (30, 35) is constructed to derive a path according to the environment of the moving body M. The environment is an event observed by at least one of the moving body M itself and its surroundings. In one example, at least a part of the environment may be observed by one or more sensors S arranged inside or outside the moving body M. The type of the sensor S is not particularly limited as long as it can observe any environment in which the moving body M moves, and may be appropriately selected depending on the embodiment. In one example, the one or more sensors S may be a camera (image sensor), a radar, a LiDAR (Light Detection and Ranging). Ranging, sonar (ultrasonic sensor), infrared sensor, GNSS (Global Navigation Satellite System) / GPS (Global Positioning Satellite) module, etc.

[0023] As long as a path can be derived from the environment of the moving body M, the input / output format of the control models (30, 35) may be appropriately selected depending on the embodiment. In one example, at least one of the control models (30, 35) may be configured to derive a path from observation data (sensor data) of the sensor S at one or more points in time. In another example, at least one of the control models (30, 35) may be configured to derive a path from a recognition result of the surrounding environment. In this case, the control device 1 may further include an analytical model that infers the recognition result of the surrounding environment from the observation data of the sensor S. Alternatively, at least one of the control models (30, 35) may include such an analytical model. The analytical model may be arbitrarily configured by a trained machine learning model or the like. Other information may be arbitrarily added to the input of at least one of the control models (30, 35). At least one of the control models (30, 35) may be configured to further receive input of arbitrary information, such as a set speed, a speed limit, map data, navigation information (route data), etc. When the moving body M is a vehicle, the arbitrary information may include driving data. The arbitrary information may be appropriately acquired from a device such as a navigation device or an on-board sensor. Furthermore, the control model (30, 35) may be configured to directly output a path, or may be configured to indirectly output a path. In the latter case, a path may be obtained by performing arbitrary information processing (interpretation processing) on ​​the output of the control model (30, 35).

[0024] The control model (30, 35) may be configured by at least one of a trained machine learning model and a rule-based model. The rule-based model is configured to match a given input (e.g., information indicating the environment, such as observation data and a result of recognition of the surrounding environment) with a rule, and derive a path according to the result of the match (according to the matching rule). The rule may be set manually or at least partially automatically. The machine learning model is configured to have one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used for the calculation of the desired inference (in this disclosure, derivation of a path). Machine learning is to adjust (optimize) the value of the calculation parameter using learning data. The machine learning model may be configured by, for example, a neural network, a support vector machine, a regression model, a decision tree model, etc. The machine learning method may be appropriately selected according to the machine learning model to be adopted (e.g., backpropagation method, etc.). Machine learning may include supervised learning, unsupervised learning, and reinforcement learning.

[0025] In one example, at least one of the control models (30, 35) may be at least partially composed of a neural network. The structure of the neural network may be appropriately determined depending on the embodiment. The structure of the neural network may be specified by, for example, the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationship between the nodes in each layer, etc. In one example, the neural network may have a recursive structure. In addition, the neural network may include any layer, such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, an unpooling layer, a normalization layer, a dropout layer, and a long short-term memory (LSTM). The neural network may have any mechanism, such as an attention mechanism. The neural network may include any model, such as a graph neural network (GNN), a diffusion model, or a generative model (e.g., a generative adversarial network, a transformer, etc.). When the neural network is used as the control model, When used in a model, the weight of the connection between each node included in the control model and the threshold value of each node are examples of the calculation parameters. When a machine learning model is adopted, an example of the control model may be configured with the structure of an end-to-end model.

[0026] The first control model 30 and the second control model 35 are constructed for the same purpose. In other words, the first control model 30 and the second control model 35 are constructed so as to be capable of performing the same type of movement in the same or overlapping environments. Since the control required for each scene may differ, the first control model 30 and the second control model 35 may be prepared for each scene. In one example, when the moving body M is a vehicle, the first control model 30 and the second control model 35 may be configured to perform, for example, lane changing, lane keeping, emergency stopping (EDSS: Emergency Driving Stop System), merging and yielding, automatic parking, and the like. It may be prepared for each scene such as a car.

[0027] (Newly deployed) The first control model 30 may be prepared as a new control model for the second control model 35. That is, the first control model 30 may be a new control model after modification, and the second control model 35 may be the original control model before modification. In one example, the modification of the control model may be a system replacement. Accordingly, the modification of the control model may be, for example, replacing a rule-based model with a trained machine learning model, replacing a trained machine learning model with a rule-based ... The change of the model may include replacing a machine learning model with a rule-based model, changing the structure of the machine learning model, etc. In another example, the change of the model may be an upgrade. Accordingly, the change of the model may include, for example, deploying a new rule-based model with modified rules, deploying a new trained machine learning model updated by additional learning or re-learning, etc. Note that each control model (30, 35) (model data) may be provided as appropriate from an external server, etc.

[0028] In one example, the first control model 30 may be configured by a trained machine learning model. The second control model 35 may be configured by a rule-based model. The structures of the trained machine learning model and the rule-based model may be different. Therefore, the characteristics of the automatic control of the trained machine learning model and the rule-based model may differ from each other. As a result, when the automatic control system is updated from the rule-based model to the trained machine learning model, the control by the trained machine learning model may not be suitable for a user who is accustomed to the control by the rule-based model. According to one example of the present embodiment, even when such an automatic control system is updated, the arbitration process by the compatibility criterion 20 can be expected to suppress an increase in the incompatibility of the automatic control caused by the model change.

[0029] In another example, the first control model 30 may be configured by a trained machine learning model after updating. Meanwhile, the second control model 35 may be configured by a trained machine learning model before updating. The update may include at least one of the replacement and version upgrade of the above-mentioned system. The update may include any process that may change the output of the control model. The update may include, for example, correcting the value of a parameter by additional learning or re-learning, changing the structure of the model, etc. According to one example of the present embodiment, it is expected that the increase in incompatibility of automatic control caused by model updating can be suppressed.

[0030] The number of control models held by the control device 1 does not necessarily have to be limited to two. The number of first control models 30 does not necessarily have to be limited to one. The number of second control models 35 does not necessarily have to be limited to one. Using the first control model 30 may include using one or more first control models 30. Using the second control model 35 may include using one or more second control models 35. In another example, the control device 1 may hold three or more control models. A hierarchy of a control model that is prioritized as more intervention occurs and a control model that is prioritized as less intervention occurs may be defined among the three or more control models so as to correspond to each of the first control model 30 and the second control model 35. The control device 1 may be configured to perform arbitration processing according to the compatibility criteria 20 among the three or more control models to generate the final path 50. That is, generating the final path 50 from the first path 40 and the second path 45 may include generating the final path 50 from three or more paths. Typically, the first control model 30 is a control model one generation after the second control model 35. However, the relationship between the first control model 30 and the second control model 35 is not limited to this example. The generation difference between the first control model 30 and the second control model 35 may be two or more generations.

[0031] (Conformity criteria) The suitability criteria 20 are defined to evaluate whether or not the automatic control by the first control model 30 is suitable for the user with respect to the discrepancy occurring between the first control model 30 and the second control model 35. If such suitability can be evaluated, the suitability criteria 20 may be set appropriately. Simply, the suitability criteria 20 may be configured with thresholds for physical indicators. In one example, the suitability criteria 20 may be configured with thresholds for speed, acceleration, timing of change, or a combination of these, for movement (going straight, turning right, turning left, etc.). In this case, adjusting the suitability criteria 20 may be configured by updating (i.e., correcting) the thresholds. The suitability criteria 20 may be configured based on the movement scene (lane change, etc.) and the movement environment (e.g., position, route, The user's suitability may be evaluated based on at least one of the following:

[0032] (Generating the final path) Generating the final path 50 from the first path 40 and the second path 45 may consist of (I) selecting one of the first path 40 and the second path 45 as the final path 50 depending on the result of the evaluation, or (II) integrating the first path 40 and the second path 45 to generate the final path 50. The final path 50 is derived finally and is used to control the moving body M.

[0033] When (I) is adopted, giving priority may be to select a preferred path. As a simple example, assume a case where the suitability criterion 20 is given as a threshold value for the speed of movement. In this case, the control device 1 may select the second path 45 as the final path 50 when the speed-related difference between the first path 40 and the second path 45 exceeds the threshold value of the suitability criterion 20, and may select the first path 40 as the final path 50 when the speed-related difference exceeds the threshold value of the suitability criterion 20. The less the intervention operation 55 occurs, the larger the threshold value of the suitability criterion 20 may be adjusted, so that the first path 40 may be more likely to be selected as the final path 50. On the other hand, the more the intervention operation 55 occurs, the smaller the threshold value of the suitability criterion 20 may be adjusted, so that the second path 45 may be more likely to be selected as the final path 50.

[0034] When (II) is adopted, the priority may be to increase the ratio in the case of integration. The fitness criterion 20 may include this integration ratio together with or instead of the threshold value. Adjusting the fitness criterion 20 may include adjusting the value of the integration ratio. The less the intervention operation 55 occurs, the higher the ratio of the first path 40 may be, and the more the intervention operation 55 occurs, the higher the ratio of the second path 45 may be. Evaluating the deviation with the fitness criterion 20 may include determining the integration ratio of each path (40, 45) according to the fitness criterion 20. The integration ratio may be determined according to the degree of deviation between the first path 40 and the second path 45, or may be determined regardless of the degree of deviation. The integration may be performed by any calculation such as summation, averaging, weighted averaging, etc. In one example, in the above case, the control device 1 may determine the ratio of each path (40, 45) according to the suitability criterion 20, and may generate the final path 50 by integrating the first path 40 and the second path 45 at the determined ratio. In another example, when the suitability criterion 20 adopts a form including the above threshold, in the above case, the control device 1 may generate the final path 50 by integrating the first path 40 and the second path 45 if the deviation in speed between the first path 40 and the second path 45 exceeds the threshold of the suitability criterion 20, and may select the first path 40 as the final path 50 if not.

[0035] In addition, evaluating the deviation using the fitness criterion 20 and generating the final path 50 according to the evaluation result may be performed based on at least one of a rule base and a computation model (machine learning model). When the rule base is adopted, the control device 1 may evaluate the deviation between the first path 40 and the second path 45 using the fitness criterion 20, and execute the above process (I) or (II) according to the evaluation result of the deviation. When the computation model base is adopted, the control device 1 may execute the evaluation of the deviation and the above process (I) or (II) as a series of processes of the computation model. In this case, the fitness criterion 20 may be incorporated into the computation model.

[0036] Furthermore, in this embodiment, when an intervention operation 55 by the user occurs, the suitability criteria 20 is adjusted to increase the usage weight of the second control model 35 (the second path 45). Increasing the usage weight means making it easier to select the second path 45 (for example, by reducing the threshold) in the above case (I), and increasing the integration ratio of the second path 45 in the above case (II). In one example, after increasing the usage weight of the second control model 35 in this way, Additionally, the suitability criteria 20 may be adjusted to gradually increase the usage ratio of the first control model 30 to the extent that no user intervention is required. This may gradually eliminate the user's sense of incongruity regarding control by the new control model (first control model 30), allowing the user to enjoy the benefits of the new control model.

[0037] In addition, even after the priority of the second path 45 is increased due to the occurrence of the user's intervention operation 55, the user's intervention operation 55 may continue to occur. In this case, in one example, the control device 1 may continue to increase the degree of priority given to the second path 45. In another example, the control device 1 may use a calculation model to generate the final path 50 from the first path 40 and the second path 45. The control device 1 may adjust the parameter value of the calculation model (for example, the integration ratio of the first path 40 and the second path 45) so that an output that matches the path by the intervention operation 55 is obtained. Note that, if automatic control that matches the user is not executed even when the second path 45 is given the highest priority, it is possible that the second control model 35 is not suited to the user in the first place, and the newly deployed first control model 30 is also not suited to the user. As a result, it may be difficult for the first control model 30 and the second control model 35 to generate a path that matches the user. Therefore, in one example, if such a situation occurs, the control device 1 may send information indicating the environmental conditions at the time of intervention and the intervention operation (or path) to a computer (e.g., an external server) that provides the control model, and prompt for further updating of the control model.

[0038] (User) The term "user" may refer to a specific user or an unspecified user. In one example, the suitability criteria 20 may be differentiated for each user. In this case, the suitability criteria 20 of the target user may reflect only the feedback of the target user. In another example, the suitability criteria 20 may not be differentiated. In this case, feedback of any user including other users other than the target user may be reflected in the suitability criteria 20, and the obtained suitability criteria 20 may be applied when the target user uses the moving body M. In addition, when the moving body M is a vehicle, the user is typically a driver.

[0039] (feedback) In one example, the user's feedback (whether or not an intervention operation 55 was performed) may be immediately reflected in the suitability criteria 20. In another example, the user's feedback may be accumulated as an intervention history 60, and the obtained intervention history 60 may be reflected in the suitability criteria 20 afterwards. The user's feedback may be reflected in adjusting the suitability criteria 20 (e.g., updating the above threshold) after identifying at least one of the movement scene and the movement environment. In this case, the conditions (scene, environment, etc.) under which the intervention occurred may be appropriately identified.

[0040] When accumulating user feedback as the intervention history 60, the items of information to be saved as the intervention history 60 may not be particularly limited as long as the fitness criterion 20 can be adjusted, and may be appropriately selected according to the embodiment. In one example, when the condition under which the intervention occurred is identified and the fitness criterion 20 is adjusted, the intervention history 60 may include information indicating the condition under which the intervention occurred. In addition, when a trained machine learning model is adopted or there is a possibility of adopting the control model (30, 35), the intervention history 60 may include information for generating learning data that can be used for training the machine learning model. In one example, in order to identify the model to be improved, the intervention history 60 may include information (e.g., an identifier, etc.) for identifying the control model that was used when the intervention occurred. In addition, the intervention history 60 may further include the user's intervention operation 55 or the path by the operation 55. The information indicating the environmental condition under which the intervention occurred, which is included in the intervention history 60, may be used as training data (input data), and the user's intervention operation 55 or the path by the operation 55 may be used as correct answer data (teacher signal, label). In this case, the control model configured with the machine learning model may be updated (re-learned, additionally learned, etc.) by the obtained intervention history 60. It may be executed within the device 1, or may be executed on a computer other than the control device 1 (for example, an external server). The data format of the intervention history 60 may be appropriately determined depending on the embodiment. The intervention history 60 may be held in any database format.

[0041] FIG. 2 is a schematic diagram showing an example of a data configuration of the intervention history 60 according to the present embodiment. In the example of FIG. 2, the records of the intervention history 60 include a time stamp, identification information of the control model used (model used), the environmental conditions in which the intervention occurred, and information indicating the operation of the intervention. The time stamp indicates the date and time when the intervention occurred. The date and time indicated by the time stamp may be used to delete old records, identify records to be reflected in the fitness standard 20 (when the intervention history within a certain period is reflected in the fitness standard 20), and the like. The records of the intervention history 60 may be generated for each intervention operation. The unit of the intervention operation indicated by the record may be determined arbitrarily. In one example, one record (a sample of the intervention history) may be generated for each intervention operation. In another example, one record may be generated in response to multiple interventions.

[0042] [2 Configuration example] 3 is a schematic diagram showing an example of a hardware configuration of the control device 1 according to the present embodiment. The control device 1 according to the present embodiment is a computer to which a control unit 11, a storage unit 12, an external interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0043] The control unit 11 includes a CPU (Central Processing Unit), a RAM (Random Access Memory ), ROM (Read Only Memory), etc., and is configured to execute any information processing based on the program and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 may be configured, for example, by a hard disk drive, a solid state drive, etc. The storage unit 12 (as well as RAM and ROM) is an example of a memory resource, and is an example of a storage unit of the present disclosure. In this embodiment, the storage unit 12 stores various information such as a control program 81, first model data 300, second model data 350, conformity standard information 200, intervention history data 600, etc.

[0044] The control program 81 is a program for causing the control device 1 to execute information processing (FIG. 4 described later) related to the control of the moving object M. The control program 81 includes a series of instructions for the information processing. The first model data 300 is configured to indicate information related to the first control model 30. The second model data 350 is configured to indicate information related to the second control model 35. In one example, when a machine learning model is adopted for the control model, the model data (300, 350) may include information indicating values ​​of calculation parameters adjusted by machine learning. In some cases, the model data (300, 350) may further include information indicating the configuration of the machine learning model (e.g., the structure of a neural network, etc.). In another example, when a rule-based model is adopted for the control model, the model data (300, 350) may include information indicating the rule-based model (rule). The suitability criterion information 200 may be appropriately configured to indicate the suitability criterion 20. The intervention history data 600 may be appropriately configured to indicate the intervention history 60.

[0045] The external interface 13 may be, for example, a Universal Serial Bus (USB) port, a dedicated port, a wireless communication port, etc., and is configured to connect to an external device by wire or wirelessly. In this embodiment, the control device 1 may be connected to the sensor S via the external interface 13. The input device 14 is, for example, a device for inputting a mouse, a keyboard, an operator, etc. The output device 15 is, for example, a device for outputting a display, a speaker, etc. The input device 14 and the output device 15 may be integrally configured by, for example, a touch panel display, etc.

[0046] The drive 16 is used to read various information such as programs stored in the storage medium 91. The control device 1 is a device. At least one of the control program 81, the first model data 300, the second model data 350, the conformity standard information 200, and the intervention history data 600 may be stored in a storage medium 91 instead of the storage unit 12 or together with the storage unit 12. The storage medium 91 is configured to accumulate various information (stored programs, etc.) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The control device 1 may acquire at least one of the control program 81, the first model data 300, the second model data 350, the conformity standard information 200, and the intervention history data 600 from the storage medium 91. The storage medium 91 may be a disk-type storage medium such as a CD or a DVD, or may be a storage medium other than a disk-type storage medium such as a semiconductor memory (for example, a flash memory). The type of the drive 16 may be appropriately selected according to the type of the storage medium 91.

[0047] In addition, regarding the specific hardware configuration of the control device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), an ECU, (Electronic Control Unit), GPU (Graphics Processing Unit), etc. At least one of the external interface 13, the input device 14, the output device 15, and the drive 16 may be omitted. The control device 1 may be provided with a communication interface and configured to perform data communication with an external computer. At least one of the input device 14, the output device 15, and the drive 16 may be connected via an external interface or a communication interface. The control device 1 may be a computer designed specifically for the service provided, as well as a general-purpose computer, a terminal device, etc. When the moving body M is a vehicle, the control device 1 may be an in-vehicle device.

[0048] [3 Example of operation] FIG. 4 shows an example of a processing procedure for controlling a moving body M by the control device 1 according to this embodiment. The control unit 11 of the control device 1 executes instructions included in a control program 81 by means of a CPU. As a result, the control device 1 operates as a computer capable of executing the following information processing. The following processing procedure is an example of a control method executed by a computer. However, the following processing procedure is merely an example, and each step may be modified as much as possible. Furthermore, steps of the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0049] In step S101, the control unit 11 acquires observation data of the sensor S. The control unit 11 may acquire the observation data from the sensor S directly or indirectly. In step S102, the control unit 11 generates the first path 40 and the second path 45 using the first control model 30 and the second control model 35. The calculation of each control model (30, 35) may be performed appropriately. In one example, when a trained machine learning model is adopted in at least one of the control models (30, 35), the control unit 11 may input at least a part of the observation data to the trained machine learning model and perform calculation processing of the trained machine learning model to obtain a path derivation result. In another example, when a rule-based model is adopted in at least one of the control models (30, 35), the control unit 11 may derive a path from an environment indicated by at least a part of the observation data according to a rule.

[0050] In step S103, the control unit 11 evaluates the deviation between the generated first path 40 and second path 45 using the compatibility criterion 20. In step S104, the control unit 11 generates a final path 50 from the first path 40 and the second path 45, reflecting the result of evaluating the deviation. In one example, the control unit 11 may select one of the first path 40 and the second path 45 as the final path 50 depending on the result of the evaluation. In another example, the control unit 11 may select one of the first path 40 and the second path 45 as the final path 50 depending on the result of the evaluation. 45 may be integrated to generate the final path 50. The processes of steps S103 and S104 may be executed separately, or may be executed as a series of processes by a computation model.

[0051] In step S105, the control unit 11 determines whether or not there is an intervening operation 55 by the user. The control unit 11 may accept an intervening operation 55 by the user at any time while controlling the operation of the moving object M. If the intervening operation 55 has not been accepted (there is no intervening operation 55), the control unit 11 advances the process to step S106. On the other hand, if the intervening operation 55 has been accepted (there is an intervening operation 55), the control unit 11 advances the process to step S107.

[0052] In step S106, the control unit 11 controls the movement (movement) of the moving object M according to the generated final path 50. When the control of the movement is completed, the control unit 11 proceeds to the next step S108. In step S107, the control unit 11 discards the generated final path 50 or overlaps with the final path 50, and controls the movement (movement) of the moving object M according to the user's intervention operation 55. When the control of the movement is completed, the control unit 11 proceeds to the next step S108.

[0053] In step S108, the control unit 11 adjusts the suitability criteria 20 based on the user's feedback. The control unit 11 adjusts the suitability criteria 20 so that the final path 50 is generated with priority given to the first path 40 as the user's intervention operation 55 is reduced (for example, by increasing the threshold value or increasing the integration ratio of the first path 40). On the other hand, the control unit 11 adjusts the suitability criteria 20 so that the final path 50 is generated with priority given to the second path 45 as the user's intervention operation 55 is increased, in contrast to the automatic control that prioritizes the first path 40 (for example, by decreasing the threshold value or increasing the integration ratio of the second path 45).

[0054] In step S109, the control unit 11 judges whether or not to end the control of the moving object M. The criteria for the judgment may be set arbitrarily. In one example, while the moving object M is activated, the control unit 11 may judge not to end the control of the moving object M, whereas in response to an arbitrary end instruction (e.g., an end operation by the user via the input device 14), the control unit 11 may judge to end the control of the moving object M. If it is judged not to end the control, the control unit 11 returns to step S101 and executes the process again from step S101. On the other hand, if it is judged to end the control, the processing procedure of the control device 1 according to this operation example is terminated.

[0055] The processing order of each step is not limited to the example of FIG. 4, and may be changed as appropriate depending on the embodiment. For example, the processing of step S108 may not be executed for each control cycle. In another example, the processing of step S108 may be executed at predetermined intervals (for example, a predetermined number of times / a predetermined time has elapsed, or the ignition is turned on / off). In yet another example, the processing of step S108 may be executed independently of the processing of steps S101 to S107. When the processing of step S108 (adjustment of the conformity criterion 20) is executed after the fact, the control unit 11 may generate information indicating the user's intervention operation 55 and store the generated information in a predetermined storage area as the intervention history 60. In one example, the predetermined storage area may be a RAM, the storage unit 12, the storage medium 91, or the like. When the control device 1 is configured to be communicable, the predetermined storage area may be an external computer. The control unit 11 may adjust the conformity criterion 20 at any timing using the obtained intervention history 60.

[0056] [Features] In this embodiment, the suitability criteria 20 is adjusted by the process of step S108 based on the user's feedback so that the more the intervention operation 55 is performed, the more the second control model 35 is prioritized, and the less the intervention operation 55 is performed, the more the first control model 30 is prioritized. As a result, in the process of step S104, the more the intervention occurs, the more the second path 45 is prioritized, and the less the intervention occurs, the more the suitability criteria 20 is adjusted based on the user's feedback so that the more the intervention operation 55 is performed, the more the second path 45 is prioritized, and the less the intervention occurs, the more the suitability criteria 20 is adjusted based on the user's feedback. The less the first path 40 is, the higher the priority is given to the first path 40, and the final path 50 is generated. As a result, the first control model 30 (first path 40) is used preferentially, and the situations in which automatic control is performed by the processing of step S106 (situations in which the first control model 30 is used) can be narrowed down to a range in which no incompatibility occurs. Therefore, according to this embodiment, it can be expected that an increase in incompatibility of automatic control caused by model change can be suppressed.

[0057] [4 Variations] Although the embodiment of the present disclosure has been described in detail above, the above description is merely an example of the present disclosure in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present disclosure. The processes and means described in the present disclosure can be freely combined and implemented as long as no technical contradiction occurs. [Explanation of symbols]

[0058] 1...Control device, 11: control unit; 12: memory unit; 20: compatibility criteria; 30...first control model, 35...second control model, 40…1st pass, 45…2nd pass, 50…Final pass, 55…(intervention) operation, 60…Intervention history M: moving object, S: sensor

Claims

1. a storage unit that stores the first control model and the second control model; and Control unit, A control device comprising: the first control model is newly deployed with respect to the second control model, The control unit is generating a first path using the first control model; generating a second path using the second control model; evaluating a deviation between the generated first path and the generated second path using a fitness criterion; generating a final path from the first path and the second path, while reflecting a result of evaluating the deviation; and Controlling the movement of a moving object according to the generated final path; and The suitability criteria is adjusted based on feedback from a user so that the less an intervention operation by a user is performed, the more the first path is prioritized in generating the final path, and the more an intervention operation by the user is performed, the more the second path is prioritized in generating the final path. Control device.

2. The first control model is configured by a trained machine learning model; The second control model is configured by a rule-based model. The control device according to claim 1 .

3. The first control model is configured by an updated trained machine learning model; The second control model is configured by a trained machine learning model before updating. The control device according to claim 1 .

4. Generating a final path from the first path and the second path includes: selecting one of the first path and the second path as the final path; or combining the first pass and the second pass to generate the final pass; Consists of: The control device according to claim 1 .

5. The moving object is a vehicle. A control device according to any one of claims 1 to 4.

Images