Adaptation system and adaptation method

Abstract

To provide an adaptation system that can prevent fluctuations of a state variable due to noise in sensors' signals or the like from disturbing convergence of learning in the final stage of optimization of a function.SOLUTION: A processing circuit of an adaptation system executes second processing (from step S210 to step S230) when the number of times of execution of a learning routine is equal to or greater than the predetermined number of times and less than the number of times of termination (step S200: YES). The second processing is performing a first trial and a second trial in each learning routine and ending the learning routine by recording, in a storage device, a change in a trial with a larger reward among the first trial and the second trial. The processing circuit executes third processing (from step S240 to step S260) when the number of times of execution of the learning routine reaches the number of times of termination (step S200: NO). The third processing is processing for calculating summary statistics of a plurality of changes and reflecting the summary statistics in a control map to complete optimization of the control map.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a fitting method and system for optimizing a function used in controlling a motor. [Background technology]

[0002] Patent Document 1 discloses a machine learning device that learns a current command value for a motor. In the learning process, this machine learning device acquires state variables while driving the motor. The machine learning device then calculates a reward based on the state variables. The machine learning device learns a current command value based on the reward. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-014838 Summary of the Invention [Problem to be solved by the invention]

[0004] When using a calibration system to automatically optimize a function for outputting a command value to a motor, the calibration system performs trials to acquire state variables while driving the motor. The calibration system evaluates the results of the trials using a reward calculated based on the state variables. The calibration system then updates and learns the function based on the evaluation. In this way, the calibration system optimizes the function by gradually updating it through repeated trials, evaluations, and learning.

[0005] It is preferable that the learning gradually converges as the optimization of the function approaches completion. However, there are cases where the learning does not converge easily due to the influence of fluctuations in the state variables caused by noise in the signals from the sensors, etc. [Means for solving the problem]

[0006] The means for solving the above problems and their effects will be described below. An adaptation system for solving the above problem includes a processing circuit and a storage device. In this adaptation system, the processing circuit optimizes the function stored in a control device that controls the motor by repeating a learning routine including: trials of driving the motor while acquiring state variables via a sensor with a change made to a function that outputs a command value to the motor; evaluation of calculating a reward based on the acquired state variables; and learning of updating the function based on the reward. If the number of times the learning routine has been executed is less than a predetermined number, the processing circuit executes a first process of performing a first trial and a second trial in which the change is made to the function so that the command value output from the function is adjusted in the opposite direction in each learning routine, and updating the function by reflecting the change in the trial with the largest reward out of the first and second trials. The processing circuit executes a second process of performing the first trial and the second trial in each learning routine, and terminating the learning routine by recording in the storage device the change in the trial with the largest reward among the first trial and the second trial without reflecting it in the function, when the number of executions of the learning routine is equal to or greater than the predetermined number and less than a termination number greater than the predetermined number. The processing circuit executes a third process of calculating summary statistics of the changes stored in the storage device without being reflected in the function, and reflecting the changes based on the summary statistics in the function, thereby terminating the learning routine and completing optimization of the function, when the number of executions of the learning routine reaches the termination number.

[0007] An adaptation method for solving the above problem is an adaptation method that uses an adaptation system including a processing circuit and a storage device to optimize a function stored in a control device that controls the motor. This adaptation method optimizes the function stored in the control device that controls the motor by repeatedly executing a learning routine by the processing circuit, the learning routine including: trials of driving the motor while acquiring state variables by a sensor with a change made to a function that outputs a command value to the motor; evaluation of calculating a reward based on the acquired state variables; and learning of updating the function based on the reward. This adaptation method includes a first step of, when the learning routine has been executed less than a predetermined number of times, causing the processing circuit to execute a first process of performing first and second trials in which the change is made to the function so that the command value output by the function is adjusted in opposite positive and negative directions in each learning routine, updating the function by reflecting the change in the trial that produced the largest reward out of the first and second trials, and terminating the learning routine. The adaptation method includes a second step of causing the processing circuit to execute a second process, when the number of executions of the learning routine is equal to or greater than the predetermined number and less than a termination number greater than the predetermined number, of performing the first trial and the second trial in each learning routine, and terminating the learning routine by recording in the storage device the change in the trial of the first trial or the second trial that had the largest reward without reflecting it in the function.The adaptation method includes a third step of causing the processing circuit to execute a third process, when the number of executions of the learning routine reaches the termination number, of calculating summary statistics of the changes stored in the storage device without being reflected in the function, and reflecting the changes based on the summary statistics in the function, thereby terminating the learning routine and completing optimization of the function. [Effects of the Invention]

[0008] This can prevent fluctuations in state variables due to noise in sensor signals, etc., from hindering the convergence of learning at the end of function optimization. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of the adaptation system and the relationship between the adaptation system and a vehicle. [Figure 2] FIG. 2 is a schematic diagram showing the configuration of a powertrain of a vehicle. [Figure 3] FIG. 3 is a time chart showing (a) the transition of the torque of the motor generator, (b) the presence or absence of ignition, and (c) the transition of the engine rotation speed when starting the engine. [Figure 4] FIG. 4 is an explanatory diagram for explaining an example of a control map of a torque command value. [Figure 5] FIG. 5 is a flowchart showing part of the flow of a series of processes executed by the adaptation system. [Figure 6] FIG. 6 is a flowchart showing part of the flow of a series of processes executed by the adaptation system. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the adaptation system will now be described with reference to FIGS. <Configuration of the adaptation system 100> As shown in FIG. 1, the adaptation system 100 includes a processing circuit 101 and a storage device 102. The storage device 102 stores programs and data. The processing circuit 101 executes the programs stored in the storage device 102. The adaptation system 100 optimizes functions to be stored in a control device 20 mounted on a vehicle 10. The vehicle 10 is equipped with a hybrid system 30. The control device 20 controls the hybrid system 30.

[0011] <Powertrain configuration of vehicle 10> 2, the power train of vehicle 10 includes a hybrid system 30. Vehicle 10 drives drive wheels 40 using hybrid system 30. Hybrid system 30 of vehicle 10 includes a first motor generator 31, a second motor generator 32, an engine 33, a power split mechanism 34, a power control unit 35, and a reduction mechanism 36.

[0012] The second motor generator 32 is connected to a power control unit 35. The second motor generator 32 is connected to drive wheels 40 via a reduction gear mechanism 36. The engine 33 is connected to drive wheels 40 via a power split mechanism 34 and the reduction gear mechanism 36. The first motor generator 31 is connected to the power split mechanism 34. The first motor generator 31 is, for example, a three-phase AC motor generator.

[0013] The power split mechanism 34 is made up of planetary gears. The power split mechanism 34 can split the driving force of the engine 33 between the first motor generator 31 and the drive wheels 40. The first motor generator 31 generates electricity using the driving force of the engine 33, and also generates electricity using the driving force from the drive wheels 40. The first motor generator 31 drives the crankshaft of the engine 33 when starting the engine 33. Therefore, the first motor generator 31 is a motor that drives the crankshaft of the engine 33 to crank the engine 33.

[0014] The first motor generator 31 and the second motor generator 32 are connected to a battery via a power control unit 35. The AC power generated by the first motor generator 31 is converted to DC by the power control unit 35 and charged into the battery. In other words, the power control unit 35 functions as an inverter.

[0015] The DC power of the battery is converted to AC by the power control unit 35 and supplied to the second motor generator 32. When the vehicle 10 is decelerating, the second motor generator 32 generates electricity using the driving force from the drive wheels 40. The generated electricity is then charged to the battery. In other words, the vehicle 10 performs regenerative charging. At this time, the second motor generator 32 functions as a generator. The AC power generated by the second motor generator 32 is converted to DC by the power control unit 35 and charged to the battery. When the engine 33 is cranked by the first motor generator 31, the power control unit 35 converts the DC power of the battery to AC and supplies it to the first motor generator 31.

[0016] <Regarding the control device 20> The control device 20 controls the engine 33, the first motor generator 31, and the second motor generator 32. The control device 20 includes an engine control unit 22 that controls the engine 33. The control device 20 includes a motor control unit 23 that controls the first motor generator 31 and the second motor generator 32 by controlling a power control unit 35. The control device 20 also includes an overall control unit 21 that is connected to the engine control unit 22 and the motor control unit 23 and that overall controls the vehicle 10. These control units are configured with processing circuits and memories that store programs executed by the processing circuits, etc.

[0017] As described above, the control device 20 controls the engine 33, the first motor generator 31, and the second motor generator 32. In other words, the control device 20 controls the powertrain of the vehicle 10. Detection signals from sensors provided in various parts of the vehicle 10 are input to the control device 20. For example, an accelerator position sensor, a brake sensor, and a vehicle speed sensor are connected to the overall control unit 21. For example, a crank position sensor, a water temperature sensor, and an air flow meter are connected to the engine control unit 22. The crank position sensor outputs a crank angle signal each time the crankshaft rotates a certain angle. The engine control unit 22 calculates the rotational phase of the crankshaft and the engine rotation speed NE, which is the rotational speed of the crankshaft, based on the crank angle signal.

[0018] The motor control unit 23 receives the current, voltage, and temperature of the battery via the power control unit 35. Based on the current, voltage, and temperature, the motor control unit 23 calculates the ratio of the remaining charge to the battery's charge capacity.

[0019] The engine control unit 22 and the motor control unit 23 are each connected to the overall control unit 21 via a communication line. The overall control unit 21, the motor control unit 23, and the engine control unit 22 mutually exchange and share information based on detection signals input from sensors and calculated information via CAN communication.

[0020] <About cranking> As described above, the first motor generator 31 is a motor that drives the crankshaft of the engine 33 to crank the engine 33. When starting the engine 33, the control device 20 drives the first motor generator 31 using the motor control unit 23 to achieve cranking.

[0021] 3 shows the transition of MG torque when starting the engine 33, the transition of whether or not ignition control is performed in the engine 33, and the transition of the engine rotation speed NE. The MG torque is the torque of the first motor generator 31. As shown in FIG. 3(a), when starting of the engine 33 begins at time t_0, the motor control unit 23 increases the MG torque and starts cranking to rotate the crankshaft using the driving force of the first motor generator 31.

[0022] As shown in FIG. 3(c), when the engine speed NE reaches the predetermined speed NEx at time t_1, ignition control in the engine 33 is turned ON as shown in FIG. 3(b), and the engine control unit 22 starts ignition control. When ignition control is performed in this manner, the engine 33 begins to operate autonomously. Therefore, the motor control unit 23 sets the MG torque to 0 as shown in FIG. 3(a) to end cranking. When the engine speed NE converges to the target speed NEt, the start of the engine 33 is completed.

[0023] Until cranking is completed in this manner, the motor control unit 23 controls the MG torque so that the engine 33 can be started quickly and completely while minimizing vibration and noise.

[0024] A control map for cranking is stored in the control device 20. This control map is a function that outputs a command value for the MG torque to the first motor generator 31 according to the elapsed time from the point in time when control of the MG torque for cranking is started.

[0025] For example, as shown by the solid line in Figure 4, this control map stores the MG torque command value for each elapsed time period from the start of control. In the following description, the MG torque command value stored in the control map is referred to as a torque variable. The motor control unit 23 reads out the torque variable for each elapsed time period from this control map. Then, the motor control unit 23 controls the first motor generator 31 in accordance with the read torque variable to perform cranking.

[0026] The control map must be designed to meet various requirements. For example, tests are repeated to find an appropriate combination of torque variables to quickly start the engine 33 while suppressing noise and vibration. The calibration system 100 automatically calibrates the control map.

[0027] As shown in FIG. 1, the adaptation work is performed with the adaptation system 100 connected to the vehicle 10 as shown in FIG. <Adaptation of control map by adaptation system 100> As shown in FIG. 1, the adaptation work is performed with an adaptation system 100 connected to the vehicle 10 as shown in FIG. 1. The adaptation system 100 is connected to a control device 20 of the vehicle 10. When performing the adaptation work, a microphone 50 and an acceleration sensor 51 are attached to the vehicle 10. The microphone 50 and the acceleration sensor 51 are connected to the adaptation system 100. The adaptation system 100 connected to the control device 20 is able to communicate with the control device 20. Therefore, the adaptation system 100 can control the hybrid system 30 via the control device 20. The adaptation system 100 can acquire various data that the control device 20 acquires using sensors mounted on the vehicle 10.

[0028] The adaptation system 100 performs the adaptation of the control map using a method called black-box optimization. This adaptation method causes the processing circuit 101 to perform trials of starting the engine 33 by driving the first motor-generator 31 and cranking the engine 33 while acquiring state variables acquired by sensors with the computational map modified. This adaptation method performs evaluation to calculate a reward based on the acquired state variables. This adaptation method performs learning to update the control map based on the reward. This adaptation method optimizes the control map stored in the control device 20 by repeatedly executing a learning routine including trial, evaluation, and learning by the processing circuit 101. In this embodiment, the state variables include the engine speed NE detected by the crank position sensor, the sound pressure detected by the microphone 50, and the acceleration detected by the acceleration sensor 51.

[0029] In the learning routine, the processing circuit 101 performs a first trial and a second trial in which the control map is modified so as to adjust the torque variable in opposite positive and negative directions. Fig. 4 shows torque variables stored in a control map for cranking. In this example, the period during which cranking control is performed is, for example, 2 seconds. The example shown in Fig. 4 shows torque variables for each elapsed time between time t_0, which is the start point of the control, and time t_10, which is the end point of the control.

[0030] In FIG. 4, the torque variable before the change is shown by a solid line. The change in the first trial is a change in which the torque variable for each elapsed time period in the control map is randomly adjusted within a predetermined adjustment range, as shown by the dashed line in FIG. 4. On the other hand, the change in the second trial is a change in which the adjustment in the first trial is reflected in the control map with the positive and negative reversed, as shown by the dashed-dotted line in FIG. 4. Therefore, as shown in FIG. 4, the torque variable in the first trial and the torque variable in the second trial are located symmetrically with respect to the torque variable before the change.

[0031] In the learning routine, the processing circuit 101 executes a first trial and a second trial while acquiring state variables. Then, the processing circuit 101 performs an evaluation to calculate a reward based on the acquired state variables. For example, the execution period of each trial is up to three seconds. The processing circuit 101 executes the trial until three seconds have elapsed since cranking began or until the engine rotation speed NE converges to the target rotation speed NEt. The processing circuit 101 calculates a score according to the elapsed time so that the reward decreases as the time elapsed from the start of cranking to the end of the trial increases. For example, the processing circuit 101 calculates a score consisting of a negative value so that the absolute value increases as the time elapsed until the end of the trial increases. When sound pressure exceeds a certain level, the processing circuit 101 calculates a score consisting of a negative value so that the absolute value increases as the sound pressure increases. When acceleration exceeds a certain level, the processing circuit 101 calculates a score consisting of a negative value so that the absolute value increases as the acceleration increases. The acceleration sensor 51 detects acceleration in three directions: up / down, left / right, and front / rear. When acceleration in any direction exceeds a certain level, the processing circuit 101 calculates a score consisting of a negative value such that the greater the acceleration, the larger the absolute value. The processing circuit 101 calculates the sum of these scores for one trial as the reward for that trial. The reward is a negative value. Therefore, the processing circuit 101 evaluates the trial as having a larger reward and a higher evaluation when the absolute value of the reward value is smaller.

[0032] Then, in the learning routine, the processing circuit 101 updates the control map by reflecting the change in the control map in the trial with the larger reward out of the first trial and the second trial. That is, in the learning routine, the processing circuit 101 updates the control map by reflecting the torque variable in the trial with the larger reward out of the first trial and the second trial in the control map.

[0033] The adaptation system 100 repeatedly executes this learning routine to gradually update the control map so that the reward increases, thereby optimizing the control map so that the engine 33 can be started quickly while minimizing vibration and noise.

[0034] As the optimization of the control map approaches completion, it is desirable for the reward to converge to a larger value and for learning to gradually converge. However, the reward value may continue to fluctuate due to the influence of fluctuations in state variables caused by various external factors, such as noise in the signals from the sensors and differences in the combustion state in the engine 33. As a result, even when the learning routine is executed many times, the torque variable in the control map fluctuates each time the learning routine is executed, making it difficult for learning to converge.

[0035] For example, the transition of the engine speed NE varies depending on the combustion state in the engine 33 after cranking ends. As a result, the time it takes for the engine speed NE to converge to the target speed NEt varies with each trial. In this case, even if the number of times the learning routine is executed increases, the reward continues to fluctuate, making it difficult for learning to converge.

[0036] Therefore, the adaptation system 100 aims to solve these problems by adopting an adaptation method that incorporates a special feature in the final learning routine. <Flow of a series of processes executed by the adaptation system 100> Next, a series of processing steps relating to the adaptation method in the adaptation system 100 will be described with reference to FIGS.

[0037] 5 and 6 are flowcharts showing the flow of a series of processes related to the adaptation method executed by the adaptation system 100. This series of processes is executed by the processing circuit 101 when the adaptation system 100 is connected to the vehicle 10, as shown in FIG.

[0038] 5, in step S100, the processing circuit 101 first initializes a torque variable in a control map stored in the storage device 102. The storage device 102 stores a control map similar to the control map stored in the control device 20 of the vehicle 10. In step S100, the processing circuit 101 initializes the torque variable in this control map to an initial value. This initial value is a torque variable that has not been optimized but is set in advance so that cranking can be performed.

[0039] In the process of step S110, the processing circuit 101 calculates a first variable. The first variable is a torque variable in the control map used in the first trial. In the process of step S110, the processing circuit 101 randomly adjusts the torque variable for each elapsed time in the control map stored in the storage device 102 within a predetermined adjustment range. The first variable is the torque variable adjusted in this way.

[0040] In the next step S120, the processing circuit 101 calculates a second variable. The second variable is a torque variable in the control map used in the second trial. In the process of step S120, the processing circuit 101 calculates the second variable by adjusting the torque variable for each elapsed time in the control map stored in the storage device 102 by reversing the positive and negative of the adjustment in the process of step S100.

[0041] In the next step S130, the processing circuit 101 performs a first trial, which is a trial using the first variable. Specifically, the processing circuit 101 performs an attempt to start the engine 33 of the vehicle 10 using the first variable. That is, the processing circuit 101 performs an attempt to start the engine 33 by having the control device 20 output the first variable as a command value for the MG torque. Then, the processing circuit 101 acquires state variables until the trial ends. The processing circuit 101 calculates scores based on the state variables as described above, and calculates the first reward by summing up the scores. When the first trial ends in this manner, the processing circuit 101 proceeds to step S140.

[0042] In the processing of step S140, the processing circuit 101 performs a second trial, which is a trial using the second variable. Specifically, the processing circuit 101 performs an attempt to start the engine 33 of the vehicle 10 using the second variable. That is, the processing circuit 101 performs an attempt to start the engine 33 by having the control device 20 output the second variable as a command value for the MG torque. Then, the processing circuit 101 acquires state variables until the trial ends. The processing circuit 101 calculates scores in the same way as the first trial and calculates a second reward by summing up the scores. When the second trial ends in this way, the processing circuit 101 proceeds to the processing of step S150.

[0043] In the process of step S150, the processing circuit 101 adds 1 to the number of times the learning routine has been executed, and sets the sum as the new number of times the learning routine has been executed. The initial value of the number of times the learning routine has been executed is 0. Next, in step S160, the processing circuit 101 determines whether the number of executions is less than a predetermined number. The adaptation system 100 repeats the learning routine until the number of executions reaches a termination number to optimize the control map. The termination number is set to, for example, 1,000 times. The default number is less than the termination number. For example, the default number is set to 900 times. The termination number and the default number are hyperparameters that are adjusted in advance when designing the adaptation method.

[0044] In the process of step S160, if the processing circuit 101 determines that the number of executions is less than the predetermined number of times (step S160: YES), the processing circuit 101 proceeds to the process of step S170. Then, in the process of step S170, the processing circuit 101 determines whether the first reward is greater than the second reward.

[0045] In the processing of step S170, if the processing circuit 101 determines that the first reward is greater than the second reward (step S170: YES), the processing circuit 101 proceeds to the processing of step S180. In the processing of step S180, the processing circuit 101 overwrites the control map with the first variable and updates the control map by changing the torque variable of the control map to the first variable. On the other hand, in the processing of step S170, if the processing circuit 101 determines that the first reward is less than or equal to the second reward (step S170: NO), the processing circuit 101 proceeds to the processing of step S190. In the processing of step S190, the processing circuit 101 overwrites the control map with the second variable and updates the control map by changing the torque variable of the control map to the second variable.

[0046] The processing from step S170 to step S190 is processing for updating the control map by reflecting changes in the first trial or the second trial, whichever trial had the larger reward, in the control map. Therefore, when the first reward and the second reward are equal, the processing circuit 101 may change the torque variable of the control map to the first variable. Alternatively, when the first reward and the second reward are equal, the processing circuit 101 may not change the torque variable of the control map.

[0047] After updating the control map by reflecting the changes in the trial with the larger reward out of the first trial and the second trial in the control map, the processing circuit 101 ends the learning routine and returns the process to step S110. That is, until the number of executions reaches the predetermined number, the processing circuit 101 repeatedly executes the learning routine that executes the first process of reflecting the changes in the trial with the larger reward in the control map.

[0048] When the number of executions reaches the predetermined number of times, the processing circuit 101 determines in the processing of step S160 that the number of executions is equal to or greater than the predetermined number of times (step S160: NO). In this case, the processing circuit 101 proceeds to the processing of step S200 shown in FIG. 6.

[0049] In the processing of step S200, the processing circuit 101 determines whether the number of executions is less than the number of terminations. In the processing of step S200, if the processing circuit 101 determines that the number of executions is less than the number of terminations (step S200: YES), the processing circuit 101 proceeds to the processing of step S210. Then, in the processing of step S210, the processing circuit 101 determines whether the first reward is greater than the second reward.

[0050] In the processing of step S210, if the processing circuit 101 determines that the first reward is greater than the second reward (step S210: YES), the processing circuit 101 proceeds to the processing of step S220. In the processing of step S220, the processing circuit 101 records the first variable in the storage device 102 without reflecting the first variable in the control map. On the other hand, in the processing of step S210, if the processing circuit 101 determines that the first reward is less than or equal to the second reward (step S210: NO), the processing circuit 101 proceeds to the processing of step S230. In the processing of step S230, the processing circuit 101 records the second variable in the storage device 102 without reflecting the second variable in the control map.

[0051] The processing from step S210 to step S230 is processing for recording the change in the trial with the larger reward out of the first trial and the second trial in the storage device 102 without reflecting it in the control map. Therefore, when the first reward and the second reward are equal, the processing circuit 101 may record the first variable in the storage device 102.

[0052] After recording the change in the trial with the larger reward out of the first trial and the second trial in the storage device 102 in this way, the processing circuit 101 ends the learning routine and returns the process to step S110 shown in Fig. 5. In other words, from the time when the number of executions reaches or exceeds the predetermined number of times until the end number of times is reached, the processing circuit 101 repeatedly executes the learning routine that executes the second process of recording the change in the trial with the larger reward in the storage device 102 without reflecting it in the control map.

[0053] When the execution count reaches the termination count, the processing circuit 101 determines in the processing of step S200 that the execution count is equal to or greater than the termination count (step S200: NO). In this case, the processing circuit 101 proceeds to the processing of step S240 shown in FIG. 6.

[0054] In the process of step S240, the processing circuit 101 calculates average values ​​as summary statistics of the first variable and the second variable recorded in the memory device 102. That is, the processing circuit 101 calculates average values ​​of the torque variables recorded in the memory device 102 without reflecting them in the control map while repeating the learning routine of the second step. The average values ​​are calculated for each elapsed time period.

[0055] Next, in step S250, processing circuit 101 overwrites the control map with this average value and updates the torque variable of the control map to this average value. Then, in the next step S260, processing circuit 101 records the torque variable of the control map stored in storage device 102 in storage device 102 as a control map to be stored in control device 20, thereby completing optimization of the control map.

[0056] The processing from step S240 to step S260 is a learning routine that executes a third process that completes optimization of the control map by reflecting changes based on summary statistics of the multiple changes recorded in storage device 102 in the control map.

[0057] The optimized control map data recorded in the storage device 102 of the adaptation system 100 is then stored in the control device 20 of the vehicle 10. This allows the vehicle 10 to quickly complete the start of the engine 33 while minimizing vibration and noise.

[0058] <Operation of this embodiment> The adaptation method performed by the adaptation system 100 includes a first step of performing a first process, a second step of performing a second process, and a third step of performing a third process. In this adaptation method, the second step and the third step, which are performed at the end of the optimization, convert the changes obtained through trials into summary statistics, smooth them, and then reflect them in the control map.

[0059] According to the above-described adaptation method and adaptation system 100, when the number of times the learning routine has been executed reaches a predetermined number, changes to the control map are put on hold and stored in the storage device 102. Then, when the number of times the learning routine has been executed reaches the termination number, a final update of the control map is performed based on summary statistics that reflect information about the multiple changes stored in the storage device 102. Therefore, according to the above-described adaptation method and adaptation system 100, even if fluctuations occur in the state variables in the final stage of optimization due to noise in the sensor signals or the like, the influence of such fluctuations is unlikely to be reflected in the learning.

[0060] <Effects of this embodiment> (1) Fluctuations in state variables due to noise in sensor signals, etc., can be prevented from hindering the convergence of learning at the final stage of control map optimization.

[0061] (2) The above-described adaptation system 100 employs average values ​​as summary statistics. The average values ​​reflect smoothed information on multiple changes recorded in the storage device 102. With the above-described configuration using average values, even if fluctuations in the state variables occur toward the end of optimization due to noise in the sensor signals, the influence of such fluctuations is less likely to be reflected in learning.

[0062] (3) The state variables include the engine rotation speed NE detected by the crank position sensor, the sound pressure detected by the microphone 50, and the acceleration detected by the acceleration sensor 51. Therefore, the adaptation system 100 can optimize the control map by reflecting information on noise and vibration during cranking.

[0063] <Example of change> This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility.

[0064] In the above embodiment, the mean value is used as the summary statistic. However, a configuration using other summary statistic values ​​may be adopted. For example, the median value may be used as the summary statistic. In the above embodiment, the adaptation of the control map of the first motor generator 31 when cranking the engine 33 is illustrated. The above-described adaptation method can also be applied to other motor controls. For example, it can be applied to the adaptation of functions used to control a drive motor for an electric vehicle or a motor that drives an electric actuator. [Explanation of symbols]

[0065] 10...vehicle, 20...control device, 21...overall control unit, 22...engine control unit, 23...motor control unit, 30...hybrid system, 31...first motor generator, 32...second motor generator, 33...engine, 34...power split mechanism, 35...power control unit, 36...reduction mechanism, 40...drive wheel, 50...microphone, 51...acceleration sensor, 100...adaptation system, 101...processing circuit, 102...storage device

Claims

1. a processing circuit and a storage device; the processing circuit is an adaptation system that optimizes the function stored in a control device that controls the motor by repeating a learning routine including: trials of driving the motor while acquiring state variables by a sensor in a state in which a function that outputs a command value to the motor has been modified; evaluation of calculating a reward based on the acquired state variables; and learning of updating the function based on the reward; the processing circuitry If the number of times the learning routine has been executed is less than a predetermined number of times, a first trial and a second trial are performed in each learning routine, in which the change is made to the function so that the command value output from the function is adjusted in the opposite direction in positive and negative, respectively, and a first process is executed in which the change in the trial in which the reward was greater, of the first trial and the second trial, is reflected in the function to update the function and terminate the learning routine; when the number of executions of the learning routine is equal to or greater than the predetermined number and less than a number of terminations that is greater than the predetermined number, a second process is executed in which the first trial and the second trial are performed in each learning routine, and the change in the trial with the larger reward out of the first trial and the second trial is recorded in the storage device without being reflected in the function, thereby terminating the learning routine; When the number of times the learning routine has been executed reaches the termination number, a third process is executed in which summary statistics of the plurality of changes that are not reflected in the function and are stored in the storage device are calculated, and the changes based on the summary statistics are reflected in the function, thereby terminating the learning routine and completing optimization of the function. Compatible systems.

2. the function is a control map for outputting the command value to the motor in accordance with the elapsed time from the start of control, the change in the first trial is the change that randomly adjusts the value of the command value for each elapsed time period in the control map within a predetermined adjustment range, The change in the second trial is a change in which the adjustment of the command value for each elapsed time period in the change in the first trial is reversed in positive and negative and applied to the value of the command value for each elapsed time period in the control map. The matching system of claim 1 .

3. The summary statistic is a mean The matching system of claim 1 .

4. the function is a function used for control when the motor drives a crankshaft of an engine mounted on a vehicle to crank the engine, the attempt is an attempt to start the engine by cranking the engine with the motor; the sensors include a crank position sensor that detects an engine rotation speed of the engine, a microphone that detects a sound emitted from the vehicle, and an acceleration sensor that detects vibrations of the vehicle; The state variables include an engine rotation speed detected by the crank position sensor, a sound pressure detected by the microphone, and an acceleration detected by the acceleration sensor. The matching system of claim 1 .

5. Using an adaptation system having a processing circuit and a storage device, an adaptation method for optimizing the function stored in a control device that controls the motor by repeatedly executing a learning routine in the processing circuit, the learning routine including: trials of driving the motor while acquiring state variables by a sensor in a state in which a function for outputting a command value to the motor has been modified; evaluation of calculating a reward based on the acquired state variables; and learning of updating the function based on the reward; a first step of causing the processing circuit to execute a first process when the number of times the learning routine has been executed is less than a predetermined number of times, in which a first trial and a second trial are performed in each learning routine by making the change to the function so that the command value output by the function is adjusted in the opposite direction in positive and negative, and updating the function by reflecting the change in the trial of the first trial or the second trial in which the reward was greater, and terminating the learning routine; a second step of causing the processing circuit to execute a second process in which, when the number of executions of the learning routine is equal to or greater than the predetermined number and less than a number of terminations greater than the predetermined number, the first trial and the second trial are performed in each learning routine, and the change in the trial with the larger reward among the first trial and the second trial is recorded in the storage device without being reflected in the function, thereby terminating the learning routine; and a third step of causing the processing circuit to execute a third process, when the number of times the learning routine has been executed reaches the termination number, of calculating summary statistics of the plurality of changes that are not reflected in the function and are stored in the storage device, and reflecting the changes based on the summary statistics in the function, thereby terminating the learning routine and completing optimization of the function. How it fits.

Images