Adaptation system and adaptation method

Abstract

To provide an adaptation system that can prevent fluctuations of a state variable due to noise in sensor's signals or the like from disturbing convergence of learning in the final stage of optimization of a control map.SOLUTION: A processing circuit of an adaptation system, until a predetermined condition is met (step S130: NO), performs a first trial and a second trial in each learning routine, and executes first processing (from step S140 to step S180) of reflecting, in a control map, a change in the trial with a larger reward, and ending the learning routine. After the predetermined condition is met (step S130: YES), the processing circuit performs the first trial and the second trial a plurality of times in each learning routine (step S190 and step S200), and executes second processing of comparing the reward for a plurality of executions of the first trial with the reward for a plurality of executions of the second trial, reflecting, in a control map, a change in the trial with the larger reward, and ending the learning routine.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a fitting method and system for optimizing a function used in controlling a motor. [Background technology]

[0002] Patent Document 1 discloses a machine learning device that learns a current command value for a motor. In the learning process, this machine learning device acquires state variables while driving the motor. The machine learning device then calculates a reward based on the state variables. The machine learning device learns a current command value based on the reward. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-014838 Summary of the Invention [Problem to be solved by the invention]

[0004] When using a calibration system to automatically optimize a function for outputting a command value to a motor, the calibration system performs trials to acquire state variables while driving the motor. The calibration system evaluates the results of the trials using a reward calculated based on the state variables. The calibration system then updates and learns the function based on the evaluation. In this way, the calibration system optimizes the function by gradually updating it through repeated trials, evaluations, and learning.

[0005] It is desirable for the learning to gradually converge as the optimization of the function approaches completion. However, there are cases where the learning is difficult to converge due to the influence of accidental fluctuations in the state variables caused by noise in the signals from the sensors, etc. [Means for solving the problem]

[0006] The means for solving the above problems and their effects will be described below. A calibration system for solving the above problem includes a processing circuit and a storage device. In this calibration system, the processing circuit optimizes the function stored in a control device that controls the motor by repeating a learning routine including: trials of driving the motor while acquiring state variables via a sensor with a change made to the function that outputs a command value to the motor; evaluation of calculating a reward based on the acquired state variables; and learning of updating the function based on the reward. In this calibration system, the processing circuit executes a first process of performing first and second trials in which the change is made to the function so that the command value output from the function is adjusted in opposite directions in each learning routine until a predetermined condition for determining that the optimization has progressed to the end is met, updating the function by reflecting the change in one of the first and second trials that resulted in the larger reward, and terminating the learning routine. In this adaptation system, after the predetermined condition is met, the processing circuit performs a second process in which the first trial and the second trial are performed multiple times in each learning routine, the rewards for the multiple first trials are compared with the rewards for the multiple second trials, and the change in the trial with the larger reward between the first trial and the second trial is reflected in the function, thereby updating the function and terminating the learning routine.

[0007] An adaptation method for solving the above problem is an adaptation method that uses an adaptation system including a processing circuit and a storage device to optimize a function stored in a control device that controls a motor. This adaptation method optimizes the function by repeatedly executing a learning routine in the processing circuit, the learning routine including: trials of driving the motor while acquiring state variables by a sensor with a change made to a function that outputs a command value to the motor; evaluation of calculating a reward based on the acquired state variables; and learning of updating the function based on the reward. This adaptation method includes a first step of causing the processing circuit to execute a first process of performing first and second trials in which the change is made to the function so that the command value output by the function is adjusted in opposite positive and negative directions in each learning routine until a predetermined condition for determining that the optimization has progressed to the end is met, updating the function by reflecting the change in one of the first and second trials that resulted in the larger reward, and ending the learning routine. This adaptation method includes a second step of causing the processing circuit to execute a second process in which, after the predetermined condition is met, the first trial and the second trial are performed multiple times in each learning routine, the rewards for the multiple first trials are compared with the rewards for the multiple second trials, and the change in one of the first and second trials with the larger reward is reflected in the function, thereby updating the function and terminating the learning routine. [Effects of the Invention]

[0008] This can prevent fluctuations in state variables due to noise in sensor signals, etc., from hindering the convergence of learning at the end of function optimization. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of the adaptation system and the relationship between the adaptation system and a vehicle. [Figure 2] FIG. 2 is a schematic diagram showing the configuration of a powertrain of a vehicle. [Figure 3]FIG. 3 is a time chart showing (a) the transition of the torque of the motor generator, (b) the presence or absence of ignition, and (c) the transition of the engine rotation speed when starting the engine. [Figure 4] FIG. 4 is an explanatory diagram for explaining an example of a control map of a torque command value. [Figure 5] FIG. 5 is a flowchart showing the flow of a series of processes executed by the adaptation system. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the adaptation system will now be described with reference to FIGS. <Configuration of the adaptation system 100> As shown in FIG. 1, the adaptation system 100 includes a processing circuit 101 and a storage device 102. The storage device 102 stores programs and data. The processing circuit 101 executes the programs stored in the storage device 102. The adaptation system 100 optimizes functions to be stored in a control device 20 mounted on a vehicle 10. The vehicle 10 is equipped with a hybrid system 30. The control device 20 controls the hybrid system 30.

[0011] <Powertrain configuration of vehicle 10> 2, the power train of vehicle 10 includes a hybrid system 30. Vehicle 10 drives drive wheels 40 using hybrid system 30. Hybrid system 30 of vehicle 10 includes a first motor generator 31, a second motor generator 32, an engine 33, a power split mechanism 34, a power control unit 35, and a reduction mechanism 36.

[0012] The second motor generator 32 is connected to a power control unit 35. The second motor generator 32 is connected to drive wheels 40 via a reduction gear mechanism 36. The engine 33 is connected to drive wheels 40 via a power split mechanism 34 and the reduction gear mechanism 36. The first motor generator 31 is connected to the power split mechanism 34. The first motor generator 31 is, for example, a three-phase AC motor generator.

[0013] The power split mechanism 34 is made up of planetary gears. The power split mechanism 34 can split the driving force of the engine 33 between the first motor generator 31 and the drive wheels 40. The first motor generator 31 generates electricity using the driving force of the engine 33, and also generates electricity using the driving force from the drive wheels 40. The first motor generator 31 drives the crankshaft of the engine 33 when starting the engine 33. Therefore, the first motor generator 31 is a motor that drives the crankshaft of the engine 33 to crank the engine 33.

[0014] The first motor generator 31 and the second motor generator 32 are connected to a battery via a power control unit 35. The AC power generated by the first motor generator 31 is converted to DC by the power control unit 35 and charged into the battery. In other words, the power control unit 35 functions as an inverter.

[0015] The DC power of the battery is converted to AC by the power control unit 35 and supplied to the second motor generator 32. When the vehicle 10 is decelerating, the second motor generator 32 generates electricity using the driving force from the drive wheels 40. The generated electricity is then charged to the battery. In other words, the vehicle 10 performs regenerative charging. At this time, the second motor generator 32 functions as a generator. The AC power generated by the second motor generator 32 is converted to DC by the power control unit 35 and charged to the battery. When the engine 33 is cranked by the first motor generator 31, the power control unit 35 converts the DC power of the battery to AC and supplies it to the first motor generator 31.

[0016] <Regarding the control device 20> The control device 20 controls the engine 33, the first motor generator 31, and the second motor generator 32. The control device 20 includes an engine control unit 22 that controls the engine 33. The control device 20 includes a motor control unit 23 that controls the first motor generator 31 and the second motor generator 32 by controlling a power control unit 35. The control device 20 further includes an overall control unit 21 that is connected to the engine control unit 22 and the motor control unit 23 and that overall controls the vehicle 10. These control units are composed of processing circuits and memories that store programs executed by the processing circuits, etc.

[0017] As described above, the control device 20 controls the engine 33, the first motor generator 31, and the second motor generator 32. In other words, the control device 20 controls the powertrain of the vehicle 10. Detection signals from sensors provided in various parts of the vehicle 10 are input to the control device 20. For example, an accelerator position sensor, a brake sensor, and a vehicle speed sensor are connected to the overall control unit 21. For example, a crank position sensor, a water temperature sensor, and an air flow meter are connected to the engine control unit 22. The crank position sensor outputs a crank angle signal each time the crankshaft rotates a certain angle. The engine control unit 22 calculates the rotational phase of the crankshaft and the engine rotation speed NE, which is the rotational speed of the crankshaft, based on the crank angle signal.

[0018] The motor control unit 23 receives the current, voltage, and temperature of the battery via the power control unit 35. Based on the current, voltage, and temperature, the motor control unit 23 calculates the ratio of the remaining charge to the battery's charge capacity.

[0019] The engine control unit 22 and the motor control unit 23 are each connected to the overall control unit 21 via a communication line. The overall control unit 21, the motor control unit 23, and the engine control unit 22 mutually exchange and share information based on detection signals input from sensors and calculated information via CAN communication.

[0020] <About cranking> As described above, the first motor generator 31 is a motor that drives the crankshaft of the engine 33 to crank the engine 33. When starting the engine 33, the control device 20 drives the first motor generator 31 using the motor control unit 23 to achieve cranking.

[0021] 3 shows the transition of MG torque when starting the engine 33, the transition of whether or not ignition control is performed in the engine 33, and the transition of the engine rotation speed NE. The MG torque is the torque of the first motor generator 31. As shown in FIG. 3(a), when starting of the engine 33 begins at time t_0, the motor control unit 23 increases the MG torque and starts cranking to rotate the crankshaft using the driving force of the first motor generator 31.

[0022] As shown in FIG. 3(c), when the engine speed NE reaches the predetermined speed NEx at time t_1, ignition control in the engine 33 is turned ON as shown in FIG. 3(b), and the engine control unit 22 starts ignition control. When ignition control is performed in this manner, the engine 33 begins to operate autonomously. Therefore, the motor control unit 23 sets the MG torque to 0 as shown in FIG. 3(a) to end cranking. When the engine speed NE converges to the target speed NEt, the start of the engine 33 is completed.

[0023] Until cranking is completed in this manner, the motor control unit 23 controls the MG torque so that the engine 33 can be started quickly and completely while minimizing vibration and noise.

[0024] A control map for cranking is stored in the control device 20. This control map is a function that outputs a command value for the MG torque to the first motor generator 31 according to the elapsed time from the point in time when control of the MG torque for cranking is started.

[0025] For example, as shown by the solid line in Figure 4, this control map stores the MG torque command value for each elapsed time period from the start of control. In the following description, the MG torque command value stored in the control map is referred to as a torque variable. The motor control unit 23 reads out the torque variable for each elapsed time period from this control map. Then, the motor control unit 23 controls the first motor generator 31 in accordance with the read torque variable to perform cranking.

[0026] The control map must be designed to meet various requirements. For example, tests are repeated to find an appropriate combination of torque variables to quickly start the engine 33 while suppressing noise and vibration. The calibration system 100 automatically calibrates the control map.

[0027] As shown in FIG. 1, the adaptation work is performed with the adaptation system 100 connected to the vehicle 10 as shown in FIG. <Adaptation of control map by adaptation system 100> As shown in FIG. 1, the adaptation work is performed with an adaptation system 100 connected to the vehicle 10 as shown in FIG. 1. The adaptation system 100 is connected to a control device 20 of the vehicle 10. When performing the adaptation work, a microphone 50 and an acceleration sensor 51 are attached to the vehicle 10. The microphone 50 and the acceleration sensor 51 are connected to the adaptation system 100. The adaptation system 100 connected to the control device 20 is able to communicate with the control device 20. Therefore, the adaptation system 100 can control the hybrid system 30 via the control device 20. The adaptation system 100 can acquire various data that the control device 20 acquires using sensors mounted on the vehicle 10.

[0028] The adaptation system 100 performs the adaptation of the control map using a method called black-box optimization. This adaptation method causes the processing circuit 101 to perform trials of starting the engine 33 by driving the first motor-generator 31 and cranking the engine 33 while acquiring state variables acquired by sensors with the computational map modified. This adaptation method performs evaluation to calculate a reward based on the acquired state variables. This adaptation method performs learning to update the control map based on the reward. This adaptation method optimizes the control map stored in the control device 20 by repeatedly executing a learning routine including trial, evaluation, and learning by the processing circuit 101. In this embodiment, the state variables include the engine speed NE detected by the crank position sensor, the sound pressure detected by the microphone 50, and the acceleration detected by the acceleration sensor 51.

[0029] In the learning routine, the processing circuit 101 performs a first trial and a second trial in which the control map is modified so as to adjust the torque variable in opposite positive and negative directions. Fig. 4 shows torque variables stored in a control map for cranking. In this example, the period during which cranking control is performed is, for example, 2 seconds. The example shown in Fig. 4 shows torque variables for each elapsed time between time t_0, which is the start point of the control, and time t_10, which is the end point of the control.

[0030] In FIG. 4, the torque variable before the change is shown by a solid line. The change in the first trial is a change in which the torque variable for each elapsed time period in the control map is randomly adjusted within a predetermined adjustment range, as shown by the dashed line in FIG. 4. On the other hand, the change in the second trial is a change in which the adjustment in the first trial is reflected in the control map with the positive and negative reversed, as shown by the dashed-dotted line in FIG. 4. Therefore, as shown in FIG. 4, the torque variable in the first trial and the torque variable in the second trial are located symmetrically with respect to the torque variable before the change.

[0031] In the learning routine, the processing circuit 101 executes a first trial and a second trial while acquiring state variables. Then, the processing circuit 101 performs an evaluation to calculate a reward based on the acquired state variables. For example, the execution period of each trial is up to three seconds. The processing circuit 101 executes the trial until three seconds have elapsed since cranking began or until the engine rotation speed NE converges to the target rotation speed NEt. The processing circuit 101 calculates a score according to the elapsed time so that the reward decreases as the time elapsed from the start of cranking to the end of the trial increases. For example, the processing circuit 101 calculates a score consisting of a negative value so that the absolute value increases as the time elapsed until the end of the trial increases. When sound pressure exceeds a certain level, the processing circuit 101 calculates a score consisting of a negative value so that the absolute value increases as the sound pressure increases. When acceleration exceeds a certain level, the processing circuit 101 calculates a score consisting of a negative value so that the absolute value increases as the acceleration increases. The acceleration sensor 51 detects acceleration in three directions: up / down, left / right, and front / rear. When acceleration in any direction exceeds a certain level, the processing circuit 101 calculates a score consisting of a negative value such that the greater the acceleration, the larger the absolute value. The processing circuit 101 calculates the sum of these scores for one trial as the reward for that trial. The reward is a negative value. Therefore, the processing circuit 101 evaluates the trial as having a larger reward and a higher evaluation when the absolute value of the reward value is smaller.

[0032] Then, in the learning routine, the processing circuit 101 updates the control map by reflecting the change in the control map in the trial with the larger reward out of the first trial and the second trial. That is, in the learning routine, the processing circuit 101 updates the control map by reflecting the torque variable in the trial with the larger reward out of the first trial and the second trial in the control map.

[0033] The adaptation system 100 repeatedly executes this learning routine to gradually update the control map so that the reward increases, thereby optimizing the control map so that the engine 33 can be started quickly while minimizing vibration and noise.

[0034] As the optimization of the control map approaches completion, it is desirable for the reward to converge to a larger value and for learning to gradually converge. However, the value of the reward may continue to fluctuate due to the influence of accidental fluctuations in the state variables caused by various external factors, such as noise in the signals from the sensors and differences in the combustion state in the engine 33. As a result, even when the learning routine is executed many times, the torque variable in the control map fluctuates each time the learning routine is executed, making it difficult for learning to converge.

[0035] For example, the transition of the engine speed NE varies depending on the combustion state in the engine 33 after cranking ends. As a result, the time it takes for the engine speed NE to converge to the target speed NEt varies with each trial. In this case, even if the number of times the learning routine is executed increases, the reward continues to fluctuate, making it difficult for learning to converge.

[0036] Therefore, the adaptation system 100 aims to solve these problems by adopting an adaptation method that incorporates a special feature in the final learning routine. <Flow of a series of processes executed by the adaptation system 100> Next, a series of processing steps relating to the adaptation method in the adaptation system 100 will be described with reference to FIG.

[0037] 5 is a flowchart showing the flow of a series of processes related to the adaptation method executed by the adaptation system 100. This series of processes is executed by the processing circuit 101 when the adaptation system 100 is connected to the vehicle 10, as shown in FIG.

[0038] As shown in Fig. 5, first, in step S100, the processing circuit 101 initializes a torque variable in a control map stored in the storage device 102. The storage device 102 stores a control map similar to the control map stored in the control device 20 of the vehicle 10. In step S100, the processing circuit 101 initializes the torque variable in this control map to an initial value. This initial value is a torque variable that has not been optimized but is set in advance so that the crankshaft can be rotated.

[0039] In the process of step S110, the processing circuit 101 calculates a first variable. The first variable is a torque variable in the control map used in the first trial. In the process of step S110, the processing circuit 101 randomly adjusts the torque variable for each elapsed time in the control map stored in the storage device 102 within a predetermined adjustment range. The first variable is the torque variable adjusted in this way.

[0040] In the next step S120, the processing circuit 101 calculates a second variable. The second variable is a torque variable in the control map used in the second trial. In the process of step S120, the processing circuit 101 calculates the second variable by adjusting the torque variable for each elapsed time in the control map stored in the storage device 102 by reversing the positive and negative of the adjustment in the process of step S100.

[0041] In the next step S130, the processing circuit 101 determines whether a predetermined condition for determining whether the optimization has progressed to the end stage is satisfied. The predetermined condition is, for example, that the number of times the learning routine has been executed is equal to or greater than a predetermined number. The adaptation system 100 repeats the learning routine until the number of executions reaches a termination number to optimize the control map. The termination number is set to, for example, 1,000 times. The preset number is less than the termination number. For example, the preset number is set to 900 times. The termination number and the preset number are hyperparameters that are adjusted in advance when designing the adaptation method. In the processing of step S130, if the number of executions is equal to or greater than the preset number, the processing circuit 101 determines that the predetermined condition is satisfied. In other words, the preset number is a threshold value for the number of executions for determining whether the optimization has progressed to the end stage.

[0042] The predetermined condition may be any condition that can determine whether optimization has progressed to the end. As optimization progresses, the reward for each trial decreases. Therefore, the predetermined condition may be that the reward for the previous learning routine is less than a predetermined value. As optimization progresses, the reward may not decrease even when the learning routine is repeated. Therefore, the predetermined condition may be that the decrease in the reward has stagnated. For example, it may be determined that the decrease in the reward has stagnated when the difference between the reward for the learning routine two times before last and the reward for the previous learning routine remains less than a predetermined value. As optimization progresses, the engine speed NE gradually approaches the target speed NEt during each trial. Therefore, the predetermined condition may be that the engine speed NE during the trial in the previous learning routine has reached or exceeded a predetermined speed.

[0043] If the processing circuit 101 determines in the process of step S130 that the predetermined condition is not met (step S130: NO), the processing circuit 101 proceeds to the process of step S140.

[0044] In the processing of step S140, the processing circuit 101 performs a first trial, which is a trial using the first variable. Specifically, the processing circuit 101 performs an attempt to start the engine 33 of the vehicle 10 using the first variable. That is, the processing circuit 101 performs an attempt to start the engine 33 by having the control device 20 output the first variable as a command value for the MG torque. Then, the processing circuit 101 acquires state variables until the end of the trial. The processing circuit 101 calculates scores based on the state variables as described above, and calculates the first reward by summing up the scores. When the first trial ends in this manner, the processing circuit 101 proceeds to the processing of step S150.

[0045] In the processing of step S150, the processing circuit 101 performs a second trial, which is a trial using the second variable. Specifically, the processing circuit 101 performs a trial to start the engine 33 of the vehicle 10 using the second variable. That is, the processing circuit 101 performs a trial to start the engine 33 by having the control device 20 output the second variable as a command value for the MG torque. Then, the processing circuit 101 acquires state variables until the trial ends. The processing circuit 101 calculates scores in the same way as the first trial and calculates a second reward by summing up the scores. When the second trial ends in this way, the processing circuit 101 proceeds to the processing of step S160.

[0046] In the process of step S160, the processing circuit 101 determines whether the first reward is greater than the second reward. In the processing of step S160, if the processing circuit 101 determines that the first reward is greater than the second reward (step S160: YES), the processing circuit 101 proceeds to the processing of step S170. In the processing of step S170, the processing circuit 101 overwrites the control map with the first variable and updates the control map by changing the torque variable of the control map to the first variable. On the other hand, in the processing of step S160, if the processing circuit 101 determines that the first reward is less than or equal to the second reward (step S160: NO), the processing circuit 101 proceeds to the processing of step S180. In the processing of step S180, the processing circuit 101 overwrites the control map with the second variable and updates the control map by changing the torque variable of the control map to the second variable.

[0047] The processing from step S160 to step S180 is processing for updating the control map by reflecting in the control map a change in one of the first and second trials, whichever trial had the larger reward. When the first reward and the second reward are equal, the processing circuit 101 may change the torque variable of the control map to the first variable. Alternatively, when the first reward and the second reward are equal, the processing circuit 101 may not change the torque variable of the control map.

[0048] The processing circuit 101 updates the control map by reflecting the changes in one of the first and second trials, whichever trial had the larger reward, and then ends the learning routine.

[0049] In this way, when the predetermined condition is not met, the processing circuit 101 performs a first trial and a second trial, and executes a first process in which the change in the trial with the larger reward between the first trial and the second trial is reflected in the control map.

[0050] Next, in the process of step S220, the processing circuit 101 adds 1 to the number of times the learning routine has been executed, and sets the sum as the new number of times the learning routine has been executed. The initial value of the number of times the learning routine has been executed is 0. Then, in the process of step S230, the processing circuit 101 determines whether the number of times the learning routine has been executed is less than the number of times the learning routine has ended. If, in the process of step S230, the processing circuit 101 determines that the number of times the learning routine has been executed is less than the number of times the learning routine has ended (step S230: YES), the processing circuit 101 returns the process to step S110. In this way, the processing circuit 101 repeatedly executes the first process until the predetermined condition is met.

[0051] Then, when the predetermined condition is met, the processing circuit 101 determines in step S130 that the predetermined condition is met (step S130: YES). If the processing circuit 101 determines in step S130 that the predetermined condition is met (step S130: YES), the processing circuit 101 proceeds to step S190.

[0052] In the process of step S190, the processing circuit 101 performs a first trial, which is a trial using the first variable, multiple times. In this embodiment, the first trial is performed three times. The processing circuit 101 calculates a score based on the state variable for each of the multiple first trials and calculates the first reward by summing the scores. When the multiple first trials are completed in this manner, the processing circuit 101 proceeds to the process of step S150.

[0053] In the processing of step S200, the processing circuit 101 performs a second trial, which is a trial using the second variable, multiple times. The number of times the first trial is performed in the processing of step S190 is the same as the number of times the second trial is performed in the processing of step S200. In this embodiment, the second trial is performed three times. As in the processing of step S190, the processing circuit 101 calculates a score for each of the multiple second trials and calculates the second reward by summing the scores. When the multiple second trials are completed in this manner, the processing circuit 101 proceeds to the processing of step S210.

[0054] In the process of step S210, the processing circuit 101 determines whether the first average reward is greater than the second average reward. The first average reward is the average value of the first rewards calculated through the process of step S190 for multiple times. The second average reward is the average value of the second rewards calculated through the process of step S200 for multiple times.

[0055] In the processing of step S210, if the processing circuit 101 determines that the first average reward is greater than the second average reward (step S210: YES), the processing circuit 101 proceeds to the processing of step S170. In the processing of step S170, the processing circuit 101 overwrites the first variable onto the control map, and updates the control map by changing the torque variable of the control map to the first variable. On the other hand, in the processing of step S210, if the processing circuit 101 determines that the first average reward is less than or equal to the second average reward (step S210: NO), the processing circuit 101 proceeds to the processing of step S180. In the processing of step S180, the processing circuit 101 overwrites the control map with the second variable, and updates the control map by changing the torque variable of the control map to the second variable. The process consisting of steps S210, S170, and S180 compares the rewards for multiple first trials with the rewards for multiple second trials, and reflects changes in one of the first and second trials with a larger reward in the control map. If the first average reward and the second average reward are equal, the processing circuit 101 may change the torque variable of the control map to the first variable. Alternatively, if the first average reward and the second average reward are equal, the processing circuit 101 may not change the torque variable of the control map. After updating the control map in this way, the processing circuit 101 ends the learning routine.

[0056] In this way, when the predetermined conditions are met, the processing circuit 101 performs the first trial and the second trial multiple times, and executes a second process in which the changes in one of the first and second trials that resulted in a larger reward are reflected in the control map.

[0057] In the process of step S230, if the processing circuit 101 determines that the number of executions is less than the termination number (step S230: YES), the processing circuit 101 returns the process to step S110. In this way, after the predetermined condition is established, the processing circuit 101 repeatedly executes the second process until the number of executions reaches the termination number.

[0058] When the number of executions reaches the termination number, the processing circuit 101 determines in the processing of step S230 that the number of executions is equal to or greater than the termination number (step S230: NO). In this case, the processing circuit 101 proceeds to the processing of step S240.

[0059] In the process of step S240, the processing circuit 101 records the torque variables of the control map stored in the storage device 102 in the storage device 102 as a control map to be stored in the control device 20, thereby completing the optimization of the control map.

[0060] The optimized control map data recorded in the storage device 102 of the adaptation system 100 is then stored in the control device 20 of the vehicle 10. This allows the vehicle 10 to quickly complete the start of the engine 33 while minimizing vibration and noise.

[0061] <Operation of this embodiment> The adaptation method executed by the adaptation system 100 includes a first step of executing a first process and a second step of executing a second process. In this adaptation method, after a predetermined condition for determining that the optimization has progressed to the end is met, a learning routine in the second step is executed. In the learning routine in the second step, multiple trials are performed without changing the torque variable, and the torque variable in one of the first and second trials that has a larger reward is adopted. In this case, the rewards from the multiple trials are used when comparing the magnitudes. Therefore, even if an accidental fluctuation in the state variable occurs in one of the trials due to noise in the signal from the sensor, it is less likely to affect the determination of the torque variable.

[0062] <Effects of this embodiment> (1) Fluctuations in state variables due to noise in sensor signals, etc., can be prevented from hindering the convergence of learning at the final stage of control map optimization.

[0063] (2) The state variables include the engine rotation speed NE detected by the crank position sensor, the sound pressure detected by the microphone 50, and the acceleration detected by the acceleration sensor 51. Therefore, the adaptation system 100 can optimize the control map by reflecting information on noise and vibration during cranking.

[0064] <Example of change> This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility.

[0065] In the second process, the control map is updated by reflecting changes in one of the first and second trials, in which the average value of the reward for multiple trials is greater than that of the other, and the learning routine is terminated. The first and second trials are each performed multiple times, and the second process is executed to reflect changes in the one of the first and second trials, in which the reward was greater, in the control map. Specific aspects of the second process are not limited to these aspects. For example, the aspect of comparing the rewards for multiple trials is not limited to comparing the average values. For example, if the first and second trials are each performed three times in the learning routine, the processing circuit 101 compares the rewards for the first trials, the rewards for the second trials, and the rewards for the third trials. Then, the processing circuit 101 reflects changes in the one of the first and second trials, in which the reward was determined to be greater more often, in the control map. Such aspects may also be employed. Alternatively, for example, the processing circuit 101 may sort the three rewards from the three first trials and the three rewards from the three second trials, totaling six rewards, in descending order. Then, the processing circuit 101 reflects in the control map a change to one of the trials in which two or more rewards are included in the top three. Such an embodiment may also be employed.

[0066] In the above embodiment, the adaptation of the control map of the first motor generator 31 when cranking the engine 33 is illustrated. The above-described adaptation method can also be applied to other motor controls. For example, it can be applied to the adaptation of functions used to control a drive motor for an electric vehicle or a motor that drives an electric actuator. [Explanation of symbols]

[0067] 10...vehicle, 20...control device, 21...overall control unit, 22...engine control unit, 23...motor control unit, 30...hybrid system, 31...first motor generator, 32...second motor generator, 33...engine, 34...power split mechanism, 35...power control unit, 36...reduction mechanism, 40...drive wheel, 50...microphone, 51...acceleration sensor, 100...adaptation system, 101...processing circuit, 102...storage device

Claims

1. a processing circuit and a storage device; the processing circuit is an adaptation system that optimizes the function stored in a control device that controls the motor by repeating a learning routine including: trials of driving the motor while acquiring state variables by a sensor in a state in which a function that outputs a command value to the motor has been modified; evaluation of calculating a reward based on the acquired state variables; and learning of updating the function based on the reward; the processing circuit executes a first process in which, until a predetermined condition for determining that optimization has progressed to the end is met, a first trial and a second trial are performed in each learning routine in which the change is made to the function so that the command value output from the function is adjusted in the opposite positive and negative directions, and the change in one of the first trial and the second trial in which the reward was greater is reflected in the function to update the function and terminate the learning routine; After the predetermined condition is established, the processing circuit executes a second process in which the first trial and the second trial are each performed a plurality of times in each learning routine, the rewards of the plurality of first trials are compared with the rewards of the plurality of second trials, and the change in one of the first trial and the second trial, whichever has the larger reward, is reflected in the function to update the function, and the learning routine is terminated. Compatible systems.

2. In the second process, the change in one of the first trial and the second trial, in which the average value of the reward for a plurality of trials is greater than the other, is reflected in the function to update the function, and the learning routine is terminated. The matching system of claim 1 .

3. the function is a control map for outputting the command value to the motor in accordance with the elapsed time from the start of control, the change in the first trial is the change that randomly adjusts the value of the command value for each elapsed time period in the control map within a predetermined adjustment range, The change in the second trial is a change in which the adjustment of the command value for each elapsed time period in the change in the first trial is reversed in positive and negative and applied to the value of the command value for each elapsed time period in the control map. The matching system of claim 1 .

4. the function is a function used for control when the motor drives a crankshaft of an engine mounted on a vehicle to crank the engine, the attempt is an attempt to start the engine by cranking the engine with the motor; the sensors include a crank position sensor that detects an engine rotation speed of the engine, a microphone that detects a sound emitted from the vehicle, and an acceleration sensor that detects vibrations of the vehicle; The state variables include an engine rotation speed detected by the crank position sensor, a sound pressure detected by the microphone, and an acceleration detected by the acceleration sensor. The matching system of claim 1 .

5. Using an adaptation system having a processing circuit and a storage device, an adaptation method for optimizing the function stored in a control device that controls the motor by repeatedly executing a learning routine in the processing circuit, the learning routine including: trials of driving the motor while acquiring state variables by a sensor in a state in which a function for outputting a command value to the motor has been modified; evaluation of calculating a reward based on the acquired state variables; and learning of updating the function based on the reward; a first step of causing the processing circuit to execute a first process of performing a first trial and a second trial in which the change is made to the function so that the command value output by the function is adjusted in the opposite direction of positive and negative in each learning routine until a predetermined condition for determining that the optimization has progressed to the end is met, and updating the function by reflecting the change in one of the first trial and the second trial in which the reward was larger, and terminating the learning routine; a second step of causing the processing circuit to execute a second process of performing the first trial and the second trial a plurality of times in each learning routine after the predetermined condition is satisfied, comparing the rewards of the plurality of first trials with the rewards of the plurality of second trials, and updating the function by reflecting the change in one of the first trial and the second trial, which had a larger reward, and terminating the learning routine. How it fits.

Images