Ultrasonic motor driver, trained model generation method, training program, and training system

Abstract

To provide a driving device for ultrasonic equipment capable of precisely operating ultrasonic equipment such as an ultrasonic motor at high speed.SOLUTION: An ultrasonic device driving device 100 includes a state measurement unit 30 that measures a plurality of operating states including the drive state of an ultrasonic motor 10, and a monitoring control device 60 that includes a learning unit 163 that undergoes deep reinforcement learning using the state measurement unit 30, and that generates a control signal by processing the operating state measured by the state measurement unit 30 in the learning unit 163, and the learning unit 163 learns to determine the output state so as to achieve a given target state.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention is Motor Ultrasonic waves for operating Motor The present invention relates to a driving device for ultrasonic vibration measurement, a method for generating a trained model, a training program, and a training system, and in particular to an ultrasonic vibration measurement device incorporating a control function using a deep neural network. Motor This relates to drive devices for vehicles. [Background technology]

[0002] Ultrasonic motors have many advantages, such as high torque per unit weight and holding power. One of the outstanding features of ultrasonic motors is their fast response, but when using conventional drive drivers currently in practical use, it is difficult to find the appropriate drive conditions (specifically, drive frequency). Conventional methods take time to achieve the drive conditions (input voltage conditions) necessary to achieve the desired drive state, and external disturbances such as temperature information are not input, resulting in insufficient optimal control.

[0003] The reason why the drive conditions of ultrasonic motors cannot be set appropriately is that many parameters, such as rotation speed, torque, temperature, etc. Also, the motor itself has nonlinear characteristics, for example, the relationship between drive frequency and speed has strong nonlinear characteristics including temperature, making it extremely difficult to model ultrasonic motors.

[0004] For driving ultrasonic motors, the use of reinforcement learning and genetic algorithms has been proposed (Patent Documents 1 and 2). In Patent Document 1, control parameters are adjusted by reinforcement learning, which optimizes an evaluation function using a history of AC voltage amplitude values, measured values ​​of rotation angles, and differential values ​​of the measured values ​​of rotation angles, thereby calculating a position error correction speed that minimizes tracking error, and then calculating a speed control voltage command value using a target angular velocity to which the position error correction speed has been added and an inverse motion model. In Patent Document 2, a PID control circuit is used as the control circuit for the ultrasonic motor, incorporating a variable virtual model internal model control system having a transfer function that reflects changes in the dynamic characteristics of the ultrasonic motor obtained using an identification method, or the output of a neural network determined by executing a genetic algorithm using a genetic algorithm circuit is added at the addition point of the output of a PID control circuit, which is not limited to the above.

[0005] The control method of the above-mentioned background art is insufficient in terms of the accuracy, stability, convergence characteristics, etc. of the control operation, and it is not easy to properly operate an ultrasonic motor which has high speed response but is related to many parameters and has strong nonlinear characteristics. [Prior art documents] [Non-patent literature]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-86922 [Patent Document 2] International Publication No. 2007 / 049412 Summary of the Invention

[0007] The present invention has been made in view of the above background art, and is directed to an ultrasonic Motor Ultrasonic waves that can operate at high speed and precision Motor The object of the present invention is to provide a drive device for a vehicle.

[0008] To solve the above-mentioned problems, an ultrasonic device driving device includes a state measurement unit that measures multiple operating states of the ultrasonic device, including the operating state of the ultrasonic device, and a monitoring control device that includes a learning unit that uses the state measurement unit to learn using a deep learning method and generates a control signal by processing the operating states measured by the state measurement unit in the learning unit. The learning unit learns to determine an output state to achieve a given target state. Here, with regard to learning, the target state is a target operating state of the ultrasonic device to be controlled, such as torque, rotational speed, efficiency, etc. Furthermore, the output state is a driving state given as an output to the ultrasonic device to be controlled, such as frequency, voltage amplitude, phase difference, etc. Here, deep learning methods are typified by, for example, deep reinforcement learning, but are not limited to, and include those using convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc.

[0009] In the ultrasonic equipment driving device described above, the monitoring and control device uses a state measurement unit to learn by deep learning, and includes a learning unit that learns to determine the output state so as to realize a given target state. Therefore, the use of deep learning provides high learning efficiency in terms of speed and stability, and enables more accurate and faster control of the operation of the ultrasonic equipment.

[0010] In a specific embodiment, the state measurement unit measures the operating speed of the ultrasonic device as the driving state. The operating speed is, for example, a rotation speed, and this makes it possible to control the rotation speed of the ultrasonic device.

[0011] In a specific aspect, the state measurement unit measures the temperature of the ultrasonic device and the frequency at which the ultrasonic device is driven. In this case, learning and operation control that reflect the temperature state of the ultrasonic device and the state of the drive signal supplied to the ultrasonic device become possible.

[0012] In a specific embodiment, the state measurement unit measures a value corresponding to the voltage supplied to the ultrasonic device as the driving state. In this case, a sensor that directly detects the operating speed of the driving state, such as an encoder, is not required, and the ultrasonic device can be made lighter and less expensive.

[0013] In a specific embodiment, the ultrasonic device further includes a control circuit that generates a command signal for operating the ultrasonic device based on the difference between a status signal indicating the operating state of the ultrasonic device and a drive command value, and the learning unit corrects the command signal by modifying the response of the control circuit. In this case, the configuration is similar to classical PID control, making it easy to incorporate into existing devices.

[0014] In a specific aspect, the ultrasonic device is an ultrasonic motor, and the state measurement unit measures at least one of the frequency and voltage amplitude output to the ultrasonic motor as the driving state.

[0015] In a specific aspect, the learning unit receives and stores the learned results as input, and calculates a control signal from the operating state of the ultrasonic device. In this case, the learned results can be obtained externally by downloading, etc., and the ultrasonic device driving device can have a relatively simple structure. Note that, as will be described in the embodiment section below, the monitoring and control device may include a learning unit that changes the control method while continuing to learn.

[0016] In a particular embodiment, the control signal provides an input voltage state of the ultrasound instrument and corresponds to an output state of the supervisory controller.

[0017] A method for generating a trained model to solve the above-mentioned problems includes the steps of inputting data indicating the input voltage state, operating state including the driving state, and target state of the ultrasonic device as input values, and learning the input values ​​using a deep learning method as learning data, thereby generating a computational model that calculates the input voltage state so that the driving state of the ultrasonic device becomes the target state when the driving state and target state are input.

[0018] In a specific aspect, the input voltage state is the phase, frequency, amplitude, or waveform of the input voltage to the electrodes provided in the ultrasonic device, or the difference between the phase, frequency, and amplitude of the input voltage, the driving state is the resulting operating speed of the ultrasonic device, the driving environment state excluding the driving state among the operating states is at least one of the temperature, load, load torque, and friction coefficient of the internal mechanism, and the target state is the target operating speed of the ultrasonic device. Note that the friction coefficient of the internal mechanism means the friction coefficient between the stator and the slider in the case of an ultrasonic motor.

[0019] In a particular embodiment, the deep learning method is a deep reinforcement learning method.

[0020] The learning program for solving the above-mentioned problems is a program for creating a program to be executed by a computer applicable to a monitoring and control device that drives an ultrasonic device, and executes the following steps: inputting data indicating the input voltage state, operating state including the driving state, and target state of the ultrasonic device as input values; and learning the input values ​​using a deep learning method as learning data, thereby generating a calculation model that calculates the input voltage state so that the driving state of the ultrasonic device becomes the target state when the driving state and target state are input.

[0021] The learning system for solving the above-mentioned problems is a learning system for creating a program to be executed by a computer applicable to a monitoring control device that drives an ultrasonic device, and includes an input unit to which data indicating the input voltage state, operating state including the driving state, and target state of the ultrasonic device is input as input values, and a calculation model generation unit that calculates the input voltage state so that the driving state of the ultrasonic device becomes the target state when the driving state and target state are input by learning the input values ​​using a deep learning method as learning data.

[0022] The trained program for solving the above-mentioned problems causes a computer applicable to a monitoring and control device that drives an ultrasonic device to execute at least the following steps: receiving data indicating an operating state and a target state, including the driving state of the ultrasonic device, as input values; and calculating an input voltage state that will cause the driving state of the ultrasonic device to become the target state, based on a computational model acquired by learning about the ultrasonic device using a deep learning method.

[0023] The trained system for solving the above-mentioned problems is a learning system applied to a monitoring control device that drives an ultrasonic device, and includes at least an input unit to which data indicating the operating state and target state including the driving state of the ultrasonic device is input as input values ​​based on a computational model acquired by learning the ultrasonic device using a deep learning method, and a calculation unit that calculates the input voltage state so that the driving state of the ultrasonic device becomes the target state. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a conceptual diagram illustrating a driving device for an ultrasonic device according to a first embodiment. [Figure 2] FIG. 2 is a block diagram illustrating a monitoring control device. [Figure 3] FIG. 10 is a diagram illustrating learning of the ultrasonic equipment driving device. [Figure 4] 10(a) and 10(b) are flowcharts showing an example of the operation of the ultrasonic device driving device. [Figure 5] FIG. 10 is a conceptual diagram illustrating a driving device for an ultrasonic device according to a second embodiment. [Figure 6] 10A and 10B are diagrams illustrating a modified example of the ultrasonic device driving device. [Figure 7] 10A and 10B are diagrams illustrating another modified example of the ultrasonic device driving device. [Figure 8] FIG. 10 is a diagram illustrating learning of the ultrasonic equipment driving device. [Figure 9] 1 is a chart showing the velocity response of an ultrasonic motor at various temperatures. [Figure 10](a) shows the velocity response when the drive frequency is swept down, and (b) shows the velocity response when the drive frequency is swept up. [Figure 11] 1 shows the temperature rise when different voltages are applied to an ultrasonic motor. [Figure 12] (a) shows the results of simulating the nonlinear velocity response of an ultrasonic motor, and (b) shows the results of simulating the temperature dependence of the velocity response. [Figure 13] Demonstrates the learning curve of the simulation. [Figure 14] (a) shows step-like speed tracking in a simulation, (b) shows frequency action according to command frequency during step-like speed tracking, (c) shows sinusoidal tracking, and (d) shows frequency action according to command frequency in the sinusoidal case. [Figure 15] The experimental learning curve is shown. [Figure 16] (a) shows the step-like speed tracking in the experiment, (b) shows the frequency action according to the command frequency during step-like speed tracking, (c) shows the sinusoidal tracking, and (d) shows the frequency action according to the command frequency in the sinusoidal case. [Figure 17] (a) and (b) show tracking to a constant target velocity when the ultrasonic motor is initialized in the pullout region. [Figure 18] 10(a) to 10(c) are diagrams for explaining experiments etc. related to the second embodiment. [Figure 19] 10(a) to 10(c) are diagrams for explaining experiments etc. related to the modified example. [Figure 20] 4 shows a feedback voltage for monitoring corresponding to the rotation state signal. DETAILED DESCRIPTION OF THE INVENTION

[0025] [First embodiment] Hereinafter, a first embodiment of an ultrasonic device driving device, a trained model generating method, a training program, a training system, and the like according to the present invention will be described with reference to the drawings.

[0026] FIG. 1 is a block diagram illustrating a motor device 200 including an ultrasonic device driving device 100 according to this embodiment. The ultrasonic device driving device 100 is a driving device that operates an ultrasonic motor 10 at a target rotational speed and includes a state measurement unit 30, a frequency driving device 50, and a monitoring and control device 60. Of these, the state measurement unit 30 detects the state of the ultrasonic motor 10. The frequency driving device 50 generates a driving signal A2, which is an AC signal that operates the ultrasonic motor 10, in response to a command signal A1, which is a control signal, from the monitoring and control device 60. The monitoring and control device 60 receives a driving command value Vref from an external device and outputs a command signal (control signal) A1 to the frequency driving device 50 based on the observation results of the state measurement unit 30, thereby operating the frequency driving device 50 appropriately.

[0027] The ultrasonic motor 10 to be driven is a well-known ultrasonic device. It consists of an annular slider 2 stacked on an annular stator 4, and these are housed in a case 9. A piezoelectric vibrator 6 that generates ultrasonic vibrations is bonded to the underside of the stator 4, and comb-like grooves 4a are formed on the upper surface. The slider 2 is rotatably supported by a bearing 8 via a rotating shaft RX. When a predetermined high-frequency voltage signal SG is applied to the electrode 3 of the piezoelectric vibrator 6 on the underside of the stator 4 to excite the piezoelectric vibrator 6, this vibration is transmitted to the stator 4, generating a traveling wave PW that travels in one direction on the stator 4, and a mass point on the stator 4 performs an elliptical motion RM. The slider 2 is biased against the stator 4 with a predetermined pressure, generating a frictional force between the vibrating stator 4 and the slider 2, which is in pressure contact with the stator 4. This frictional force causes the slider 2 to move in the opposite direction to the traveling wave PW of the stator 4. That is, the slider 2 supported by the bearing 8 rotates around the rotation axis RX, and the ultrasonic motor 10 is rotationally driven. An encoder device 5 is provided in association with the ultrasonic motor 10, making it possible to detect the rotation speed of the stator 4 or the rotation axis RX as a target state. A thermometer 7 is also provided in association with the ultrasonic motor 10, and detects the overall temperature of the ultrasonic motor 10.

[0028] In the ultrasonic device driver 100, the state measurement unit 30 is configured with a digital circuit including an AD conversion function and is realized, for example, by a DSP or a microcomputer. The state measurement unit 30 measures the drive signal A2 of the frequency driver 50 as a voltage value, measures, for example, frequency f from this voltage value, and outputs this frequency f to the monitoring control device 60 as one element of the operating state S. Note that frequency f is not limited to an observed value, but may also be a command value output from the monitoring control device 60 to the frequency driver 50. The state measurement unit 30 detects the rotation speed β, which is the target state of the ultrasonic motor 10, based on the output of the encoder 5, and outputs a rotation status signal V, which is a voltage value corresponding to the rotation speed β, to the monitoring control device 60 as one element of the operating state S. The rotation status signal V is the resulting operating speed and is paired with a drive command value Vref, which is the target operating speed. The state measurement unit 30 detects the temperature T of the ultrasonic motor 10 based on the output of the thermometer 7 and outputs this temperature T to the monitoring control device 60 as one element of the operating state S.

[0029] The frequency driver 50 is an oscillator that operates the ultrasonic motor 10, and can utilize an existing electronic circuit for the ultrasonic motor 10. The frequency driver 50 generates an AC drive signal A2 at a frequency f that corresponds to a command signal (control signal) A1 output from the monitoring and control device 60. The drive signal A2 corresponds to a high-frequency voltage signal SG that is applied to the electrode 3 of the piezoelectric vibrator 6. The frequency driver 50 receives, for example, a DC voltage signal as the command signal A1.

[0030] The monitoring and control device 60 is a computer, and outputs a command signal (control signal) A1 for driving the frequency driver 50 while monitoring the state of the ultrasonic motor 10.

[0031] As shown in FIG. 2, the monitoring control device 60 has a processor 61a, a memory 61b, and an interface 61c. These elements are connected to each other via a bus 61q so that they can communicate with each other. The processor 61a reads and executes programs stored in the memory 61b. The memory 61b stores an operating system, various programs, data, etc. that are loaded into the processor 61a. The interface 61c enables communication with the outside and the exchange of data. The monitoring control device 60 is equipped with application software that collects data from the state measurement unit 30 and drives the frequency driver 50, as well as application software that executes various processes such as a deep learning method (described later), more specifically, deep reinforcement learning.

[0032] Returning to FIG. 1, the monitoring and control device 60 operates according to a learning program installed on the computer shown in FIG. 2, enabling unsupervised deep reinforcement learning for driving the ultrasonic motor 10. It functions as an agent AG during deep reinforcement learning. To perform deep learning, the monitoring and control device 60 includes an input unit 62 and a neural network 63. The input unit 62 combines multiple input values, namely, frequency f, rotation state signal V, and temperature T, input from the state measurement unit 30 with an externally input drive command value Vref to output a state signal representing the operating state. The input unit 62 also outputs a difference Verr obtained by subtracting one input value, the drive command value Vref, from the rotation state signal V indicating the drive state of the ultrasonic motor 10. The neural network 63 is a learning unit 163 that performs deep reinforcement learning using the state measurement unit 30, and the learning unit 163 is trained by deep reinforcement learning. The neural network 63 is not limited to being incorporated alone as a training target, but two or more neural networks may be incorporated for learning purposes. The learning unit 163 also serves as a computational model CM generated by learning. The neural network 63 itself is a well-known deep neural network, including an input layer LI, one or more hidden layers LM, and an output layer LO. The input layer LI includes, for example, four neurons NR, the hidden layer LM includes many neurons NR, and the output layer LO includes one neuron NR. The neuron NR receives one or more inputs and outputs the weighted sum of the inputs through an activation function. The input layer LI is provided with a plurality of operating states S (f, T, V, Vref) corresponding to the outputs of the input unit 62, corresponding to the four neurons NR. The operating states S serve as training data for the agent AG. The output layer LO outputs an operation or action A corresponding to one neuron NR. The action A corresponds to a command signal A1 as a voltage value and drives the frequency driver 50 with a frequency change Δf. The action A corresponds to the input voltage state or control signal and is the output state to the electrode 3 provided in the ultrasonic motor 10. The action A is not limited to frequency but also includes information such as the phase, amplitude, and waveform of the input voltage.

[0033] When the monitoring and control device 60 has completed deep reinforcement learning, i.e., training, the neural network 63 becomes fixed. This allows the action A to be calculated quickly from the operating state S, which includes the drive command value Vref, frequency f, etc. The monitoring and control device 60 that has completed learning is a trained system.

[0034] The monitoring control device may be one that continues learning for the neural network 63 or the learning unit 163. In this case, since learning continues, action A changes continuously and gradually, and the control method changes from moment to moment. This makes it possible to deal with, for example, cases where the wear surface of the ultrasonic motor 10 changes. The monitoring control device 60 that continues learning can be said to be a trained system in that it reflects the results of learning up to that point.

[0035] The monitoring and control device 60 may be configured to operate offline a neural network 63 that has been acquired online from a higher-level system or the Internet or that has been trained online. In this case, the monitoring and control device 60 does not function as a deep reinforcement learning agent, so it is sufficient that the response speed of the neural network 63 satisfies the requirements for the responsiveness of the ultrasonic motor 10, and it can be configured using a relatively slow computer or the like.

[0036] FIG. 3 is a diagram illustrating a device for pre-training the monitoring and control device 60 of the ultrasonic equipment driving device 100 shown in FIG. 1 . In this case, a virtual ultrasonic motor 110 is incorporated instead of the ultrasonic motor 10. The virtual ultrasonic motor 110 is configured with a computer or microcomputer and is realized by a function of application software that executes a simulation to reproduce the state and response of the target ultrasonic motor 10. The virtual ultrasonic motor 110 is realized by an equivalent circuit model based on an LCR circuit. If the neural network 63 of the monitoring and control device 60 is pre-trained by deep reinforcement learning using the virtual ultrasonic motor 110, deep reinforcement learning can be performed quickly when the virtual ultrasonic motor 110 is replaced with the actual ultrasonic motor 10, thereby improving the reliability of deep reinforcement learning for controlling the ultrasonic motor 10. The virtual ultrasonic motor 110 can be realized by various simulation models, not just equivalent circuit models.

[0037] FIG. 4(a) is a diagram conceptually illustrating an example of the learning operation of the ultrasonic equipment driving device 100. The processor 61a of the monitoring and controlling device 60 receives input of the target state Vref and the driving environment state, such as temperature T, for the ultrasonic motor 10 (step S11). The target state and the driving environment state do not need to be external signals, but may be selected from initial values ​​previously stored in the memory 61b. The processor 61a provisionally determines an input voltage state, including, for example, frequency f, that achieves the target state Vref, and drives the ultrasonic motor 10 based on the determined input voltage state (step S12). At this time, the processor 61a outputs the input voltage state to the frequency driving device 50 via the interface 61c. As a response from the ultrasonic motor 10, the processor 61a obtains the operating state, such as the rotation state signal V corresponding to the driving state, from the state measurement unit 30 via the interface 61c, and calculates the difference Verr of the rotation state signal V (step S13). The processor 61a of the monitoring and control device 60 calculates a reward based on the difference Verr obtained in step S13, and modifies the learning unit 163 using deep reinforcement learning (step S14). At this time, a policy is selected taking entropy into consideration so as to maximize the cumulative value of the reward, taking into account the delay in the reward, and a neural network is used to realize such a policy. The above-described learning process is repeated until the episode is completed, and multiple episodes are repeated. In the learning process after the first time, the input voltage state that realizes the target state Vref is determined in step S12 based on the learning results from the previous time.

[0038] FIG. 4(b) is a diagram conceptually illustrating an example of the learned operation of the ultrasonic equipment driving device 100. The processor 61a receives input of a target state Vref and an input of a driving environment state, such as temperature T, for the ultrasonic motor 10 (step S21). The target state is an external signal. The processor 61a determines an input voltage state, including, for example, frequency f, that realizes the target state Vref based on a computational model acquired by performing deep reinforcement learning on the ultrasonic motor 10 in advance, as shown in FIG. 4(a), and drives the ultrasonic motor 10 based on the determined input voltage state (step S22). The above-described control is repeated as long as the target state Vref is provided.

[0039] The ultrasonic equipment driving device 100 of the first embodiment described above includes a state measurement unit 30 that measures multiple operating states including the driving state of the ultrasonic motor 10, and a monitoring control device 60 that includes a learning unit 163 that performs deep reinforcement learning using the state measurement unit 30 and generates a control signal by processing the operating state measured by the state measurement unit 30 in the learning unit 163, and the learning unit 163 learns to determine the output state so as to realize a given target state.

[0040] In the ultrasonic equipment driving device described above, the monitoring control device 60 includes a learning unit 163 that has learned to determine the output state so as to realize a given target state through deep reinforcement learning using the state measurement unit 30, and the control of the ultrasonic motor 10 can be formulated as a Markov decision process, so that the learning efficiency is high in terms of speed and stability, and more accurate and high-speed control of the operation of the ultrasonic equipment becomes possible.

[0041] Second Embodiment A second embodiment of the ultrasonic device driving device etc. according to the present invention will be described below. The driving device of the second embodiment is a partial modification of the driving device of the first embodiment, and the same parts are designated by the same reference numerals and will not be described again.

[0042] 5 is a block diagram illustrating a motor device 200 including an ultrasonic device driving device 100 according to this embodiment. The ultrasonic device driving device 100 includes a state measurement unit 30, a control circuit 40, a frequency driving device 50, and a monitoring and control device 60. The control circuit 40 generates a command signal A1 corresponding to the manipulated variable from a difference addition signal A0 obtained from a rotation state signal V indicating the driving state of the ultrasonic motor 10. The monitoring and control device 60 corrects the command signal A1 by modifying the response of the control circuit 40.

[0043] The state measurement unit 30 detects the rotation speed β of the ultrasonic motor 10 based on the output of the encoder device 5, and outputs a rotation state signal V, which is a voltage value corresponding to the rotation speed β, to the difference adder 81, and also outputs this rotation state signal V to the monitoring control device 60 as one element of the operating state S. The state measurement unit 30 outputs the temperature T and the frequency f to the monitoring control device 60 as one element of the operating state S, and outputs the drive command value Vref to the monitoring control device 60 as one element of the operating state S.

[0044] The control circuit 40 is composed of digital circuits including AD and DA conversion functions. The control circuit 40 corresponds to a PID controller and generates a command signal A1 to be input to the frequency drive device 50 based on a difference Verr, i.e., a difference sum signal A0, obtained by subtracting a drive command value Vref from a rotation status signal V indicating the drive status of the ultrasonic motor 10. In this case, the control circuit 40 operates under the control of the monitoring and control device 60, undergoes adjustments reflecting deep reinforcement learning, and outputs a command signal A1 corrected by the monitoring and control device 60. The control circuit 40 uses the difference Verr as its original manipulated variable, but the corrections by the monitoring and control device 60 result in highly accurate operating characteristics incorporating proportional control elements. The command signal A1 is an input voltage state or control signal corresponding to action A in the device of the first embodiment. In this case, the command signal A1 is a differential value, and includes information such as not only a frequency difference but also the phase difference, amplitude difference, and waveform difference of the input voltage.

[0045] The monitoring and control device 60 is a computer that controls the operating state of the control circuit 40 while monitoring the state of the ultrasonic motor 10, and outputs a corrected command signal A1. The monitoring and control device 60 outputs the operation or action At determined by the neural network 63 as a control signal. The action (control signal) At specifies a voltage correction value ΔV or Δf for the command signal A1, and is information equivalent to the frequency correction value Δf for the frequency driver 50.

[0046] The monitoring control device 60, like the monitoring control device 60 shown in Figure 1, enables deep reinforcement learning without a teacher, but unlike the monitoring control device 60 shown in Figure 1, it does not have anything equivalent to the input unit 62, in response to the addition of a differential adder 81 and a control circuit 40 to the ultrasonic equipment driving device 100.

[0047] Figure 6 shows the 5 1 is a diagram illustrating a modified example in which the ultrasonic device driving device 100 shown in FIG. 1 is partially modified. In this case, the control circuit 140 incorporates the functions of the frequency driving device 50. Therefore, the control circuit 140 outputs a frequency, a voltage amplitude, and a phase difference, and two AC voltages based on these states are input to the ultrasonic motor 10 as a driving signal A2.

[0048] 7 is a diagram illustrating another modified example of the ultrasonic device driving device 100 shown in FIG. 1. In this case, the correction device 160 incorporates a neural network 63 that has completed deep reinforcement learning. Because the correction device 160 does not operate as a deep reinforcement learning agent AG, it may be a relatively slow computer as long as the response speed of the neural network 63 satisfies the requirements for the responsiveness of the ultrasonic motor 10. The correction device 160 may be a device that operates offline using a neural network 63 that has been acquired online by download or that has been trained online as a client.

[0049] Figure 8 shows the 7The monitoring and control device 60 of the ultrasonic device driving device 100 shown in FIG. ,160 1 is a diagram illustrating a method for learning the above in advance. In this case, a virtual ultrasonic motor 110 is incorporated in place of the ultrasonic motor 10. The virtual ultrasonic motor 110 is configured by a computer or a microcomputer, and is realized by the function of application software that executes a simulation to reproduce the state and response of the target ultrasonic motor 10.

[0050] 5 to 8 does not need to be an independent unit as shown, but can be incorporated into the monitoring control device 60 as one function of the monitoring control device 60. Alternatively, a combination of the differential adder 81 and the state measurement unit 30 is called an observation device 130, and such observation device 130 can be realized by a single DSP or microcomputer (see FIG. 5).

[0051] [Research results on ultrasonic motor control] [1. Overview] Control of the rotational speed of the ultrasonic motor (USM) 10 (hereinafter referred to as speed control) can be formulated as a Markov decision process (MDP). Therefore, a deep reinforcement learning (DRL)-type controller (agent AG), specifically the supervisory control device 60 shown in Figure 1, can be applied to optimize the performance of the ultrasonic motor by executing optimal control actions in specific states. The agent AG is trained online by interacting with the ultrasonic motor without explicit model definition. The trained agent AG can then control the ultrasonic motor offline for faster response. A soft actor-critic (SAC) configuration was selected for its sample efficiency, fast convergence, and stable learning. To reduce training time, the SAC agent AG was first trained in simulation and then experimentally. The trained agent AG can expand the speed operating range, minimize response time, increase robustness around the resonance region, overcome speed hysteresis, and stabilize performance under temperature drift.

[0052] [2. Characteristic Evaluation of Ultrasonic Motor (USM)] (2a) Experimental equipment A Shinsei USR60 motor was used as the target for speed control, i.e., the ultrasonic motor 10. The ultrasonic motor was driven by two sinusoidal signals with a 90° phase difference, a voltage amplitude of approximately 300 Vpp, and a drive frequency of approximately 40 kHz. While commercially available drivers could be used as the frequency driver 50 for the ultrasonic motor 10, a customized experimental device was developed to enhance the controllability of the drive signal. An NF WF1968 function generator was used to generate two pure sinusoidal signals, and two HSA4052 amplifiers amplified the signals to the desired level. A UNIPULSE UTM III encoder was used as the encoder device 5 to measure the rotational speed of the ultrasonic motor, and the speed analog signal was received via a UNIPULSE TM380 monitor. The temperature inside the ultrasonic motor was estimated via a K-type thermocouple installed inside the motor, which served as the thermometer 7. The thermocouple was connected to an NF DM2561A digital multimeter for temperature measurement.

[0053] A Python environment was used for the monitoring and control device 60, i.e., the computer. Using Python's Pyserial library, the computer was connected via RS232 to the DSPACE1104 board, which constitutes the control frequency driver 50. Through this connection, the commanded drive frequency was transmitted from the computer to the DSPACE1104 board. The commanded frequency was applied as an analog signal from the DSP to a function generator. By varying the analog signal between ±1 V, the drive frequency was output between 39 and 45 kHz through frequency modulation. Similarly, for temperature measurements, Python's PyVisa library enabled communication between the computer and the digital multimeter via a general-purpose interface bus (GPIB) connection.

[0054] (2b) Experimental evaluation Using the experimental setup described above, we evaluated the nonlinear characteristics of the ultrasonic motor's velocity response. Figure 9 shows the velocity response at various temperatures. Under no-load conditions and a voltage amplitude of 250 Vpp, the drive frequency was swept between 45 and 39 kHz at various temperatures. As the frequency decreased, the rotational speed increased at a continuous rate and then suddenly stopped due to a pull-out phenomenon, primarily caused by nonlinear contact mechanics. Ultrasonic motor controllers typically set a lower limit on the drive frequency to avoid the pull-out region, which limits motor output. In the experimental setup, the velocity response remained nearly constant at low speeds with a relatively small slope, even with increasing temperature. However, at higher speeds, increasing temperature caused a drift in the velocity response for low frequencies, resulting in a slight increase in the peak velocity.

[0055] Another nonlinear feature of the velocity response is velocity hysteresis, where the response of an ultrasonic motor depends on the frequency sweep direction. Figure 10(a) shows the velocity response under increasing voltage amplitude as the drive frequency is swept down. As the amplitude increases, there is a slight shift in the response characteristic to higher frequencies, a pullout at non-zero rotational speeds, and a change in the knee (the point where the slope increases sharply). Furthermore, as shown in Figure 10(b), when the drive frequency is swept up, the voltage amplitude affects the velocity response. At higher voltage amplitudes, the ultrasonic motor resumes normal operation at a lower frequency, or pull-in frequency. For example, a voltage amplitude of 400 Vpp showed nearly identical responses in both sweep directions, resulting in reduced hysteresis. On the other hand, at a voltage amplitude of 250 Vpp, there was a large gap between the pull-out and pull-in frequencies. Commercial drivers use higher voltage amplitudes to maintain more consistent operation after pullout, but such high amplitudes do not contribute to a significant increase in output power and shorten the lifespan of the ultrasonic motor due to increased power loss and overheating, as shown in Figure 11. In order to solve more difficult control problems and maintain higher nonlinearity, we set the voltage amplitude to 250 Vpp throughout this training and evaluation of the reinforcement learning controller for the ultrasonic motor.

[0056] (2c) Simulation model To evaluate the effectiveness of the proposed reinforcement learning controller, we conducted simulations before experimental evaluation. Several approaches, varying in complexity and accuracy, are available for simulating the operation of ultrasonic motors, including the finite element method (FEM), the calculus of variations, and the equivalent circuit model (ECM). The finite element method is one of the more accurate approaches and can be applied during the design phase, but it tends to be relatively slow due to its computational complexity. On the other hand, the calculus of variations allows for accurate modeling of the system by formulating a set of differential equations that describe the system's behavior. In the calculus of variations, reaching a steady state starting from certain initial conditions is computationally expensive, and directly solving the steady-state solution can result in multiple solutions that lead to model inaccuracies and instabilities. The final common approach is the equivalent circuit model. Equivalent circuit models can be less accurate because they cannot electrically model contact mechanics, but they enable fast online calculation of the motor state. Due to their simplicity and the fact that an accurate model is not required for controller design, using equivalent circuit models for ultrasonic motor simulations speeds up controller prototyping. The piezoelectric ring is represented by a damping capacitance (Cd) in parallel with an LCR circuit representing the stator. The stator is powered by a transformer with a coupling coefficient Θ that represents the electromechanical transformation in the piezoelectric ring. Equation (1) represents the equivalent circuit model of the stator, where L m , R m , and C m represent the mechanical inductance (mass), resistance (damping), and capacitance (inverse stiffness), respectively. TIFF0007783616000001.tif68168 ECM parameters were estimated using an Agilent 4294A impedance analyzer as shown in Table 1. [Table 1] For a given voltage amplitude V0 and angular frequency ω (=2πf), the vibration amplitude ξ0 can be analytically solved using equation (1).

[0057] For simplicity, we neglect contact mechanics and assume that the rotor speed is equal to the maximum tangential vibration velocity of the stator, as in equation (2): where h is the stator height and r is the stator radius. The resulting rotational speed is the mechanical resonant frequency The pull-out phenomenon is introduced by setting the driving frequency f as shown in equation (3). out (f out =f r ), the rotation speed is set to zero. Furthermore, the speed hysteresis is in Lower (f in =1.05f r ), if the last rotation speed is also zero as in equation (3), adjust the current rotation speed to zero.

[0058] Modeling the temperature dependence required shifting the stator ECM parameters according to the operating temperature using two empirical equations. Equation (4) equivalently models the softening stiffness of the stator as the temperature increases (increasing capacitance). The minimum temperature T of the current T min Depending on the deviation from the capacitance C m is the initial capacitance C m0 From coefficient α c Similarly, equation (5) is an equivalent model of the change in damping, with a slight decrease in damping and a slight increase in peak velocity as temperature increases. The empirical parameter α c and α r were assumed to be 0.0015 1 / °C and 0.003 1 / °C, respectively.

[0059] Using the ECM configured as described above, we were able to simulate the nonlinear velocity response of an ultrasonic motor, as shown in Figure 12(a). We also simulated the temperature dependence, as shown in Figure 12(b). The results of the simulation model closely resembled the experimental behavior, and even without an exact match, this facilitates the transition from simulation to experiment during controller design or prototyping.

[0060] [3. Deep Reinforcement Learning] (3a) Overview Reinforcement learning is a type of machine learning that generally involves developing optimal control strategies by configuring a controller agent to interact with the environment (plant) and improve its policy (control law) through trial and error. (In this invention, this means that by placing the supervisory control device 60 as the agent AG in an environment of the effects and future responses on an ultrasonic device such as an ultrasonic motor, the learning unit 163 of the supervisory control device 60 is trained to determine the output state, or action A, for the ultrasonic device so as to achieve a target state for the ultrasonic device.) Deep reinforcement learning (DRL) can provide more optimal decision-making through model-free offline calculations, and therefore can execute optimal actions over an infinite planning horizon, making it superior to model predictive control (MPC). Deep reinforcement learning has been proposed for speed control of electromagnetic motors, and simulation results have shown superior performance compared to PID controllers. Deep reinforcement learning has also been applied to robotic hand manipulation, etc. Reinforcement learning, or RL, problems involve determining the state (s t The best action to take (a t ) and define the immediate reward (r t ) as well as the expected reward (R t ) is maximized. A common representation of an RL problem is a Markov Decision Process (MDP). A Markov Decision Process determines the future state (s t+1 ) and reward (r t ) depends only on the current state and action, and not on any other previous states or actions. Following a Markov decision process, we obtain the following sequence (s t → a t → r t (s t ,a t ) → s t+1 → a t+1 → r t+1 (s t+1 ,a t+1 ) → …). Under the Markov assumption, as shown in equation (6), t ) and action (at We define Q as the sum of expected future rewards (either finite or discounted infinite) that are generated by performing the following: Q can be estimated as the sum of the immediate reward and the discounted expected future reward. TIFF0007783616000004.tif51166 Using a discount factor γ [0 to 1], we can perform infinite horizon planning, guaranteeing that the expected reward is finite. A discount factor of 1 considers all future rewards equally, while a discount factor of 0 considers only immediate rewards. t ) is selected based on the current policy π(a t |s t ), the goal of learning is to find a policy π that maximizes this Q-function by choosing the optimal action as in equation (7). For discrete states and actions, a tabular representation of Q or π works, but for continuous states and actions, which reduces to a deep reinforcement learning algorithm, a deep neural network is chosen.

[0061] There are various model-free algorithms for finding optimal policies, each with different advantages and disadvantages. One of the first algorithms to emerge from the interest of deep reinforcement learning was deep Q-learning for games, which uses a single Q-network to output the expected Q-values ​​for all discrete actions in a given state, and the optimal action is greedily selected from them. Q-based algorithms are sample-efficient because they use all previous data and rely on the policy against it, but they are less stable because they may converge to an incorrect Q-function. Another approach that provides more stable learning is to learn the policy directly using policy gradient methods, such as proximal policy gradient. These types of algorithms tend to require more time to converge and be sample-inefficient because they learn using only the most recent data based on the policy, but are more stable because they are optimized for the true objective, the policy. To achieve the best of both worlds, actor-critic methods were developed to be more stable while also being sample-efficient. Deep decision policy gradient (DDPG) is an actor-critic algorithm that enables learning complex motor skills in continuous environments. Deep decision policy gradient methods are sample-efficient because all interactions with the environment (experiences) are stored in an experience buffer for later learning. Deep decision policy gradient methods use two main networks: one to learn the Q-function (critic) and the other to learn the policy (actor). A variation of deep decision policy gradient methods is twin-delay DDPG (TD3). TD3 uses two Q-networks to provide a more conservative Q-estimation and more stable learning. A further improvement is made by using stochastic actions rather than the decision actions of deep decision policy gradient methods proposed in the soft actor-critic (SAC) algorithm. Soft actor-critic uses a stochastic actor that outputs the mean and standard deviation of the action given the state, and the action is selected by sampling a Gaussian distribution.The soft actor-critic uses a modified objective to find the optimal policy by introducing entropy (H) as in equation (8). The entropy term determines how confident one is in their current action. This facilitates exploration to find actions that maximize reward and ensure stable behavior. Unlike deep decision policy gradient methods, soft actor-critic algorithms are more robust to parameter initialization and have better convergence properties. To implement deep reinforcement learning for ultrasonic motor speed control, the soft actor-critic algorithm was selected for its stable convergence and excellent exploratory capabilities. This algorithm is also suitable for the stochastic behavior of ultrasonic motors, where speed noise is common.

[0062] (3b) Deep reinforcement learning for ultrasonic motor speed control Given the significant success of deep reinforcement learning in recent years, applying it to overcome current challenges in ultrasonic motor control seems promising. To begin with, we test it for speed control. Previous studies have applied neural networks to speed control, but they suffer from instability and slow convergence, and are prone to myopia due to their goal of minimizing instantaneous speed error. On the other hand, soft actor-critic algorithms are highly sample-efficient thanks to experience buffers and Q-networks, and select optimal actions in long-term planning. Furthermore, their stochastic nature makes them more robust against signal noise.

[0063] As mentioned previously, the speed response of ultrasonic motors suffers from high nonlinearity, and their behavior depends on operating conditions including preload, load torque, voltage amplitude, and temperature. Here, we simplify the problem by focusing on no-load operation with a constant voltage amplitude. A voltage amplitude of 250 Vpp is applied, resulting in significant hysteresis. To realize a soft actor-critic agent (controller) AG for ultrasonic motor speed control, we need to define the input state, control action, and reward function. The state vector is chosen to satisfy the Markov property, and the system is fully observable in terms of rotational speed. Therefore, the state vector includes the drive frequency (f) as shown in Equation (9). t ), motor temperature (T t ), motor rotation speed (V rt ), and target speed (V targett : target rotation speed corresponding to drive command value Vref) is desirably included. As explained in equation (3), the current rotation speed depends on the driving frequency and the previous rotation speed due to the speed hysteresis. As a result, to estimate the future rotation speed (reward) in the Q network, the driving frequency f t and motor rotation speed V rt Temperature changes can cause changes in speed response, and the motor temperature T t If V is not included in the state definition, it may lead to suboptimal behavior. To learn the optimal policy for various target speeds, we first need to consider the target speed V targett It is desirable to add a control action (a t) is the frequency update applied to the current frequency as in equation (10). The most challenging part of reinforcement learning is the proper definition of the reward function, which influences the learning process and optimality of the agent. Specific guidelines for designing reward functions are unknown, and no universal function has been found that can be applied to all problems. However, mean squared error (MSE) or mean absolute error (MAE) are typical choices for problems. For position control problems, some literature suggests that absolute error performs best compared to quadratic or square root error. However, for our speed control problem, square root error has been found to provide faster response and smaller steady-state error. Equation (11) is our proposed reward function. The reward is calculated by multiplying the current rotational speed (V rt ) and target speed (V targett ) is equal to the negative of the square root of the absolute velocity error between the target and the target. Additionally, the reward is weighted (w a ) Absolute action is included. The simplified Markov decision process of the proposed speed control is as shown in Figure 1, and the simulations and experiments described here are based on the circuit of the first embodiment.

[0064] [4. Results] The SAC agent described above was implemented in both simulations and experiments using the design parameters listed in Table 2. Parameter tuning is a common challenge in machine learning. Therefore, these parameters may need to be further adjusted depending on the agent's objectives. Relu activation and Adam optimizer are common neural network options. The network size must be optimized to minimize computational effort without sacrificing accuracy. The discount factor γ and entropy weight αH can influence the optimal action selection depending on the stochasticity of the environment and the long-term dependencies between states. The update rate ε controls the rate at which the target network is updated. A small ε results in slower but more stable learning; conversely, a larger ε results in faster but potentially more unstable learning. Similarly, the learning rates Lra and Lra control the size of the step gradient in the network update and can affect learning speed and stability. A larger batch size results in more stable step gradients but is computationally more expensive. While the current parameter choices appear to produce appropriate results, further parameter tuning is a topic for future work. [Table 2]

[0065] To speed up the convergence of the learning process, the input states need to be normalized. Gaussian normalization was applied to the states before feeding them into the network. The ranges of the input state variables were: drive frequency 39-45 kHz, temperature 20-60°C, and rotation speed 0-300 rpm. To further stabilize the learning process, the frequency action was limited to ±2 kHz.

[0066] (4a) Simulation The proposed Soft Actor-Critic (SAC) agent was evaluated using the simulation model described in the Ultrasonic Motor Characterization section. The agent was trained for 1000 episodes, each with 30 steps. At the beginning of each episode, the target velocity V target and motor temperature T t was uniformly sampled from each range and held constant throughout the remainder of the episode. Figure 13 shows the learning curves of the simulation. Here, the solid line shows the reward from the actor neural network, the dotted line shows the reward from the critic neural network, and the dashed line shows the average of the two. Because the network was initialized with random weights, the rewards were low in the first few episodes. As the agent learned an appropriate policy, the reward increased to a nearly stable value around episode 200. Training costs were low, and further training was performed for further optimization. In Figure 13, the average reward (over a window of five episodes) is plotted to demonstrate a more stable learning curve. The expected reward is the reward expected by the Q-network given the initial state. Initially, there was a significant mismatch between the actual and expected rewards, but they converged to nearly the same value in subsequent episodes. This confirms that the Q-network can accurately predict the system's behavior.

[0067] After 1000 training episodes, the agent was evaluated using both stepped and sinusoidally varying target speeds. During evaluation, the network parameters were fixed and no online optimization was performed. As a result, the evaluation process was much faster than training. The horizontal axis is labeled "steps" because the SAC algorithm is time-independent. Depending on hardware power, application requirements, and system responsiveness, steps can vary from a fraction of a second to several seconds. Figure 14(a) shows the stepped target speed, varying from 0 to 300 rpm in 50-rpm steps. The rotational speed was stepped every 50 steps. Near-perfect tracking was achieved at all target speeds, resulting in an error of approximately 1 rpm. Because there is always a one-sample-time delay between the target speed and the actual rotational speed, the speed error jumps to 50 rpm when the target speed is updated, but the response is nearly instantaneous, taking approximately one step to reach the target speed. Figure 14(b) shows the speed control command frequency and the frequency action output of the SAC agent. The frequency action was always zero except when a new target speed was commanded. The frequency action reached up to 2 kHz as needed, significantly reducing response time. At low speeds and high drive frequencies, the frequency action was higher than at high speeds and low drive frequencies. This directly reflects the nonlinearity of the speed response, which cannot be compensated for with a conventional linear controller. Further evaluation was performed by commanding a sinusoidal change, as shown in Figure 14(c). The rotational speed was varied between 0 and 300 rpm for 10 cycles, with each cycle lasting 100 steps. Again, a complete trace was achieved, and the resulting speed error was due to the time delay of the samples. Figure 14(d) shows the change in frequency action as a function of the commanded speed. The frequency action changed rapidly at low speeds due to the nonlinear speed response.

[0068] (4b) Experiment Following the agent's successful performance in simulation, the SAC agent was experimentally implemented. It is noteworthy that, using the current experimental setup, one agent step takes approximately 10 milliseconds. The main limiting factor is communication delays via the RS232 and GPIB connections. The agent was not randomly initialized as in simulation; instead, a network trained in simulation was used as the initial weights for the network. This initialization allows for faster learning and more stable initial behavior, and is safer for the hardware than a fully randomized approach. The agent was trained for 500 episodes, achieving acceptable performance after only 100 episodes, as shown in Figure 15. After training was complete, the agent was experimentally evaluated, as shown in Figure 16. Unlike simulation, the speed signal in the experiment was noisy. As a result, even with perfect tracking, the rotational speed error was approximately 4 rpm due to speed ripple.

[0069] The SAC agent not only executes optimal actions that maximize immediate rewards, but can also execute actions that do not directly contribute to immediate rewards but increase the expected reward. Figure 17 shows tracking of a constant target speed when the ultrasonic motor is initialized in the pullout region. A linear controller such as a PID controller will simply diverge when operating in the pullout region. To escape the pullout region, the drive frequency must be increased above the pull-in frequency so that the ultrasonic motor can resume operation and command the optimal frequency. Such trajectory planning is difficult to achieve even using fuzzy rules or supervised learning techniques. By sacrificing model identification and online optimization, long-term planning can be achieved using a model predictive controller. The proposed SAC agent can track the desired target speed, as shown in Figure 17(a). The ultrasonic motor was initialized with a drive frequency of 39 kHz, a value at which the ultrasonic motor cannot operate. The agent commanded frequency actions to escape the pullout region by increasing the frequency into the pull-in region. Then, the frequency was decreased to achieve the target speed, as shown in Figure 17(b).

[0070] Figure 18 shows the 5 18(a) and 18(c) show the results of an evaluation experiment conducted on the ultrasonic device driving device 100 of the second embodiment shown in FIG. 18(a). FIG. 18(a) shows the training curve of the simulation. FIG. 18(b) and 18(c) explain the evaluation of the SAC agent for speed control. FIG. 18(b) shows step-like speed tracking in the experiment, and FIG. 18(c) shows sinusoidal speed tracking. It can be seen that the results obtained when the circuit configuration of the second embodiment is used are similar to those when the circuit configuration of the first embodiment is used.

[0071] In the above, the frequency f, temperature T, rotation state signal V, and drive command value Vref are set as the operating state S monitored by the monitoring control device 60, but elements different from those described above can be added to the operating state S. Furthermore, the operating environment state excluding the driving state (frequency f, rotation state signal V) among the operating state S is not limited to the temperature T, and may also be the load or the load torque.

[0072] The frequency f and temperature T may be omitted from the operating state S monitored by the monitoring control device 60, and information such as the frequency f and the temperature T may be estimated from other input signals such as the rotation state signal V, the drive command value Vref, and the voltage supplied to the ultrasonic motor 10. In other words, the ultrasonic equipment driving device 100 may be sensorless or may use fewer sensors.

[0073] FIG. 19 shows the results of an evaluation experiment conducted on the sensorless ultrasonic device driver 100. FIG. 19(a) shows a simulation training curve, and FIGS. 19(b) and 19(c) show two stepped speed tracking patterns used in the experiment. FIG. 20 shows the monitoring feedback voltage corresponding to the rotational state signal V. In this evaluation, the ultrasonic device driver 100 controlled the rotational speed of the ultrasonic motor 10 by omitting angle information from the encoder and instead inputting a feedback voltage, which is vibration amplitude information detected from the ultrasonic motor 10. The feedback voltage corresponds to the vibration signal of the ultrasonic motor 10 (a blank voltage generated in a small electrode provided on the piezoelectric element separately from the drive voltage, and output when the piezoelectric element vibrates). The feedback voltage corresponds to the voltage supplied to the ultrasonic motor 10.

[0074] Recently, attention has been focused on master-slave remote medical surgery using 5G high-speed communications, and for such remote medical surgery, there is a demand for high-output motors with fast response. The ultrasonic device driver 100 described above will broaden the application of ultrasonic motors to haptic devices and robots, and is expected to be incorporated into devices used in remote medical surgery, etc.

[0075] The present invention is not limited to the above examples, and includes various modifications within the scope of the concept. The controlled object by the ultrasonic device driving device 100 is not limited to a rotary ultrasonic motor, but can also include a linear ultrasonic motor, a powerful ultrasonic vibrator, etc. In the case of a linear ultrasonic motor, the rotational speed expressed above is a linear speed or other sliding speed.

[0076] In the above, the rotational speed of the ultrasonic motor is controlled as the target state, but the target state may also be torque, efficiency, etc. In this case, the monitoring and control device 60 performs learning in the learning unit 163 so as to optimize the output state, such as frequency, voltage amplitude, phase, etc., in order to achieve the target torque, efficiency, etc.

[0077] The monitoring and control device 60 is not limited to a computer in the narrow sense, and the learned learning unit 163 in particular can be configured with a microcomputer, etc. In other words, a microcomputer is also a computer in the broad sense. [Explanation of symbols]

[0078] 2...slider, 3...electrode, 4...stator, 5...encoder device, 6...piezoelectric vibrator, 7...thermometer, 10...ultrasonic motor, 30...status measurement unit, 40, 140...control circuit, 50...frequency drive device, 60...monitoring control device, 61a...processor, 61b...memory, 61c...interface, 62...input unit, 63...neural network, 81...difference adder, 100...drive device for ultrasonic equipment, 110...virtual ultrasonic motor, 130...observation device, 160...correction device, 163...learning unit, 200...motor device, A0...difference addition signal, A1...command signal, A2...drive signal, AG...agent, A...action, CM...computational model, RX...rotating axis, S...operating state, V...rotation state signal

Claims

1. A state measurement unit that measures a plurality of operating states including a driving state of an ultrasonic motor; a monitoring control device including a learning unit that learns by a deep learning method using the state measurement unit, and that generates a control signal for the ultrasonic motor by processing the operating state measured by the state measurement unit in the learning unit; Equipped with the state measurement unit measures, as the operating state, a resultant operating speed of the ultrasonic motor corresponding to the driving state, an internal temperature of the ultrasonic motor as an environmental state, and a frequency at which the ultrasonic motor is driven; The ultrasonic motor driving device, wherein the learning unit learns to determine the control signal that realizes a given target speed of the ultrasonic motor.

2. A driving device for an ultrasonic motor as described in claim 1, wherein the control signal provides an input voltage state of the ultrasonic motor, and the input voltage state includes the phase, frequency, amplitude, and waveform of the input voltage to electrodes provided on the ultrasonic motor, or the difference in phase, frequency, and amplitude of the input voltage.

3. a control circuit that generates a command signal for operating the ultrasonic motor based on a difference obtained by subtracting a drive command value from a status signal that indicates the operating speed of the ultrasonic motor; 3. The ultrasonic motor driving device according to claim 1, wherein the monitoring control device corrects the command signal by modifying the response of the control circuit using the learning section that has learned.

4. 4. The ultrasonic motor driving device according to claim 1, wherein the learning unit receives and stores the learned results as input, and calculates the control signal from the operating state of the ultrasonic motor.

5. A driving device for an ultrasonic motor as described in any one of claims 1 to 4, wherein when the operating state of the ultrasonic motor enters the pull-out region, the monitoring control device drives the ultrasonic motor to the target speed by increasing the frequency at which the ultrasonic motor is driven above the pull-in frequency to cause the ultrasonic motor to escape from the pull-out region.

6. A state measurement unit that measures a plurality of operating states including the resulting operating speed of the ultrasonic motor; a monitoring control device including a learning unit that learns by a deep learning method using the state measurement unit, and that generates a control signal that realizes a target speed of the ultrasonic motor based on the operating state measured by the state measurement unit; Equipped with When the operating state of the ultrasonic motor enters a pull-out region, the monitoring and control device drives the ultrasonic motor so that the operating speed of the ultrasonic motor becomes the target speed by increasing the frequency at which the ultrasonic motor is driven above a pull-in frequency to cause the ultrasonic motor to escape from the pull-out region.

7. A step of inputting data indicating an operating state including a driving state of an ultrasonic motor and a target state of the ultrasonic motor as input values; a step of generating a calculation model that calculates a control signal for causing the driving state of the ultrasonic motor to become the target state when the operating state and the target state are input by learning the input value using a deep learning method as learning data; Including, The operating conditions include a resultant operating speed of the ultrasonic motor corresponding to the driving conditions, an internal temperature of the ultrasonic motor which is an environmental condition, and a frequency at which the ultrasonic motor is driven; The target state is a target speed. How to generate a trained model.

8. A program for creating a program to be executed by a computer applicable to a monitoring control device that drives an ultrasonic motor, a step of inputting data indicating an operating state including a driving state of the ultrasonic motor and a target state of the ultrasonic motor as input values; a step of generating a calculation model that calculates a control signal for causing the driving state of the ultrasonic motor to become the target state when the operating state and the target state are input by learning the input value using a deep learning method as learning data; Execute The operating conditions include a resultant operating speed of the ultrasonic motor corresponding to the driving conditions, an internal temperature of the ultrasonic motor which is an environmental condition, and a frequency at which the ultrasonic motor is driven; The target state is a target speed. Learning program.

9. A learning system for creating a program to be executed by a computer that is applicable to a monitoring control device that drives an ultrasonic motor, comprising: an input unit to which data indicating an operating state including a driving state of the ultrasonic motor and a target state of the ultrasonic motor is input as input values; a calculation model generation unit that calculates a control signal for causing the driving state of the ultrasonic motor to become the target state when the operating state and the target state are input by learning the input value using a deep learning method as learning data; Equipped with The operating conditions include a resultant operating speed of the ultrasonic motor corresponding to the driving conditions, an internal temperature of the ultrasonic motor which is an environmental condition, and a frequency at which the ultrasonic motor is driven; The target state is a target speed. Learning system.

10. A computer applicable to a monitoring control device that drives an ultrasonic motor, comprising at least: receiving data indicating an operating state including a driving state of the ultrasonic motor and a target state as input values; A procedure for calculating a control signal for causing the driving state of the ultrasonic motor to become the target state based on a calculation model acquired by performing learning on the ultrasonic motor using a deep learning method; Execute The operating conditions include a resultant operating speed of the ultrasonic motor corresponding to the driving conditions, an internal temperature of the ultrasonic motor which is an environmental condition, and a frequency at which the ultrasonic motor is driven; The target state is a target speed. Learned programs.

11. A learning system applied to a monitoring control device that drives an ultrasonic motor, comprising at least: An input unit to which data indicating an operating state and a target state including a driving state of the ultrasonic motor is input as input values ​​based on a calculation model acquired by learning the ultrasonic motor using a deep learning method; a calculation unit that calculates a control signal for causing the driving state of the ultrasonic motor to become the target state; Equipped with The operating conditions include a resultant operating speed of the ultrasonic motor corresponding to the driving conditions, an internal temperature of the ultrasonic motor which is an environmental condition, and a frequency at which the ultrasonic motor is driven; The target state is a target speed. Trained system.

Images