Method and device for calibrating output frequency of radio frequency power supply

Abstract

The invention provides a radio frequency power supply output frequency calibration method and device. The method comprises the following steps: determining a current frequency output state and environmental parameters of a radio frequency power supply; inputting the current frequency output state and the environmental parameters into a frequency regulation model to obtain a regulation coefficient of a frequency register in a digital frequency synthesizer of the radio frequency power supply output by the frequency regulation model, regulating the frequency register based on the regulation coefficient of the frequency register, and regulating the frequency register based on the current actual output frequency of the radio frequency power supply. And updating the current calibration output frequency, repeatedly calling the above steps based on a comparison result of the current calibration output frequency and the target frequency to realize iterative calibration of the output frequency of the radio frequency power supply, and performing output frequency adjustment by using a frequency adjustment model in a training stage based on a calibration strategy learned by a specially designed behavior reward value. Accurate, efficient and stable output frequency calibration for the radio frequency power supply is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of radio frequency power supply, and in particular to a method and device for calibrating the output frequency of a radio frequency power supply. Background Art

[0002] RF power supply is a device used to provide high-frequency power signals. It can generate high-frequency AC power signals to drive various RF devices and equipment, such as RF transmitters, antennas, plasma processing devices for electronic devices, etc. In semiconductor processes, RF power supplies are often used in process steps such as plasma etching, chemical vapor deposition, and physical vapor deposition. In semiconductor processes, for some specific process steps, such as plasma etching, the accuracy of the output frequency of the RF power supply directly affects the accuracy and stability of the process. If the output frequency of the RF power supply is inaccurate, it will cause changes in plasma parameters, which will in turn affect process parameters such as etching depth and etching rate, and reduce the accuracy and consistency of device processing. In addition, inaccurate output frequency of the RF power supply may cause overload or other failures of the equipment, affecting the stability and life of the equipment. Therefore, frequency calibration can protect the equipment from unnecessary damage and loss.

[0003] However, as the use time increases, the electronic components inside the RF power supply may age, resulting in drift or instability in the output frequency. In addition, factors such as changes in ambient temperature and humidity, clock source deviation, and device manufacturing deviation may affect the characteristics of the internal components of the RF power supply, thereby affecting the output frequency. Since it is unknown which specific factors cause the output frequency of the RF power supply to drift, and the digital frequency synthesizer in the RF power supply itself will cause frequency adjustment errors due to various reasons (such as insufficient internal accuracy, nonlinear effects, and environmental factors, etc.), the relationship between the above factors and the output frequency error is nonlinear and difficult to quantify. Therefore, it is not possible to directly determine the adjustment method of the digital frequency synthesizer based on the difference between the actual output frequency and the expected frequency to achieve calibration. Therefore, a more accurate and efficient way to calibrate the output frequency of the RF power supply is needed. Summary of the invention

[0004] The present invention provides a method and device for calibrating the output frequency of a radio frequency power supply, which are used to solve the defects of low efficiency and poor effect of the output frequency calibration of the radio frequency power supply in the prior art.

[0005] The present invention provides a method for calibrating the output frequency of a radio frequency power supply, comprising:

[0006] State determination step: determining the current frequency output state and environmental parameters of the RF power supply; the current frequency output state of the RF power supply includes the real-time difference between the current calibration output frequency of the RF power supply and the target frequency and the historical difference between the historical calibration output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located;

[0007] Register adjustment step: inputting the current frequency output state and environmental parameters into a frequency adjustment model, obtaining an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjusting the frequency register based on the adjustment coefficient of the frequency register;

[0008] Iterative calibration step: based on the current actual output frequency of the RF power supply, updating the current calibration output frequency; if the current calibration output frequency is inconsistent with the target frequency, repeating the state determination step and the register adjustment step.

[0009] According to a method for calibrating the output frequency of a radio frequency power supply provided by the present invention, the current frequency output state and environmental parameters are input into a frequency adjustment model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the radio frequency power supply output by the frequency adjustment model, specifically comprising:

[0010] Obtaining a random value, and comparing the random value with a preset threshold;

[0011] If the random value is less than the preset threshold, randomly selecting an adjustment coefficient of a frequency register in a digital frequency synthesizer of the radio frequency power supply;

[0012] If the random value is greater than a preset threshold, the current frequency output state and environmental parameters are input into a frequency regulation model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency regulation model.

[0013] According to a method for calibrating the output frequency of a radio frequency power supply provided by the present invention, the frequency adjustment model is trained based on the following method:

[0014] Sample state determination step: determining the current frequency output state and environmental parameters of the sample RF power supply; the current frequency output state of the sample RF power supply includes the real-time difference between the current calibration output frequency of the sample RF power supply and the sample target frequency and the historical difference between the historical calibration output frequency of the sample RF power supply and the sample target frequency during the historical calibration process;

[0015] Sample register adjustment step: inputting the current frequency output state and environmental parameters of the sample RF power supply into the initial adjustment model, obtaining the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and adjusting the corresponding frequency register based on the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply;

[0016] Model parameter updating step: based on the current actual output frequency of the sample RF power supply, updating the current calibrated output frequency of the sample RF power supply, calculating the current behavior reward value based on the current calibrated output frequency of the sample RF power supply, and updating the parameters of the initial adjustment model based on the current behavior reward value;

[0017] Iterative training step: repeatedly executing the sample state determination step, the sample register adjustment step and the model parameter update step until a preset training cutoff condition is reached; wherein the initial adjustment model after training is completed is the frequency adjustment model.

[0018] According to a method for calibrating the output frequency of a radio frequency power supply provided by the present invention, the current frequency output state and environmental parameters of the sample radio frequency power supply are input into an initial adjustment model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the sample radio frequency power supply output by the initial adjustment model, specifically comprising:

[0019] Input the current frequency output state and environmental parameters of the sample RF power supply into the initial adjustment model, and obtain the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model; the initial adjustment model is constructed based on a deep Q network; and each adjustment option includes adjustment parameters with different adjustment precisions and different adjustment directions;

[0020] Based on the Q values ​​of the adjustment options corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply, the adjustment option corresponding to the maximum Q value is determined as the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply.

[0021] According to a method for calibrating the output frequency of a radio frequency power supply provided by the present invention, the current behavior reward value is calculated based on the current calibrated output frequency of the sample radio frequency power supply, specifically comprising:

[0022] Determine a frequency deviation reward value based on a real-time difference between a current calibrated output frequency of the sample RF power source and the sample target frequency; the greater the difference between the current calibrated output frequency of the sample RF power source and the sample target frequency, the smaller the frequency deviation reward value;

[0023] Determine a calibration efficiency bonus value based on a historical difference between a historical calibration output frequency of the sample RF power source and the sample target frequency during a previous calibration process, and a real-time difference between a current calibration output frequency of the sample RF power source and the sample target frequency;

[0024] determining a stability bonus value based on a difference between a historical calibration output frequency of the sample RF power source in a previous calibration process and a current calibration output frequency of the sample RF power source;

[0025] The current behavior reward value is determined based on the frequency deviation reward value, the calibration efficiency reward value, and the stability reward value.

[0026] According to a method for calibrating the output frequency of a radio frequency power supply provided by the present invention, the updating of the parameters of the initial adjustment model based on the current behavior reward value specifically includes:

[0027] Determining a next frequency output state of the sample RF power supply;

[0028] Inputting the next frequency output state of the sample RF power supply and the environmental parameters into the initial adjustment model, obtaining the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and determining the sum of the maximum Q value and the current behavior reward value as the target Q value;

[0029] Based on the difference between the target Q value and the Q value corresponding to the current adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply, the model loss is determined, and the parameters of the initial adjustment model are updated based on the model loss.

[0030] According to a method for calibrating the output frequency of a radio frequency power supply provided by the present invention, determining the current behavior reward value based on the frequency deviation reward value, the calibration efficiency reward value, and the stability reward value specifically includes:

[0031] Determining the current behavior reward value based on the frequency deviation reward value and its weight, the calibration efficiency reward value and its weight, and the stability reward value and its weight;

[0032] The weight of the frequency deviation reward value is greater than the weight of the stability reward value and the weight of the calibration efficiency reward value.

[0033] The present invention also provides a radio frequency power supply output frequency calibration device, comprising:

[0034] A state determination unit, used to determine the current frequency output state and environmental parameters of the RF power supply; the current frequency output state of the RF power supply includes the real-time difference between the current calibration output frequency of the RF power supply and the target frequency and the historical difference between the historical calibration output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located;

[0035] A register adjustment unit, used to input the current frequency output state and environmental parameters into a frequency adjustment model, obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the radio frequency power supply output by the frequency adjustment model, and adjust the frequency register based on the adjustment coefficient of the frequency register;

[0036] The iterative calibration unit is used to update the current calibrated output frequency based on the current actual output frequency of the RF power supply; if the current calibrated output frequency is inconsistent with the target frequency, the state determination unit and the register adjustment unit are repeatedly called.

[0037] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned methods for calibrating the output frequency of a radio frequency power supply is implemented.

[0038] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for calibrating the output frequency of a radio frequency power supply as described in any one of the above is implemented.

[0039] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for calibrating the output frequency of a radio frequency power supply.

[0040] The present invention provides a method and device for calibrating the output frequency of a radio frequency power supply. The method and device determine the current frequency output state and environmental parameters of the radio frequency power supply, wherein the current frequency output state of the radio frequency power supply includes the real-time difference between the current calibrated output frequency of the radio frequency power supply and the target frequency and the historical difference between the historical calibrated output frequency of the radio frequency power supply and the target frequency during the historical calibration process, and the environmental parameters include the temperature and humidity of the external environment in which the radio frequency power supply is located. The current frequency output state and environmental parameters are then input into a frequency adjustment model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the radio frequency power supply output by the frequency adjustment model, and the frequency register is adjusted based on the adjustment coefficient of the frequency register, and the current calibrated output frequency is updated based on the current actual output frequency of the radio frequency power supply. The above steps are repeatedly called based on the comparison result between the current calibrated output frequency and the target frequency to implement iterative calibration of the output frequency of the radio frequency power supply, and the output frequency is adjusted using a calibration strategy learned by the frequency adjustment model based on a specially designed behavior reward value during the training phase, thereby implementing accurate, efficient and stable output frequency calibration for the radio frequency power supply. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a flow chart of a method for calibrating the output frequency of a radio frequency power supply provided by the present invention;

[0043] Figure 2 It is a flowchart of the current behavior reward value calculation method provided by the present invention;

[0044] Figure 3 It is a flow chart of the parameter updating method of the initial adjustment model provided by the present invention;

[0045] Figure 4 It is a structural schematic diagram of a radio frequency power supply output frequency calibration device provided by the present invention;

[0046] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Figure 1 FIG. 1 is a flow chart of a method for calibrating the output frequency of a radio frequency power supply provided by the present invention, such as Figure 1 As shown, the method includes:

[0049] State determination step 110: determining the current frequency output state and environmental parameters of the RF power supply; the current frequency output state of the RF power supply includes the real-time difference between the current calibration output frequency of the RF power supply and the target frequency and the historical difference between the historical calibration output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located;

[0050] Register adjustment step 120: inputting the current frequency output state and environmental parameters into a frequency adjustment model, obtaining an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjusting the frequency register based on the adjustment coefficient of the frequency register;

[0051] Iterative calibration step 130: based on the current actual output frequency of the RF power supply, updating the current calibration output frequency; if the current calibration output frequency is inconsistent with the target frequency, repeating the state determination step and the register adjustment step.

[0052] Here, the actual output frequency of the RF power supply can be obtained and compared with the target frequency. If the actual output frequency of the RF power supply is inconsistent with the target frequency, the output frequency of the RF power supply needs to be calibrated. The output frequency calibration of the RF power supply is an iterative process, which adjusts the output frequency of the RF power supply iteratively so that its actual output frequency is equal to the target frequency.

[0053] In a round of calibration, considering that the previous round of calibration will change the output frequency of the RF power supply by adjusting the adjustment coefficient of the frequency register in the digital frequency synthesizer of the RF power supply, thereby changing the state of the RF power supply, the current frequency output state of the RF power supply will be determined first. In addition, the output frequency of the RF power supply will also be affected by the environment, so the current environmental parameters can also be obtained. Among them, the current frequency output state of the RF power supply includes the real-time difference between the current calibration output frequency of the RF power supply (that is, the actual output frequency of the RF power supply after the previous round of calibration, initially the initial output frequency of the RF power supply) and the target frequency, and the historical difference between the historical calibration output frequency of the RF power supply in the historical calibration process (that is, the actual output frequency of the RF power supply after the corresponding calibration process, initially empty) and the target frequency; the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located.

[0054] Subsequently, the current frequency output state and environmental parameters can be input into the frequency adjustment model to obtain the adjustment coefficient of the frequency register in the digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and the frequency register can be adjusted based on the adjustment coefficient of the frequency register. In some embodiments, a random value can be first obtained based on a random algorithm, and the random value can be compared with a preset threshold value; if the random value is less than the preset threshold value, the adjustment coefficient of the frequency register in the digital frequency synthesizer of the RF power supply is randomly selected, for example, one of the preset multiple adjustment options corresponding to the frequency register can be randomly selected as the adjustment coefficient of the frequency register; if the random value is greater than the preset threshold value, the current frequency output state and environmental parameters can be input into the frequency adjustment model to obtain the adjustment coefficient of the frequency register in the digital frequency synthesizer of the RF power supply output by the frequency adjustment model. In other embodiments, after the current frequency output state and environmental parameters of the RF power supply are input into the frequency adjustment model, the frequency adjustment model will determine the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the RF power supply, and determine the adjustment option corresponding to the maximum Q value as the adjustment coefficient of the frequency register in the digital frequency synthesizer of the RF power supply. Among them, each adjustment option includes adjustment parameters with different adjustment precisions and different adjustment directions. For example, the adjustment option may include adjustment parameters vf1, vf2, ..., vfn, where vfi (1≤i≤n) can be a positive value, a negative value or 0, and vfi≠vfj, where 1≤j≤n and i≠j.

[0055] After the adjustment is completed, the current round of calibration process ends. At this time, the actual output frequency of the RF power supply can be determined, and the current calibration output frequency can be updated based on the actual output frequency. If the current calibration output frequency is inconsistent with the target frequency, the next round of calibration process is continued, and the above state determination step and register adjustment step are repeated.

[0056] Here, the frequency adjustment model can be constructed based on a model suitable for reinforcement learning, such as a deep Q network (DQN). The input layer nodes of the frequency adjustment model correspond to the real-time difference between the current calibrated output frequency of the RF power supply and the target frequency in the current frequency output state of the RF power supply, the historical difference between the historical calibrated output frequency of the RF power supply and the target frequency in the preset number of historical calibration processes, and the temperature and humidity of the external environment. When the number of historical calibration processes is insufficient, 0 can be filled based on the difference between the number of historical calibration processes and the above preset number. The output layer nodes of the frequency adjustment model can output the Q value of each adjustment option corresponding to the frequency register. The trained frequency adjustment model can calculate the Q value of each adjustment option according to the current frequency output state and environmental parameters based on the learned calibration strategy, where the larger the Q value of any adjustment option, the higher the reward for selecting the adjustment option for frequency calibration, and the more suitable it is for the current state of the RF power supply. It can be seen that the calibration strategy learned by the frequency adjustment model through reinforcement learning during the training process is the core of calibrating the output frequency of the RF power supply quickly, accurately and stably.

[0057] In some embodiments, the frequency adjustment model is trained based on:

[0058] Sample state determination step: determining the current frequency output state and environmental parameters of the sample RF power supply; the current frequency output state of the sample RF power supply includes the real-time difference between the current calibration output frequency of the sample RF power supply and the sample target frequency and the historical difference between the historical calibration output frequency of the sample RF power supply and the sample target frequency during the historical calibration process;

[0059] Sample register adjustment step: inputting the current frequency output state and environmental parameters of the sample RF power supply into the initial adjustment model, obtaining the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and adjusting the corresponding frequency register based on the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply;

[0060] Model parameter updating step: based on the current actual output frequency of the sample RF power supply, updating the current calibrated output frequency of the sample RF power supply, calculating the current behavior reward value based on the current calibrated output frequency of the sample RF power supply, and updating the parameters of the initial adjustment model based on the current behavior reward value;

[0061] Iterative training step: repeatedly executing the sample state determination step, the sample register adjustment step and the model parameter update step until a preset training cutoff condition is reached; wherein the initial adjustment model after training is completed is the frequency adjustment model.

[0062] Specifically, the operating mechanism of the frequency regulation model in the training stage and the actual application stage is similar. The training stage includes multiple rounds of iterative training processes. In one round of training, the current frequency output state and environmental parameters of the sample RF power supply are first determined. Among them, the current frequency output state of the sample RF power supply is similar to the current frequency output state of the above-mentioned RF power supply, including the real-time difference between the current calibrated output frequency of the sample RF power supply (i.e., the actual output frequency of the sample RF power supply after the calibration process in the previous round of training, and the initial output frequency of the sample RF power supply in the initial case) and the sample target frequency, and the historical difference between the historical calibrated output frequency of the sample RF power supply and the sample target frequency in the historical calibration process (i.e., the calibration process in the historical training process), and the environmental parameters include the temperature and humidity of the external environment in which the sample RF power supply is located. Among them, by introducing the real-time difference between the current calibrated output frequency and the sample target frequency and the historical difference between the historical calibrated output frequency of the sample RF power supply and the sample target frequency in the historical calibration process, the model can be guided to comprehensively consider the deviation of the output frequency of the current RF power supply and the calibration effect in the historical calibration process, so as to select a better adjustment method. Subsequently, the current frequency output state and environmental parameters of the sample RF power supply are input into an initial adjustment model (e.g., a DQN model) to obtain the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and the corresponding frequency register is adjusted based on the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply. Here, the current frequency output state and environmental parameters of the sample RF power supply can be input into the initial adjustment model, and the initial adjustment model calculates the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply according to the input, and based on the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply, determines the adjustment option corresponding to the maximum Q value as the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply.

[0063] Different from the actual application stage, in this round of training, after adjusting the corresponding frequency register based on the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply, the reward value will be calculated and the model parameters of the initial adjustment model will be adjusted according to the reward value, and the setting of the reward value is directly related to whether the initial adjustment model can learn the optimal frequency calibration strategy. Among them, the current calibrated output frequency of the sample RF power supply can be updated based on the current actual output frequency of the sample RF power supply, and the current behavior reward value can be calculated based on the current calibrated output frequency of the sample RF power supply, so as to update the parameters of the initial adjustment model based on the current behavior reward value.

[0064] In some embodiments, Figure 2 As shown, the current behavior reward value can be calculated based on the current calibrated output frequency of the sample RF power source in the following manner:

[0065] Step 210, determining a frequency deviation reward value based on a real-time difference between a current calibrated output frequency of the sample RF power source and the sample target frequency; the greater the difference between the current calibrated output frequency of the sample RF power source and the sample target frequency, the smaller the frequency deviation reward value;

[0066] Step 220, determining a calibration efficiency bonus value based on a historical difference between a historical calibration output frequency of the sample RF power source and the sample target frequency in a previous calibration process, and a real-time difference between a current calibration output frequency of the sample RF power source and the sample target frequency;

[0067] Step 230, determining a stability bonus value based on a difference between a historical calibration output frequency of the sample RF power source in a previous calibration process and a current calibration output frequency of the sample RF power source;

[0068] Step 240: determining the current behavior reward value based on the frequency deviation reward value, the calibration efficiency reward value, and the stability reward value.

[0069] Specifically, based on the real-time difference between the current calibrated output frequency of the sample RF power source and the sample target frequency, the frequency deviation reward value can be determined. The greater the difference between the current calibrated output frequency of the sample RF power source and the sample target frequency, the smaller the frequency deviation reward value. For example, the frequency deviation reward value can be calculated using the formula 1 / |fi-ft|, where fi is the current calibrated output frequency and ft is the sample target frequency. It can be seen that the frequency deviation reward value reflects the quality of the adjustment options selected by the current model according to the learned calibration strategy in terms of frequency calibration accuracy.

[0070] Based on the historical difference between the historical calibration output frequency of the sample RF power supply in the last calibration process (i.e., the calibration process in the last round of training process) (i.e., the actual output frequency of the sample RF power supply after adjusting the frequency register in the digital frequency synthesizer of the sample RF power supply based on the adjustment coefficient output by the initial adjustment model in the last round of training process) and the sample target frequency, and the real-time difference between the current calibration output frequency of the sample RF power supply and the sample target frequency, the calibration efficiency reward value is determined. For example, the historical difference between the historical calibration output frequency of the sample RF power supply in the last calibration process and the sample target frequency can be subtracted from the real-time difference between the current calibration output frequency of the sample RF power supply and the sample target frequency, and the difference can be used as the calibration efficiency reward value. It can be seen that the calibration efficiency reward value reflects the advantages and disadvantages of the adjustment options selected by the current model according to the learned calibration strategy in the dimension of frequency calibration efficiency.

[0071] Based on the difference between the historical calibration output frequency of the sample RF power supply in the last calibration process (i.e., the calibration process in the last round of training process) (i.e., the actual output frequency of the sample RF power supply after adjusting the frequency register in the digital frequency synthesizer of the sample RF power supply based on the adjustment coefficient output by the initial adjustment model in the last round of training process) and the current calibration output frequency of the sample RF power supply, the stability reward value is determined. Among them, the greater the difference between the historical calibration output frequency of the sample RF power supply in the last calibration process and the current calibration output frequency of the sample RF power supply, the smaller the stability reward value. It can be seen that the stability reward value reflects the advantages and disadvantages of the adjustment options selected by the current model according to the learned calibration strategy in the dimension of frequency calibration stability.

[0072] Based on the above-mentioned frequency deviation reward value, calibration efficiency reward value and stability reward value, the current behavior reward value can be determined. The current behavior reward value designed in this way can reflect the frequency calibration accuracy, frequency calibration efficiency and frequency calibration stability corresponding to the adjustment option selected by the current model. By adjusting the parameters of the initial adjustment model based on the current behavior reward value, the model can be guided to learn a better calibration strategy and select a more appropriate adjustment option to accurately, quickly and stably adjust the output frequency of the sample RF power supply to the sample target frequency. In some embodiments, a weighted summation process can be performed based on the frequency deviation reward value and its weight, the calibration efficiency reward value and its weight, and the stability reward value and its weight to obtain the current behavior reward value. Among them, the weight of the frequency deviation reward value is greater than the weight of the stability reward value and the weight of the calibration efficiency reward value.

[0073] After the current behavior reward value is calculated, the parameters of the initial adjustment model can be updated based on the current behavior reward value. Figure 3 As shown, the parameters of the initial adjustment model can be updated as follows:

[0074] Step 310, determining the next frequency output state of the sample RF power supply;

[0075] Step 320, inputting the next frequency output state of the sample RF power source and the environmental parameters into the initial adjustment model, obtaining the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power source output by the initial adjustment model, and determining the sum of the maximum Q value and the current behavior reward value as the target Q value;

[0076] Step 330: determine the model loss based on the difference between the target Q value and the Q value corresponding to the current adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply, and update the parameters of the initial adjustment model based on the model loss.

[0077] Specifically, the next frequency output state of the sample RF power supply in the next round of training can be determined based on the current calibrated output frequency of the sample RF power supply, that is, the real-time difference between the current calibrated output frequency of the sample RF power supply and the sample target frequency, and the historical difference between the historical calibrated output frequency of the sample RF power supply and the sample target frequency in the historical calibration process (that is, the calibration process in the historical training process including the current round of training process). The next frequency output state of the sample RF power supply and the environmental parameters are input into the initial adjustment model to obtain the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and the sum of the maximum Q value and the current behavior reward value is determined as the target Q value. Based on the difference between the target Q value and the Q value corresponding to the current adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply output by the model in the current round of training, the model loss is determined, and back propagation is performed based on the model loss to update the parameters of the initial adjustment model.

[0078] The sample state determination step, the sample register adjustment step and the model parameter update step are repeatedly performed until the preset training cutoff condition is reached. The initial adjustment model after the training is completed is a frequency adjustment model.

[0079] In summary, the method provided by the embodiment of the present invention determines the current frequency output state and environmental parameters of the RF power supply, wherein the current frequency output state of the RF power supply includes the real-time difference between the current calibrated output frequency of the RF power supply and the target frequency and the historical difference between the historical calibrated output frequency of the RF power supply and the target frequency during the historical calibration process, and the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located; then the current frequency output state and the environmental parameters are input into the frequency adjustment model to obtain the adjustment coefficient of the frequency register in the digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjust the frequency register based on the adjustment coefficient of the frequency register, and update the current calibrated output frequency based on the current actual output frequency of the RF power supply, and repeatedly call the above steps based on the comparison result of the current calibrated output frequency and the target frequency to implement iterative calibration of the output frequency of the RF power supply, and use the calibration strategy learned by the frequency adjustment model based on a specially designed behavioral reward value during the training phase to adjust the output frequency, thereby realizing accurate, efficient and stable output frequency calibration for the RF power supply.

[0080] A radio frequency power supply output frequency calibration device provided by the present invention is described below. The radio frequency power supply output frequency calibration device described below and the radio frequency power supply output frequency calibration method described above can correspond to each other.

[0081] Based on any of the above embodiments, Figure 4 Schematic diagram of a radio frequency power supply output frequency calibration device provided by the present invention. Figure 4 As shown, the device comprises:

[0082] The state determination unit 410 is used to determine the current frequency output state and environmental parameters of the RF power supply; the current frequency output state of the RF power supply includes the real-time difference between the current calibration output frequency of the RF power supply and the target frequency and the historical difference between the historical calibration output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located;

[0083] A register adjustment unit 420, configured to input the current frequency output state and environmental parameters into a frequency adjustment model, obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjust the frequency register based on the adjustment coefficient of the frequency register;

[0084] The iterative calibration unit 430 is used to update the current calibrated output frequency based on the current actual output frequency of the RF power supply; if the current calibrated output frequency is inconsistent with the target frequency, the state determination unit and the register adjustment unit are repeatedly called.

[0085] The device provided by the embodiment of the present invention determines the current frequency output state and environmental parameters of the RF power supply, wherein the current frequency output state of the RF power supply includes the real-time difference between the current calibrated output frequency of the RF power supply and the target frequency and the historical difference between the historical calibrated output frequency of the RF power supply and the target frequency during the historical calibration process, and the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located; then the current frequency output state and the environmental parameters are input into the frequency adjustment model to obtain the adjustment coefficient of the frequency register in the digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjust the frequency register based on the adjustment coefficient of the frequency register, and update the current calibrated output frequency based on the current actual output frequency of the RF power supply, and repeatedly call the above steps based on the comparison result of the current calibrated output frequency and the target frequency to implement iterative calibration of the output frequency of the RF power supply, and use the calibration strategy learned by the frequency adjustment model based on the specially designed behavior reward value in the training phase to adjust the output frequency, thereby realizing accurate, efficient and stable output frequency calibration for the RF power supply.

[0086] Based on any of the above embodiments, the inputting of the current frequency output state and the environmental parameters into the frequency adjustment model to obtain the adjustment coefficient of the frequency register in the digital frequency synthesizer of the RF power supply output by the frequency adjustment model specifically includes:

[0087] Obtaining a random value, and comparing the random value with a preset threshold;

[0088] If the random value is less than the preset threshold, randomly selecting an adjustment coefficient of a frequency register in a digital frequency synthesizer of the radio frequency power supply;

[0089] If the random value is greater than a preset threshold, the current frequency output state and environmental parameters are input into a frequency regulation model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency regulation model.

[0090] Based on any of the above embodiments, the frequency adjustment model is trained based on the following method:

[0091] Sample state determination step: determining the current frequency output state and environmental parameters of the sample RF power supply; the current frequency output state of the sample RF power supply includes the real-time difference between the current calibration output frequency of the sample RF power supply and the sample target frequency and the historical difference between the historical calibration output frequency of the sample RF power supply and the sample target frequency during the historical calibration process;

[0092] Sample register adjustment step: inputting the current frequency output state and environmental parameters of the sample RF power supply into the initial adjustment model, obtaining the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and adjusting the corresponding frequency register based on the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply;

[0093] Model parameter updating step: based on the current actual output frequency of the sample RF power supply, updating the current calibrated output frequency of the sample RF power supply, calculating the current behavior reward value based on the current calibrated output frequency of the sample RF power supply, and updating the parameters of the initial adjustment model based on the current behavior reward value;

[0094] Iterative training step: repeatedly executing the sample state determination step, the sample register adjustment step and the model parameter update step until a preset training cutoff condition is reached; wherein the initial adjustment model after training is completed is the frequency adjustment model.

[0095] Based on any of the above embodiments, the inputting the current frequency output state and environmental parameters of the sample RF power supply into the initial adjustment model to obtain the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model specifically includes:

[0096] Input the current frequency output state and environmental parameters of the sample RF power supply into the initial adjustment model, and obtain the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model; the initial adjustment model is constructed based on a deep Q network; and each adjustment option includes adjustment parameters with different adjustment precisions and different adjustment directions;

[0097] Based on the Q values ​​of the adjustment options corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply, the adjustment option corresponding to the maximum Q value is determined as the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply.

[0098] Based on any of the above embodiments, the calculating the current behavior reward value based on the current calibrated output frequency of the sample RF power source specifically includes:

[0099] Determine a frequency deviation reward value based on a real-time difference between a current calibrated output frequency of the sample RF power source and the sample target frequency; the greater the difference between the current calibrated output frequency of the sample RF power source and the sample target frequency, the smaller the frequency deviation reward value;

[0100] Determine a calibration efficiency bonus value based on a historical difference between a historical calibration output frequency of the sample RF power source and the sample target frequency during a previous calibration process, and a real-time difference between a current calibration output frequency of the sample RF power source and the sample target frequency;

[0101] determining a stability bonus value based on a difference between a historical calibration output frequency of the sample RF power source in a previous calibration process and a current calibration output frequency of the sample RF power source;

[0102] The current behavior reward value is determined based on the frequency deviation reward value, the calibration efficiency reward value, and the stability reward value.

[0103] Based on any of the above embodiments, the updating of the parameters of the initial adjustment model based on the current behavior reward value specifically includes:

[0104] Determining a next frequency output state of the sample RF power supply;

[0105] Inputting the next frequency output state of the sample RF power supply and the environmental parameters into the initial adjustment model, obtaining the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and determining the sum of the maximum Q value and the current behavior reward value as the target Q value;

[0106] Based on the difference between the target Q value and the Q value corresponding to the current adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply, the model loss is determined, and the parameters of the initial adjustment model are updated based on the model loss.

[0107] Based on any of the foregoing embodiments, determining the current behavior reward value based on the frequency deviation reward value, the calibration efficiency reward value, and the stability reward value specifically includes:

[0108] Determining the current behavior reward value based on the frequency deviation reward value and its weight, the calibration efficiency reward value and its weight, and the stability reward value and its weight;

[0109] The weight of the frequency deviation reward value is greater than the weight of the stability reward value and the weight of the calibration efficiency reward value.

[0110] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor (processor) 510, a memory (memory) 520, a communication interface (Communications Interface) 530 and a communication bus 540, wherein the processor 510, the memory 520, and the communication interface 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 520 to execute a method for calibrating the output frequency of an RF power supply, which method includes: a state determination step: determining the current frequency output state and environmental parameters of the RF power supply; the current frequency output state of the RF power supply includes the real-time difference between the current calibrated output frequency of the RF power supply and the target frequency and the historical difference between the historical calibrated output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located; a register adjustment step: inputting the current frequency output state and environmental parameters into a frequency adjustment model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjusting the frequency register based on the adjustment coefficient of the frequency register; an iterative calibration step: updating the current calibrated output frequency based on the current actual output frequency of the RF power supply; if the current calibrated output frequency is inconsistent with the target frequency, repeating the state determination step and the register adjustment step.

[0111] In addition, the logic instructions in the above-mentioned memory 520 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0112] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a method for calibrating the output frequency of an RF power supply provided by the above methods, the method comprising: a state determination step: determining the current frequency output state and environmental parameters of the RF power supply; the current frequency output state of the RF power supply includes the real-time difference between the current calibrated output frequency of the RF power supply and the target frequency and the historical difference between the historical calibrated output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located; a register adjustment step: inputting the current frequency output state and environmental parameters into a frequency adjustment model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjusting the frequency register based on the adjustment coefficient of the frequency register; an iterative calibration step: updating the current calibrated output frequency based on the current actual output frequency of the RF power supply; if the current calibrated output frequency is inconsistent with the target frequency, repeating the state determination step and the register adjustment step.

[0113] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the above-mentioned method for calibrating the output frequency of an RF power supply, the method comprising: a state determination step: determining the current frequency output state and environmental parameters of the RF power supply; the current frequency output state of the RF power supply includes the real-time difference between the current calibrated output frequency of the RF power supply and the target frequency and the historical difference between the historical calibrated output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment in which the RF power supply is located; a register adjustment step: inputting the current frequency output state and environmental parameters into a frequency adjustment model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjusting the frequency register based on the adjustment coefficient of the frequency register; an iterative calibration step: updating the current calibrated output frequency based on the current actual output frequency of the RF power supply; if the current calibrated output frequency is inconsistent with the target frequency, repeating the state determination step and the register adjustment step.

[0114] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calibrating the output frequency of a radio frequency power supply, characterized in that: include: State determination step: determining the current frequency output state and environmental parameters of the RF power supply; The current frequency output state of the RF power supply includes the real-time difference between the current calibrated output frequency of the RF power supply and the target frequency and the historical difference between the historical calibrated output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment where the RF power supply is located; Register adjustment step: inputting the current frequency output state and environmental parameters into a frequency adjustment model, obtaining an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency adjustment model, and adjusting the frequency register based on the adjustment coefficient of the frequency register; Iterative calibration step: updating the current calibration output frequency based on the current actual output frequency of the RF power supply; If the current calibration output frequency is inconsistent with the target frequency, the state determination step and the register adjustment step are repeatedly performed.

2. A method for calibrating the output frequency of a radio frequency power supply according to claim 1, characterized in that: The step of inputting the current frequency output state and the environmental parameters into the frequency adjustment model to obtain the adjustment coefficient of the frequency register in the digital frequency synthesizer of the radio frequency power supply output by the frequency adjustment model specifically includes: Obtaining a random value, and comparing the random value with a preset threshold; If the random value is less than the preset threshold, randomly selecting an adjustment coefficient of a frequency register in a digital frequency synthesizer of the radio frequency power supply; If the random value is greater than a preset threshold, the current frequency output state and environmental parameters are input into a frequency regulation model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the RF power supply output by the frequency regulation model.

3. A method for calibrating the output frequency of a radio frequency power supply according to claim 1, characterized in that: The frequency regulation model is trained based on the following method: Sample state determination step: determining the current frequency output state and environmental parameters of the sample RF power supply; the current frequency output state of the sample RF power supply includes the real-time difference between the current calibration output frequency of the sample RF power supply and the sample target frequency and the historical difference between the historical calibration output frequency of the sample RF power supply and the sample target frequency during the historical calibration process; Sample register adjustment step: inputting the current frequency output state and environmental parameters of the sample RF power supply into the initial adjustment model, obtaining the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and adjusting the corresponding frequency register based on the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply; Model parameter updating step: based on the current actual output frequency of the sample RF power supply, updating the current calibrated output frequency of the sample RF power supply, calculating the current behavior reward value based on the current calibrated output frequency of the sample RF power supply, and updating the parameters of the initial adjustment model based on the current behavior reward value; Iterative training step: repeatedly executing the sample state determination step, the sample register adjustment step and the model parameter update step until a preset training cutoff condition is reached; wherein the initial adjustment model after training is completed is the frequency adjustment model.

4. A method for calibrating the output frequency of a radio frequency power supply according to claim 3, characterized in that: The step of inputting the current frequency output state and environmental parameters of the sample RF power supply into an initial adjustment model to obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the sample RF power supply output by the initial adjustment model specifically includes: Input the current frequency output state and environmental parameters of the sample RF power supply into the initial adjustment model, and obtain the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model; the initial adjustment model is constructed based on a deep Q network; and each adjustment option includes adjustment parameters with different adjustment precisions and different adjustment directions; Based on the Q values ​​of the adjustment options corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply, the adjustment option corresponding to the maximum Q value is determined as the adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply.

5. A method for calibrating the output frequency of a radio frequency power supply according to claim 4, characterized in that: The calculating of the current behavior reward value based on the current calibration output frequency of the sample radio frequency power source specifically includes: Determine a frequency deviation reward value based on a real-time difference between a current calibrated output frequency of the sample RF power source and the sample target frequency; the greater the difference between the current calibrated output frequency of the sample RF power source and the sample target frequency, the smaller the frequency deviation reward value; Determine a calibration efficiency bonus value based on a historical difference between a historical calibration output frequency of the sample RF power source and the sample target frequency during a previous calibration process, and a real-time difference between a current calibration output frequency of the sample RF power source and the sample target frequency; determining a stability bonus value based on a difference between a historical calibration output frequency of the sample RF power source in a previous calibration process and a current calibration output frequency of the sample RF power source; The current behavior reward value is determined based on the frequency deviation reward value, the calibration efficiency reward value, and the stability reward value.

6. A method for calibrating the output frequency of a radio frequency power supply according to claim 5, characterized in that: The updating of the parameters of the initial adjustment model based on the current behavior reward value specifically includes: Determining a next frequency output state of the sample RF power supply; Inputting the next frequency output state of the sample RF power supply and the environmental parameters into the initial adjustment model, obtaining the Q value of each adjustment option corresponding to the frequency register in the digital frequency synthesizer of the sample RF power supply output by the initial adjustment model, and determining the sum of the maximum Q value and the current behavior reward value as the target Q value; Based on the difference between the target Q value and the Q value corresponding to the current adjustment coefficient of the frequency register in the digital frequency synthesizer of the sample RF power supply, the model loss is determined, and the parameters of the initial adjustment model are updated based on the model loss.

7. A method for calibrating the output frequency of a radio frequency power supply according to claim 5, characterized in that: The determining the current behavior reward value based on the frequency deviation reward value, the calibration efficiency reward value, and the stability reward value specifically includes: Determining the current behavior reward value based on the frequency deviation reward value and its weight, the calibration efficiency reward value and its weight, and the stability reward value and its weight; The weight of the frequency deviation reward value is greater than the weight of the stability reward value and the weight of the calibration efficiency reward value.

8. A radio frequency power supply output frequency calibration device, characterized in that: include: A state determination unit, used to determine the current frequency output state and environmental parameters of the radio frequency power supply; The current frequency output state of the RF power supply includes the real-time difference between the current calibrated output frequency of the RF power supply and the target frequency and the historical difference between the historical calibrated output frequency of the RF power supply and the target frequency during the historical calibration process; the environmental parameters include the temperature and humidity of the external environment where the RF power supply is located; A register adjustment unit, used to input the current frequency output state and environmental parameters into a frequency adjustment model, obtain an adjustment coefficient of a frequency register in a digital frequency synthesizer of the radio frequency power supply output by the frequency adjustment model, and adjust the frequency register based on the adjustment coefficient of the frequency register; An iterative calibration unit, configured to update the current calibrated output frequency based on the current actual output frequency of the radio frequency power supply; If the current calibration output frequency is inconsistent with the target frequency, the state determination unit and the register adjustment unit are repeatedly called.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, a method for calibrating the output frequency of a radio frequency power supply as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a method for calibrating the output frequency of a radio frequency power supply as claimed in any one of claims 1 to 7 is implemented.

Images