Self-constructing method of wavelet network approximation framework

Abstract

The application relates to a self-construction method of a wavelet network approximation framework, which comprises the following steps: setting a nonlinear function, obtaining training data, and processing the nonlinear function by using a wavelet network approximation framework, which comprises the following steps: determining an initial wavelet space for approximating the nonlinear function by using an initial wavelet frequency estimator of the wavelet network approximation framework; and increasing wavelet base functions on the basis of the initial wavelet space to further approximate the nonlinear function based on a wavelet base function increasing mechanism of the wavelet network approximation framework. The scheme can reduce the calculation complexity and accelerate the neural network training speed.

Description

Technical Field

[0001] This invention relates to the fields of wavelet functions and neural networks, and in particular to a self-construction method for a wavelet network approximation framework. Background Technology

[0002] In actual production processes, many dynamic systems often have nonlinear uncertainties, and these uncertainties continue to increase. In order for the control system to track a given target trajectory, it is usually necessary to parameterize the nonparametric uncertainties and then use adaptive control methods or adaptive iterative learning control methods to control the operation of the system. One basic idea is to represent the nonlinear function of the system as the product of unknown parameters and known nonlinear basis functions. Currently, there are many neural network methods for parameterization to approximate nonlinear functions.

[0003] Currently, there are numerous neural network methods for approximating nonlinear functions, including wavelet neural networks that use wavelet bases as activation functions. Unlike general basis functions, wavelet functions possess the characteristic of simultaneous time-domain and frequency-domain localization analysis. More importantly, the multi-resolution analysis of orthogonal wavelet functions provides the theoretical basis for training the network stepwise. Since the complexity of the nonlinear function being approximated cannot be obtained before training the network, the number of basis functions used for approximation is also uncertain. Therefore, adopting a strategy of adding orthogonal wavelet bases stepwise can save computational resources, and the orthogonality of wavelets ensures that the original wavelet basis weight coefficients remain unchanged after adding basis functions.

[0004] However, while some studies combining wavelet approximation networks with multiresolution analysis can effectively approximate nonlinear functions, the incremental increase of wavelets in multiresolution analysis slows down the training process. Due to the unique structure of wavelet bases, the number of wavelet bases increases rapidly with each increment, becoming even larger in high-dimensional cases, thus presenting limitations. Summary of the Invention

[0005] To address the aforementioned issues, the purpose of this application is to provide a self-constructed method for wavelet network approximation frameworks, which can process nonlinear functions based on wavelet network approximation frameworks, and can reduce computational complexity and accelerate training convergence speed.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, this application provides a self-construction method for wavelet network approximation frameworks, the method comprising:

[0008] A nonlinear function is set, and a wavelet network approximation framework is used to process the nonlinear function;

[0009] Processing the nonlinear function includes:

[0010] Using the initial wavelet frequency estimator of the wavelet network approximation framework, the initial wavelet space for approximating the nonlinear function is determined; and

[0011] Based on the wavelet basis function addition mechanism of the wavelet network approximation framework, wavelet basis functions are added to the initial wavelet space to further approximate the nonlinear function.

[0012] In one implementation of this application, the initial wavelet frequency estimator calculates the energy value of the nonlinear function projected onto each wavelet space, and determines the initial wavelet space based on an evaluation index of the variation law of the energy value of the nonlinear function projected onto each wavelet space.

[0013] In one implementation of this application, calculating the energy value of the nonlinear function projected onto each wavelet space includes:

[0014] From wavelet resolution Begin by selecting the wavelet space. The displacement centers are set by a small number of equally spaced wavelet bases. The energy of the nonlinear function projected onto the selected wavelet bases is calculated. For higher frequency wavelet spaces The energy estimation method first selects... The center of displacement in space is closest to The basis functions of the selected wavelet basis displacement centers in space are determined, and the energies of these newly selected basis functions are calculated. .

[0015] In one implementation of this application, the evaluation metric includes setting an exponential moving average (EMA).

[0016] In one implementation of this application, the evaluation index The array is the exponential moving average (EMA).

[0017] when Select As the initial wavelet frequency, and As the initial wavelet space.

[0018] In one implementation of this application, the wavelet basis function augmentation mechanism specifically includes: calculating the projection of the nonlinear function onto the initial wavelet space. The energy of each basis function in the space is determined according to the initial wavelet space. The basis function energy will The basis functions of the space are divided into several parts, and preferred to be used in the following ways: Basis functions with high spatial energy approximate nonlinear functions.

[0019] In one implementation of this application, the step of basing the initial wavelet space... The basis function energy will The basis functions of the space are divided into several parts, specifically including:

[0020] right The spatial basis functions are arranged in descending order of energy, and the wavelet basis is divided into several parts to find the spatial basis functions. The center of displacement of the basis function is closest to the space Wavelet basis of the basis function shift center, and according to The division will The space is also divided in the same way.

[0021] In one implementation of this application, the preferred use Basis functions with high spatial energy approximate nonlinear functions, specifically including:

[0022] Will The wavelet energies of the wavelet base parts obtained by spatial partitioning decrease sequentially. The part with the largest energy is selected to approximate the nonlinear function. If the approximation accuracy is not achieved, the part with the second largest energy is selected to approximate the nonlinear function. This process continues until the given accuracy is achieved.

[0023] An industrial control method, characterized in that it includes:

[0024] For the dynamic system to be controlled, a corresponding nonlinear system equation, the desired tracking trajectory of the control target, and the tracking error equation are set, wherein the nonlinear system equation contains a nonlinear function characterizing the uncertainty of the system.

[0025] The dynamic system is controlled by a preset adaptive control law, wherein the nonlinear function is approximated by a self-construction method of wavelet network approximation framework during the control process;

[0026] The self-construction method of the wave network approximation framework is the method of the first aspect.

[0027] The present invention has the following advantages due to the adoption of the above technical solutions:

[0028] This invention designs a self-constructed wavelet network approximation framework, which is suitable for approximating nonlinear functions. Compared with existing technologies, it provides a solution to accelerate training speed and reduce computational complexity. Attached Figure Description

[0029] Figure 1 This shows how the approximation error changes with the iteration axis in Example 1;

[0030] Figure 2This shows how the number of basis functions used for approximation varies along the iteration axis in Example 1;

[0031] Figure 3 This shows the variation of system tracking error over time in Example 2.

[0032] Figure 4 This shows how the number of basis functions used for approximation varies along the time axis in Example 2;

[0033] Figure 5 This illustration shows how the maximum tracking error of a control system, targeted by an industrial control method, varies along the iteration axis. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0035] This application provides a self-construction method for wavelet network approximation frameworks, the method comprising:

[0036] A nonlinear function is set, and a wavelet network approximation framework is used to process the nonlinear function;

[0037] Processing the nonlinear function includes:

[0038] Using the initial wavelet frequency estimator of the wavelet network approximation framework, the initial wavelet space for approximating the nonlinear function is determined; and

[0039] Based on the wavelet basis function addition mechanism of the wavelet network approximation framework, wavelet basis functions are added to the initial wavelet space to further approximate the nonlinear function.

[0040] In the embodiments of this application, the following class of multidimensional orthogonal wavelet basis functions are considered:

[0041]

[0042] in express 3D integer set, function By analyzing the wavelet mother function Obtained by scaling and translation, the size of the scaling and translation is determined by... and control; Indicates wavelet resolution. Indicates the wavelet frequency center. This indicates the wavelet shift center.

[0043] In the embodiments of this application, the following type of nonlinear function is considered:

[0044]

[0045] in Indicates input, express The dimension of nonlinear functions. yes Local Lipschitz functions on [the surface].

[0046] The initial wavelet space for approximating the nonlinear function is estimated using a wavelet network approximation framework, including:

[0047] A metric is provided to measure whether the spatial energy difference of wavelets is significant.

[0048] Estimate the energy of the nonlinear function projected onto the wavelet space and find the initial wavelet space based on the evaluation index.

[0049] A metric is provided to measure whether the spatial energy difference of wavelets is significant, specifically including:

[0050] Based on the exponential moving average (EMA) method, a metric is given as follows to measure whether the spatial energy difference of wavelets is significant:

[0051]

[0052] in, This represents the data obtained from the original calculation. This represents the result of an exponential moving average. Given the approximation accuracy;

[0053] Estimating the energy of the nonlinear function projected onto the wavelet space and finding the initial wavelet space based on evaluation metrics, specifically including:

[0054] set up Indicates resolution as The wavelet space, and Indicates frequency ratio to space In a higher wavelet space, a method for calculating the energy of the system's nonlinear function projected onto the wavelet subspace is given as follows:

[0055]

[0056] in Represents the nonlinear function of the system Projection in wavelet subspace Energy estimates, It is an orthogonal wavelet basis used to approximate the nonlinear function of a system. Describing wavelets The estimated value of the coefficient, The following is a three-step initial wavelet frequency estimation algorithm.

[0057] One: Order Let wavelet space The wavelet basis displacement center is , ,and This indicates the number of wavelet bases. From... The displacement centers are selected from a small number of equally spaced basis functions, and their displacement centers are set to be... ,in , and It is a relatively small constant determined by its own computational resources. Its corresponding basis function is expressed as: Let the training data be... ,in This indicates the amount of training data. A three-layer neural network, consisting of an input layer, hidden layers, and an output layer, is used to approximate a nonlinear function. And obtain the estimated wavelet basis coefficients, which are also the weights of the hidden layer activation function. First, set the initial values ​​of the wavelet basis coefficients. Set to 0. Train the neural network once and obtain the wavelet basis coefficients using the following update method. :

[0058]

[0059] in Indicates the learning rate. This represents the output of the neural network. Subsequently, wavelet spatial energy calculation is used to obtain... :

[0060]

[0061] make .

[0062] Second: Order Select The wavelet displacement center is closest to in space Spatial selected wavelet displacement center The basis functions are expressed as... The corresponding wavelet basis is represented as Newly selected wavelet basis coefficient estimates Initialized to 0 Similar to the first step, training the wavelet neural network once yields the updated wavelet coefficients:

[0063]

[0064] And according to Calculated and .if If so, the algorithm ends and a selection is made. Use this as the initial wavelet frequency; otherwise, proceed to step three.

[0065] Three: Order Repeat step two until... , and select As the initial wavelet frequency.

[0066] In the first and second steps, the neural network is trained only once to obtain the wavelet coefficient estimates. However, in practical applications, the neural network can be trained multiple times according to the actual situation to obtain the wavelet coefficient estimates.

[0067] According to the wavelet network approximation framework, new basis functions are added to the initial wavelet space to approximate the system's nonlinear function until the approximation error converges to a given accuracy, including:

[0068] Based on the initial wavelet space Basis functions are selected from higher frequency wavelet spaces. Basis functions are used to partition the space; wavelet space is utilized preferentially. High-energy basis functions approximate the nonlinear functions of the system, including:

[0069] 1. Set the initial wavelet space. The displacement center is ,in , express The number of spatial wavelet bases. Based on the training data. Utilizing scale space and wavelet space The center of displacement is Wavelet bases are used as activation functions to train wavelet neural networks for practical applications. Given a cutoff threshold... Let the first The approximation error in the next iteration is expressed as:

[0070]

[0071] in This represents the output value of the wavelet neural network in the current iteration:

[0072]

[0073] in Indicates the first In the nth iteration The estimated values ​​of the wavelet coefficients are calculated, and all wavelet coefficient estimates are initialized to 0. If a given approximation accuracy is not achieved, the neural network is trained until... ;set up Separation factor and It is one-integer fraction. Representation space Energy is the largest and accounts for the largest proportion of energy. The number of wavelets, in practical applications, is rounded up to an integer, and the system nonlinear function is selected. Projected on The front with the highest energy in space Let each of the basis functions be a base function and let its displacement center be a base function. The details are as follows:

[0074]

[0075] in ,in Indicates the first Displacement center in each dimension ;

[0076] Second: Based on the aforementioned displacement center Select The center of displacement in space is closest to The wavelet basis functions are given, and their displacement centers are set to... The details are as follows:

[0077]

[0078]

[0079] in , Representing space The number of wavelet bases, the function The definition is as follows:

[0080] represent The smallest of this vector There are values, among which It is a vector;

[0081] Will take all The two elements in.

[0082] Third: Prioritize the use of wavelet space High-energy basis functions approximate the nonlinear functions of the system, specifically including:

[0083] set up This represents the set of wavelet basis functions, initially empty; according to the... Space-specific wavelet bases will , All basis functions in the space and the newly selected In space Each base function is added to the pool. In, and the newly selected The coefficients of the spatial basis functions are initialized to 0; similar to the first step, using... The basis functions in the training method are used to train the neural network until the truncation threshold is reached. Continue to select the system nonlinear function Projected on The front with the highest energy in space basis functions, and calculate the obtained Spatial wavelet basis functions are placed into the pool. Select The basis functions are selected in the first step. of Based on this, it has already been placed in the pool. of The basis functions do not need to be added repeatedly; the coefficients of newly added basis functions are initialized to 0, similar to the first step, using... The basis functions continue to be trained, and this process continues until the approximate accuracy is reached. .

[0084] The following two specific numerical examples, Example 1 and Example 2, further illustrate the practical effect of the above method.

[0085] Calculation example 1:

[0086] Consider the following nonlinear function:

[0087]

[0088] in . From Obtained by uniform sampling in the middle, according to We then obtained the dataset The dataset was then divided into a training set. and test set ,in The data was randomly assigned to the training set. This simulation uses the Sinc wavelet function. Loss Used to represent the approximation error during training ;parameter Set as The calculation of the initial wavelet frequency using the wavelet initial frequency estimator includes:

[0089] Wavelet shift center Set as ,because 0.36, obtained The initial wavelet resolution was calculated to be 2, meaning the initial wavelet frequency was... Adding wavelet bases to the initial wavelet space includes:

[0090] Initial wavelet frequency Substituting into the wavelet basis addition algorithm, then for .parameter , Set as Train the wavelet neural network until a given accuracy is achieved.

[0091] We name the proposed method in this application CWNN. In Example 1, we compare the proposed CWNN with two other methods: a general neural network GNN with ReLU activation function and a wavelet neural network WNN that progressively increases wavelet basis functions according to resolution. It is worth noting that GNN lacks the ability to adjust the network structure. When the expected approximation accuracy for a given training set is not met, both CWNN and WNN can add new basis functions. The difference between CWNN and WNN is that WNN randomly selects the initial wavelet basis, and each time WNN adds a wavelet basis, it adds basis functions to the entire wavelet subspace. Example 1 uses backpropagation to update its weights. Notably, as a proof-of-concept illustration, all three methods—CWNN, GNN, and WNN—comprise three layers: an input layer, a hidden layer, and an output layer. The number of nodes in the input layer of GNN is fixed at 500.

[0092] Figure 1 This explains the use of training data The training process approximates the error. (Loss) The transformation situation, where (·) Let represent the logarithmic function with base 2. The results show that CWNN converges to the desired accuracy significantly faster than GNN and WNN. Notably, GNN failed to converge to the predefined accuracy achieved by other methods. To further evaluate the approximation accuracy, we compare it on the test set... The effectiveness of the three neural networks is evaluated using the loss on the loss scale. Figure 1 The accuracy and standard deviation of these three methods are summarized. The results show that CWNN achieves similar accuracy to WNN, but CWNN exhibits additional advantages in other aspects.

[0093] Besides convergence speed and accuracy during training, computational cost is crucial for neural networks. Figure 2 The number of basis (activation) functions used in each iteration of the three neural networks is given. WNN extraction Training begins with all 50 basis functions, and a new one is added during the 517th iteration. The GNN uses 81 base cells for training until convergence; GNN uses a fixed number of 500 activation functions; while CWNN, similar to WNN, extracts... Training begins with all 50 basis functions, but a new one is added during the 129th iteration. Train the 162 bases with the highest energy until convergence.

[0094] Calculation example 2:

[0095] Example 2 applies the self-constructed wavelet neural network (CWNN) described in this application to adaptive wavelet control (AWC) to approximate uncertainties in a dynamical system. Consider the following nonlinear dynamical system:

[0096]

[0097] in It is the system status. It is system input. It is a nonlinear function to be approximated. The target tracking trajectory is... Furthermore, the tracking accuracy is set to 0.04, with the goal of ensuring the system state tracks the given trajectory by controlling the system input. The CWNN proposed in this application can approximate... And tracked online .

[0098] To evaluate tracking performance over a period of time and determine when to add a new wavelet basis, we introduce a parameter called dwell time. It calculates the cumulative tracking performance over this time period, and adds a new wavelet basis if the distance between the maximum tracking errors of two adjacent dwell times is less than a given threshold. In Example 2, the dwell time is set to 50. The system input in this example... The following is given:

[0099]

[0100] in And it is a constant. Indicates time The total number of wavelets used for approximation Indicates the first The coefficient estimates of the wavelet, And it is a constant. Indicates system state error. This indicates the tracking error.

[0101] This simulation example applies a wavelet network framework to industrial system control and uses an adaptive control method to solve the problem. The wavelet parameter update method used is adapted to the actual application context and differs from the update method described in the framework. The parameters in this example... The update law is given as follows:

[0102]

[0103] In this simulation, the parameters All values ​​are 20.

[0104] parameter Set to 0.36, 0.02, in the wavelet initial frequency estimator Set as The initial state error is set to 0, i.e., The initial wavelet resolution is calculated to be 1, and the initial wavelet frequency is then obtained. Subsequently, a new wavelet basis is added to the initial wavelet space to further approximate the nonlinear uncertainty, and the separation factor is... Set as In the wavelet basis addition algorithm for Threshold-based cutoff mechanism in It is triggered and a new wavelet basis is added.

[0105] Figure 3 The system's dwell time was displayed. Achieve a given tracking accuracy . Figure 4 In the middle, initial All 50 basis functions of the space are used, in hour, Twelve basis functions are added in the space to achieve the given tracking accuracy.

[0106] These two examples demonstrate that the method proposed in this invention, based on its designed wavelet network framework, can accelerate training and reduce computational complexity.

[0107] Calculation example 3:

[0108] Consider a dynamical system with the following characteristics:

[0109]

[0110] The system state is: nonlinear functions , Representing the iterative batch, the system target tracking trajectory is: This system considers a finite time interval. The iterative learning control is used, with the initial state error set to 0 for each iteration.

[0111] The system input in this example The following is given:

[0112]

[0113] in And it is a constant. Indicates time The total number of wavelets used for approximation Indicates the first The wavelet in the th... The coefficient estimates for the next iteration. And it is a constant. Indicates the first The system state error of the next iteration. Indicates the first The tracking error of the next iteration.

[0114] This simulation example applies a wavelet network framework to industrial system control and uses an adaptive iterative learning control method to solve the problem. The wavelet parameter update method used is adapted to the practical application context and differs from the update method described in the framework. The parameters in this example... The update law is given as follows:

[0115]

[0116]

[0117] In this simulation, the parameters All values ​​are 20, parameter Set to 0.8, system tracking accuracy Set to 0.02.

[0118] Based on the method for finding the initial wavelet frequency in the aforementioned wavelet network framework, the following is calculated: The initial wavelet space is then calculated as follows: .

[0119] First, select the initial wavelet space. The center of displacement is A total of 25 wavelet bases were used as the initial wavelet bases for approximation. During the approximation process, according to the threshold truncation mechanism of the added wavelet basis, the truncation threshold of the added wavelet is set to 0.002. In example 3, the threshold threshold is... The space is divided into three parts, that is... Subsequently, based on the aforementioned wavelet network framework applied to the adaptive iterative learning method, the system tracking error was continuously controlled to converge to the tracking accuracy.

[0120] from Figure 5 As can be seen, the method proposed in this invention, based on its designed wavelet network framework, can quickly converge to a given tracking accuracy.

[0121] In the embodiments provided by this invention, it should be understood that the disclosed methods can be implemented in other ways. For example, the embodiments described above are merely illustrative. The integrated units implemented as software functional units can be stored in a computer-readable storage medium. The software functional units stored in a storage medium include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An industrial control method, characterized in that, include: For the dynamic system to be controlled, a corresponding nonlinear system equation, the desired tracking trajectory of the control target, and the tracking error equation are set, wherein the nonlinear system equation contains a nonlinear function characterizing the uncertainty of the system. The dynamic system is controlled by a preset adaptive control law, wherein the nonlinear function is approximated by a self-construction method of wavelet network approximation framework during the control process; The self-construction method of the wavelet network approximation framework includes: A nonlinear function is set, and a wavelet network approximation framework is used to process the nonlinear function; Processing the nonlinear function includes: Using the initial wavelet frequency estimator of the wavelet network approximation framework, the initial wavelet space for approximating the nonlinear function is determined; and Based on the wavelet basis function addition mechanism of the wavelet network approximation framework, wavelet basis functions are added on the basis of the initial wavelet space to further approximate the nonlinear function. The wavelet basis function augmentation mechanism specifically includes: calculating the projection of the nonlinear function onto the initial wavelet space. The energy of each basis function in the space is determined according to the initial wavelet space. The basis function energy will The basis functions of the space are divided into several parts, and preferred to be used in the following ways: Basis functions with high spatial energy approximate nonlinear functions; The initial wavelet space The basis function energy will The basis functions of the space are divided into several parts, specifically including: The spatial basis functions are arranged in descending order of energy, and the wavelet basis is divided into several parts to find the spatial basis functions. The center of displacement of the basis function is closest to the space Wavelet basis of the basis function shift center, and according to The division will The space is also divided in the same way; The preferred use Basis functions with high spatial energy approximate nonlinear functions, specifically including: The wavelet energies of the wavelet base parts obtained by spatial partitioning decrease sequentially. The part with the largest energy is selected to approximate the nonlinear function. If the approximation accuracy is not achieved, the part with the second largest energy is selected to approximate the nonlinear function. This process continues until the given accuracy is achieved.

2. The industrial control method according to claim 1, characterized in that, The initial wavelet frequency estimator calculates the energy value of the nonlinear function projected onto each wavelet space, and determines the initial wavelet space based on the evaluation index of the energy value variation law of the nonlinear function projected onto each wavelet space.

3. The industrial control method according to claim 2, characterized in that, The calculation of the energy value of the nonlinear function projected onto each wavelet space includes: From wavelet resolution Begin by selecting the wavelet space. The displacement centers are set by a small number of equally spaced wavelet bases. The energy of the nonlinear function projected onto the selected wavelet bases is calculated. For higher frequency wavelet spaces The energy estimation method first selects... The center of displacement in space is closest to The basis functions of the selected wavelet basis displacement centers in space are determined, and the energies of these newly selected basis functions are calculated. .

4. The industrial control method according to claim 3, characterized in that, The evaluation metrics include the exponential moving average (EMA).

5. The industrial control method according to claim 4, characterized in that, The evaluation indicators The value is the exponential moving average (EMA). when Select As the initial wavelet frequency, and As the initial wavelet space.

6. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the industrial control method according to any one of claims 1 to 5.

Images