Neural adversarial network optimization method based on KAN generator replacement and acyclic constraint
By replacing the generator as a KAN generator in the neural adversarial network and introducing a ring-free linear constraint term, the generation task performance and stability of the neural adversarial network are optimized, and the problems of unstable convergence and pattern crash in traditional network generation tasks are solved, thereby achieving higher quality and diverse data generation.
Patent Information
- Application Number
- CN202411818342.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-11
- Publication Date
- 2025-04-08
AI Technical Summary
Traditional neural adversarial networks have problems such as unstable convergence and pattern collapse in the generation task, and the feasibility of KAN neural adversarial networks in actual applications is insufficient, and the quality and diversity of generated data are poor.
The neural adversarial network optimization method based on KAN generator replacement and acyclic constraints is adopted. The SAM generator of the SAM neural adversarial network is replaced as the KAN generator, and a ring-free linear constraint term is introduced into the loss function to optimize the loss function and parameter update process.
The performance and stability of neural adversarial networks in the generation task are improved, the quality and diversity of generated data are enhanced, making it closer to the real data distribution, covering more data patterns, and avoiding problems such as pattern crashes.
Smart Images
Figure CN119783728A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning technology, and in particular to a neural adversarial network optimization method based on KAN generator replacement and acyclicity constraints. Background Art
[0002] Neural adversarial network is a deep learning framework that completes model training through the mutual adversarial process between the generator and the discriminator. Although traditional neural adversarial networks have achieved remarkable results in the fields of image generation and sequence data generation, they still have some problems, such as unstable convergence and mode collapse. In order to solve these problems, new regularization terms can be introduced, loss functions can be improved, etc.
[0003] The KAN neural adversarial network is superior to the traditional multi-layer perceptron in terms of accuracy and interpretability. It uses spline functions as activation functions, has learnable parameters, and can more flexibly represent nonlinear relationships. However, the feasibility of the KAN neural adversarial network in practical applications is insufficient, the quality and diversity of the generated data are poor, the data patterns covered are scarce, and its comparison results with the multi-layer perceptron are also limited by the experimental results under different parameters or floating-point operations. The training process is not stable enough and is prone to mode collapse. The SAM neural adversarial network is a neural adversarial network for specific applications with a unique network structure and loss function. Therefore, the SAM generator of the SAM neural adversarial network can be replaced by the KAN generator of the KAN neural adversarial network.
[0004] Therefore, how to improve the generator part of the neural adversarial network and optimize the loss function to improve the performance and stability of the neural adversarial network in generation tasks has become a problem that needs to be solved urgently. Summary of the invention
[0005] In order to solve the above problems, the present invention proposes a neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint, which can improve the generator part of the neural adversarial network and optimize the loss function, improve the performance and stability of the neural adversarial network in generation tasks, and is widely used in the fields of image processing and sequence data generation.
[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:
[0007] The present invention is a neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint, comprising the following steps:
[0008] Collecting a training data set and performing a preprocessing operation on the training data set;
[0009] Constructing a KAN neural adversarial network and a SAM neural adversarial network, replacing the SAM generator of the SAM neural adversarial network with the KAN generator of the KAN neural adversarial network, and keeping the SAM discriminator of the SAM neural adversarial network unchanged;
[0010] Setting an acyclic linear constraint term and introducing it into the loss function corresponding to the SAM neural adversarial network;
[0011] Based on the preprocessed training data set, the SAM neural adversarial network is trained in an iterative update manner, and the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network are updated based on the loss function;
[0012] The performance of the trained SAM neural adversarial network is evaluated, and the SAM neural adversarial network is adjusted and optimized according to the evaluation results.
[0013] A further improvement of the present invention is that the pre-processing operation specifically includes:
[0014] Cleaning the collected training data set, that is, removing fuzzy and repeated training data;
[0015] Removing noise from the training data set based on a Gaussian filtering algorithm;
[0016] Using a histogram equalization method to enhance the contrast of the training data set;
[0017] The training data set is normalized.
[0018] A further improvement of the present invention is that the KAN generator uses a B-spline activation function, and the mathematical formula of the B-spline activation function is:
[0019]
[0020] In the formula, B i (x) represents the value of the i-th B-spline activation function at the generated data point x; a i,k represents the activation coefficient of the i-th B-spline activation function at order k; yi represents the node position corresponding to the i-th B-spline activation function; n represents the total order of the i-th B-spline activation function.
[0021] A further improvement of the present invention is that the setting of the acyclic linear constraint term and its introduction into the loss function corresponding to the SAM neural adversarial network specifically includes:
[0022] Set the KAN generator to G, and map the random noise vector z to the generated data point x, that is, x=G(z);
[0023] The acyclic linear constraint term represents a constraint on the distance between the generated data points x. The mathematical formula of the acyclic linear constraint term is:
[0024]
[0025] Where L represents the acyclic linear constraint; N represents the total number of generated data points; z p represents the pth random noise vector; z q represents the qth random noise vector; d(G(z p ), G(z q )) represents the generated data point G(z p ) and G(z q ) between them; δ represents the positive threshold corresponding to the acyclic linear constraint, which is used to control the minimum distance between generated data points; max() represents the maximum value operation.
[0026] A further improvement of the present invention is that: a regularization term is introduced into the loss function corresponding to the SAM neural adversarial network, and the regularization term is related to the gradient of the SAM discriminator of the SAM neural adversarial network;
[0027] Set the SAM discriminator to D, input the generated data point x to the SAM discriminator D, and output the corresponding scalar value, i.e., the gradient penalty term. The mathematical formula of the gradient penalty term is:
[0028]
[0029] Where T represents the gradient penalty term; λ represents the regularization coefficient, which is used to control the intensity of the gradient penalty; Represents any sampling distribution between the training data distribution and the generated data distribution; Represents the sampling distribution The sampled data points in ; Indicates that the SAM discriminator D is at the sampling data point The L2 norm of the gradient at .
[0030] A further improvement of the present invention is that: based on the preprocessed training data set, the SAM neural adversarial network is trained in an iterative update manner, and the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network are updated based on the loss function, specifically including:
[0031] Calculate the gradients of the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network based on the loss function;
[0032] Based on the gradient, the Adam optimization algorithm is used to update the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network. The update formula of the Adam optimization algorithm is as follows:
[0033] m t =β1m t-1 +(1-β1)g t
[0034] V t =β2V t-1 +(1-β2)g t 2
[0035]
[0036]
[0037]
[0038] In the formula, m t and m t-1 Represents the first-order moment estimate of the gradient at the current time step t and the previous time step t-1 respectively; v t and v t-1 Represents the second-order moment estimate of the gradient at the current time step t and the previous time step t-1 respectively; g t represents the gradient of the current time step t; β1 and β2 represent the decay rates corresponding to the first-order moment estimate and the second-order moment estimate, respectively, which are used to control the deviations of the first-order moment estimate and the second-order moment estimate, respectively; η represents the learning rate, which is used to control the step size of parameter update; ∈ represents the stability constant of the Adam optimization algorithm; θ t and θ t-1 Represent the update parameters of the current time step t and the previous time step t-1 respectively; and They represent the first-order moment estimate and second-order moment estimate of the gradient of the current time step t after deviation correction, respectively.
[0039] A further improvement of the present invention is that the performance of the trained SAM neural adversarial network is evaluated, and the SAM neural adversarial network is adjusted and optimized according to the evaluation result, specifically including:
[0040] For the trained SAM neural adversarial network, evaluating the performance of the SAM neural adversarial network, and monitoring the change of the loss function of the SAM neural adversarial network, the quality of the generated data, and the accuracy of the SAM discriminator;
[0041] The parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network are adjusted according to the monitoring results to optimize the SAM neural adversarial network.
[0042] The beneficial effects of the present invention are as follows: the present invention proposes a neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint, by collecting a training data set and performing a preprocessing operation on the training data set; constructing a KAN neural adversarial network and a SAM neural adversarial network, replacing the SAM generator of the SAM neural adversarial network with the KAN generator of the KAN neural adversarial network, and keeping the SAM discriminator of the SAM neural adversarial network unchanged; setting an acyclic linear constraint term and introducing it into the loss function corresponding to the SAM neural adversarial network; based on the preprocessed training data set, training the SAM neural adversarial network in an iterative update manner, and updating the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network based on the loss function; evaluating the performance of the trained SAM neural adversarial network, and adjusting and optimizing the SAM neural adversarial network according to the evaluation results, thereby improving the quality and diversity of the generated data, making it closer to the real data distribution, and being able to cover more data patterns; and the training process is more stable, avoiding problems such as pattern collapse, and improving the feasibility and reliability of the neural adversarial network in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 A schematic flow chart of a neural adversarial network optimization method based on KAN generator replacement and acyclicity constraints provided in an embodiment of the present invention;
[0044] Figure 2 A schematic diagram of a preprocessing operation provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0045] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0046] See also Figure 1 The neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint of this embodiment includes the following steps:
[0047] Collecting a training data set and performing a preprocessing operation on the training data set;
[0048] Constructing a KAN neural adversarial network and a SAM neural adversarial network, replacing the SAM generator of the SAM neural adversarial network with the KAN generator of the KAN neural adversarial network, and keeping the SAM discriminator of the SAM neural adversarial network unchanged;
[0049] Setting an acyclic linear constraint term and introducing it into the loss function corresponding to the SAM neural adversarial network;
[0050] Based on the preprocessed training data set, the SAM neural adversarial network is trained in an iterative update manner, and the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network are updated based on the loss function;
[0051] The performance of the trained SAM neural adversarial network is evaluated, and the SAM neural adversarial network is adjusted and optimized according to the evaluation results.
[0052] First, representative, diverse, and high-quality data samples can be collected from real or simulated environments as training data sets to ensure that they can fully cover various situations of the target problem. Subsequently, the training data sets can be preprocessed, including data cleaning, data normalization, and data enhancement, so as to improve data quality and reduce bias and variance during model training. Specifically, data cleaning includes removing noise and filling missing values; data normalization includes adjusting the data scale to eliminate dimensional effects; and data enhancement includes generating more training samples through transformation to enhance model robustness.
[0053] Furthermore, two neural adversarial networks can be constructed, namely the KAN neural adversarial network and the SAM neural adversarial network, and the SAM generator in the SAM neural adversarial network can be replaced by the KAN generator in the KAN neural adversarial network, while keeping the SAM discriminator of the SAM neural adversarial network unchanged, so as to integrate the advantages of the two networks and make full use of the feature generation ability of the KAN generator and the advantages of the SAM discriminator in the selective attention mechanism.
[0054] In order to further improve the stability and training effect of the model, acyclic linear constraints can be introduced and integrated into the loss function corresponding to the SAM neural adversarial network. This constraint can reduce the cyclic dependency and redundant information in the model output and ensure the consistency and diversity of the generated data.
[0055] Based on the preprocessed high-quality training data set, the SAM neural adversarial network can be trained in an iterative update manner. During the training process, the parameters of the KAN generator and the SAM discriminator can be dynamically adjusted and updated based on the gradient information of the loss function, and the network structure can be continuously optimized through the back-propagation algorithm until the preset training stop condition is reached.
[0056] Finally, a comprehensive performance evaluation can be performed on the trained SAM neural adversarial network, including quantitative and qualitative evaluations. Specifically, the quantitative evaluation includes the quality score of the generated samples, the discrimination accuracy, etc.; the qualitative evaluation includes the diversity of the generated samples and the subjective evaluation of authenticity. Based on the evaluation results, model tuning strategies can be adopted, such as adjusting the network architecture, modifying the loss function, and adding regularization terms, to optimize the SAM neural adversarial network.
[0057] The pre-processing operation specifically includes:
[0058] Cleaning the collected training data set, that is, removing fuzzy and repeated training data;
[0059] Removing noise from the training data set based on a Gaussian filtering algorithm;
[0060] Using a histogram equalization method to enhance the contrast of the training data set;
[0061] The training data set is normalized.
[0062] Understandably, see Figure 2 , the collected training data set can be cleaned to remove ambiguous, duplicate or invalid data to ensure the purity and representativeness of the data set. In actual operation, data cleaning tools or specific cleaning scripts can be used to automatically identify and remove data items that do not meet the requirements by setting a series of rules and conditions, such as data value range, data type matching, etc. In addition, for the processing of fuzzy data, fuzzy matching algorithms or manual review can be used to ensure the accuracy of the data.
[0063] Secondly, the noise in the training data set can be removed based on the Gaussian filtering algorithm. Noise is an inevitable part of the data set, which may come from errors in the data acquisition process, distortion in the transmission process, etc. As a common image processing and signal processing algorithm, the Gaussian filtering algorithm can perform weighted averaging of pixel values in the neighborhood by constructing a Gaussian function as a weight, thereby effectively suppressing noise and retaining the main features of the image or signal. In the preprocessing of the training data set, the Gaussian filtering algorithm can improve the signal-to-noise ratio of the training data set and provide a more reliable data basis for subsequent analysis and modeling.
[0064] The contrast of the training data set can be enhanced by using the histogram equalization method. Contrast is an indicator that measures the degree of difference in light and dark in an image or data. For tasks such as image recognition and classification, high contrast often means higher recognition accuracy. The histogram equalization method can adjust the grayscale distribution of the image to make the grayscale value evenly distributed throughout the range, thereby enhancing the contrast of the image. In the preprocessing of the training data set, this method can improve the visual effect of the data set and provide clearer data support for subsequent feature extraction and model training.
[0065] Finally, the training data set can be normalized. By scaling the data to a specific range, such as between 0 and 1, the dimensional differences between different features can be eliminated, allowing the model to converge faster during training and improve the generalization ability of the model. In the preprocessing of the training data set, normalization can ensure that the data has been fully preprocessed and optimized before entering the model.
[0066] The KAN generator uses a B-spline activation function, and the mathematical formula of the B-spline activation function is:
[0067]
[0068] In the formula, B i (x) represents the value of the i-th B-spline activation function at the generated data point x; a i,k represents the activation coefficient of the i-th B-spline activation function at order k; y i represents the node position corresponding to the i-th B-spline activation function; n represents the total order of the i-th B-spline activation function.
[0069] It is understandable that the KAN generator uses the B-spline activation function, which can efficiently generate data with specific characteristics. Specifically, the B-spline activation function can significantly improve the expression and generalization capabilities of the neural network. The corresponding mathematical formula is: Among them, B i (x) represents the value of the i-th B-spline activation function at the generated data point x, which depends not only on the current data point but also on multiple parameters of the B-spline activation function. i，k represents the activation coefficient of the i-th B-spline activation function at order k, which determines the contribution of B-spline functions of different orders to the generation of data points x, and by adjusting these activation coefficients, the neural network can flexibly adjust its output to adapt to different data distributions and task requirements.
[0070] y irepresents the node position corresponding to the i-th B-spline activation function. These node positions are key points in the domain of the B-spline function, which determine the shape and characteristics of the B-spline function. By setting these node positions reasonably, the neural network can better capture the local features of the data, thereby improving the performance of the model.
[0071] n represents the total order of the i-th B-spline activation function. Generally speaking, the higher the order, the better the smoothness of the B-spline activation function, but its computational complexity will also increase accordingly.
[0072] By adopting the B-spline activation function, the KAN generator can improve the expressiveness and generalization capabilities of the neural network, providing it with more flexibility and controllability when processing complex data relationships.
[0073] The setting of the acyclic linear constraint term and introducing it into the loss function corresponding to the SAM neural adversarial network specifically includes:
[0074] Set the KAN generator to G, and map the random noise vector z to the generated data point x, that is, x=G(z);
[0075] The acyclic linear constraint term represents a constraint on the distance between the generated data points x. The mathematical formula of the acyclic linear constraint term is:
[0076]
[0077] Where L represents the acyclic linear constraint; N represents the total number of generated data points; z p represents the pth random noise vector; z q represents the qth random noise vector; d(G(z p ), G(z q )) represents the generated data point G(z p ) and G(z q ) between them; δ represents the positive threshold corresponding to the acyclic linear constraint, which is used to control the minimum distance between generated data points; max() represents the maximum value operation.
[0078] Among them, first, the KAN generator can be set to G. The function of the KAN generator G is to map the random noise vector z to the generated data point x, and the corresponding formula is x=G(z). The random noise vector z usually comes from a preset probability distribution, such as a normal distribution or a uniform distribution, and the generated data point x is generated by the KAN generator G according to the random noise vector z, which is used to simulate the sample of the real data distribution.
[0079] Next, we can introduce acyclic linear constraints to constrain the distance between generated data points x to prevent the generated data points from being too close or forming a ring structure. During the training process, if the generated data points are too close, it may lead to mode collapse, that is, the generator only generates a few similar samples and cannot cover the entire real data distribution. The introduction of acyclic linear constraints can solve this problem and improve the diversity and quality of generated data.
[0080] The corresponding formula for the acyclic linear constraint term is: Where L represents the acyclic linear constraint term; N represents the total number of generated data points, that is, the number of samples generated by the KAN generator G according to the random noise vector z in one training iteration; z p represents the pth random noise vector; z q represents the qth random noise vector; d(G(z p ), G(z q )) represents the generated data point G(z p ) and G(z q ), which can be calculated by metrics such as Euclidean distance and cosine similarity; δ represents the positive threshold corresponding to the acyclic linear constraint, which is used to control the minimum distance between generated data points to avoid data points being too close; max() represents the maximum value operation, which is used to ensure that when the distance between two generated data points is less than δ, the acyclic linear constraint L can generate a positive penalty value, thereby guiding the KAN generator G to adjust its parameters to increase the distance between generated data points.
[0081] In practical applications, the introduction of acyclic linear constraints requires corresponding modifications to the loss function of the SAM neural adversarial network. Specifically, the acyclic linear constraint term L can be added to the original loss function as a regularization term to form a new loss function. During the training process, by minimizing the new loss function, the KAN generator G can meet the requirements of acyclic linear constraints while generating high-quality data, thereby improving the diversity and quality of generated data.
[0082] Introducing a regularization term into the loss function corresponding to the SAM neural adversarial network, and the regularization term is related to the gradient of the SAM discriminator of the SAM neural adversarial network;
[0083] Set the SAM discriminator to D, input the generated data point x to the SAM discriminator D, and output the corresponding scalar value, i.e., the gradient penalty term. The mathematical formula of the gradient penalty term is:
[0084]
[0085] Where T represents the gradient penalty term; λ represents the regularization coefficient, which is used to control the intensity of the gradient penalty; Represents any sampling distribution between the training data distribution and the generated data distribution; Represents the sampling distribution The sampled data points in ; Indicates that the SAM discriminator D is at the sampling data point The L2 norm of the gradient at .
[0086] Among them, the SAM discriminator can be set as D to distinguish between real data points and generated data points. In order to introduce a regularization effect, the generated data point x can be input into the SAM discriminator D, and the corresponding scalar value, i.e., the gradient penalty term, can be output. The gradient penalty term is used to constrain the size of the discriminator gradient to prevent the discriminator from changing too drastically near the data point, so that the discriminator maintains a certain smoothness.
[0087] Specifically, the mathematical formula for the gradient penalty term is: Among them, T represents the gradient penalty term, and λ represents the regularization coefficient, which is used to adjust the intensity of the gradient penalty. In practical applications, the value of the regularization coefficient λ needs to be fine-tuned according to the specific tasks and data to achieve the best regularization effect. represents any sampling distribution between the training data distribution and the generated data distribution, Represents the sampling distribution The sampled data points in are used for gradient calculation and penalty. Indicates that the SAM discriminator D is at the sampling data point The L2 norm of the gradient at , which measures the size of the discriminator's gradient at that point. By comparing the L2 norm of the gradient with 1 and calculating the square of the difference, the specific value of the gradient penalty term can be obtained. The larger this value is, the more drastic the gradient change of the discriminator near the data point is, so a larger penalty can be imposed on it to make it smoother.
[0088] By introducing a regularization term related to the SAM discriminator gradient into the loss function of the SAM neural adversarial network and combining it with the corresponding gradient penalty term, the generalization ability and stability of the neural adversarial network can be effectively enhanced.
[0089] The method of training the SAM neural adversarial network in an iterative updating manner based on the preprocessed training data set and updating the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network based on the loss function specifically includes:
[0090] Calculate the gradients of the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network based on the loss function;
[0091] Based on the gradient, the Adam optimization algorithm is used to update the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network. The update formula of the Adam optimization algorithm is as follows:
[0092] m t =β1m t-1 +(1-β1)g t
[0093] v t =β2v t-1 +(1-β2)g t 2
[0094]
[0095]
[0096]
[0097] In the formula, m t and m t-1 Represents the first-order moment estimate of the gradient at the current time step t and the previous time step t-1 respectively; v t and v t-1 Represents the second-order moment estimate of the gradient at the current time step t and the previous time step t-1 respectively; g t represents the gradient of the current time step t; β1 and β2 represent the decay rates corresponding to the first-order moment estimate and the second-order moment estimate, respectively, which are used to control the deviations of the first-order moment estimate and the second-order moment estimate, respectively; η represents the learning rate, which is used to control the step size of parameter update; ∈ represents the stability constant of the Adam optimization algorithm; θ t and θ t-1 Represent the update parameters of the current time step t and the previous time step t-1 respectively; and They represent the first-order moment estimate and second-order moment estimate of the gradient of the current time step t after deviation correction, respectively.
[0098] The step of evaluating the performance of the trained SAM neural adversarial network and adjusting and optimizing the SAM neural adversarial network according to the evaluation result specifically includes:
[0099] For the trained SAM neural adversarial network, evaluating the performance of the SAM neural adversarial network, and monitoring the change of the loss function of the SAM neural adversarial network, the quality of the generated data, and the accuracy of the SAM discriminator;
[0100] The parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network are adjusted according to the monitoring results to optimize the SAM neural adversarial network.
[0101] In practical applications, we can first conduct a comprehensive performance evaluation of the trained SAM neural adversarial network. Specifically, by monitoring the change trend of the loss function with the number of training rounds, we can determine whether the network has fully converged and whether there is a risk of overfitting or underfitting; by comparing the difference between the generated data and the real data, we can evaluate the effect of the generator; and by comparing the accuracy of the SAM discriminator, the discriminator's task is to distinguish between real data and generated data, and its accuracy will directly affect the effect of adversarial training.
[0102] After completing the performance evaluation, the SAM neural adversarial network can be adjusted and optimized according to the monitoring results. Specifically, the Adam optimizer can be fully utilized to fine-tune the parameters of the KAN generator and the SAM discriminator, and then the performance of the neural adversarial network can be further improved by optimizing the parameter configuration. At the same time, the generalization ability and robustness of the neural adversarial network can be improved by introducing regularization technology and data enhancement strategies.
[0103] Through the introduction of the above embodiments, the present invention adopts a neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint, collects training data sets, and preprocesses the training data sets; constructs a KAN neural adversarial network and a SAM neural adversarial network, replaces the SAM generator of the SAM neural adversarial network with the KAN generator of the KAN neural adversarial network, and keeps the SAM discriminator of the SAM neural adversarial network unchanged; sets acyclic linear constraint terms and introduces them into the loss function corresponding to the SAM neural adversarial network; based on the preprocessed training data set, trains the SAM neural adversarial network in an iterative update manner, and updates the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network based on the loss function; evaluates the performance of the trained SAM neural adversarial network, and adjusts and optimizes the SAM neural adversarial network according to the evaluation results, thereby improving the quality and diversity of the generated data, making it closer to the real data distribution, and being able to cover more data patterns; at the same time, the training process is more stable, avoiding problems such as mode collapse, and improving the feasibility and reliability of the neural adversarial network in practical applications.
[0104] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have the same meaning as in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless defined similarly here.
[0105] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A neural adversarial network optimization method based on KAN generator replacement and acyclicity constraints, characterized in that: The steps include: Collecting a training data set and performing a preprocessing operation on the training data set; Constructing a KAN neural adversarial network and a SAM neural adversarial network, replacing the SAM generator of the SAM neural adversarial network with the KAN generator of the KAN neural adversarial network, and keeping the SAM discriminator of the SAM neural adversarial network unchanged; Setting an acyclic linear constraint term and introducing it into the loss function corresponding to the SAM neural adversarial network; Based on the preprocessed training data set, the SAM neural adversarial network is trained in an iterative update manner, and the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network are updated based on the loss function; The performance of the trained SAM neural adversarial network is evaluated, and the SAM neural adversarial network is adjusted and optimized according to the evaluation results.
2. The neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint according to claim 1, characterized in that: The pre-processing operation specifically includes: Cleaning the collected training data set, that is, removing fuzzy and repeated training data; Removing noise from the training data set based on a Gaussian filtering algorithm; Using a histogram equalization method to enhance the contrast of the training data set; The training data set is normalized.
3. The neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint according to claim 1, characterized in that: The KAN generator uses a B-spline activation function, and the mathematical formula of the B-spline activation function is: In the formula, B i (x) represents the value of the i-th B-spline activation function at the generated data point x; a i,k represents the activation coefficient of the i-th B-spline activation function at order k; yi represents the node position corresponding to the i-th B-spline activation function; n represents the total order of the i-th B-spline activation function.
4. The neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint according to claim 1, characterized in that: The setting of the acyclic linear constraint term and introducing it into the loss function corresponding to the SAM neural adversarial network specifically includes: Set the KAN generator to G, and map the random noise vector z to the generated data point x, that is, x=G(z); The acyclic linear constraint term represents a constraint on the distance between the generated data points x. The mathematical formula of the acyclic linear constraint term is: Where L represents the acyclic linear constraint; N represents the total number of generated data points; z p represents the pth random noise vector; z q represents the qth random noise vector; d(G(z p ), G(z q )) represents the generated data point G(z p ) and G(z q ) between them; δ represents the positive threshold corresponding to the acyclic linear constraint, which is used to control the minimum distance between generated data points; max() represents the maximum value operation.
5. The neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint according to claim 4, characterized in that: Introducing a regularization term into the loss function corresponding to the SAM neural adversarial network, and the regularization term is related to the gradient of the SAM discriminator of the SAM neural adversarial network; Set the SAM discriminator to D, input the generated data point x to the SAM discriminator D, and output the corresponding scalar value, i.e., the gradient penalty term. The mathematical formula of the gradient penalty term is: Where T represents the gradient penalty term; λ represents the regularization coefficient, which is used to control the intensity of the gradient penalty; Represents any sampling distribution between the training data distribution and the generated data distribution; Represents the sampling distribution The sample data points in ; Indicates that the SAM discriminator D is at the sampling data point The L2 norm of the gradient at .
6. The neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint according to claim 1, characterized in that: The method of training the SAM neural adversarial network in an iterative updating manner based on the preprocessed training data set and updating the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network based on the loss function specifically includes: Calculate the gradients of the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network based on the loss function; Based on the gradient, the Adam optimization algorithm is used to update the parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network. The update formula of the Adam optimization algorithm is as follows: m t =β1m t-1 +(1-β1)g t v t =β2v t-1 +(1-β2)g t 2 In the formula, m t and m t-1 Represents the first-order moment estimate of the gradient at the current time step t and the previous time step t-1 respectively; v t and v t-1 Represents the second-order moment estimate of the gradient at the current time step t and the previous time step t-1 respectively; g t represents the gradient of the current time step t; β1 and β2 represent the decay rates corresponding to the first-order moment estimate and the second-order moment estimate, respectively, which are used to control the deviations of the first-order moment estimate and the second-order moment estimate, respectively; η represents the learning rate, which is used to control the step size of parameter update; ε represents the stability constant of the Adam optimization algorithm; θ t and θ t-1 Represent the update parameters of the current time step t and the previous time step t-1 respectively; and They represent the first-order moment estimate and second-order moment estimate of the gradient of the current time step t after deviation correction, respectively.
7. The neural adversarial network optimization method based on KAN generator replacement and acyclicity constraint according to claim 1, characterized in that: The step of evaluating the performance of the trained SAM neural adversarial network and adjusting and optimizing the SAM neural adversarial network according to the evaluation result specifically includes: For the trained SAM neural adversarial network, evaluating the performance of the SAM neural adversarial network, and monitoring the change of the loss function of the SAM neural adversarial network, the quality of the generated data, and the accuracy of the SAM discriminator; The parameters of the KAN generator and the SAM discriminator of the SAM neural adversarial network are adjusted according to the monitoring results to optimize the SAM neural adversarial network.