Adaptive feedforward network optimization method based on feedback enhancement and customized large model

Abstract

The invention discloses an adaptive feedforward network optimization method based on feedback enhancement and a customized large model. The method comprises the following steps: initializing parameters of a feedforward network of the model; inputting the characteristic matrix of the feed-forward network, and calculating the output of each layer of the feed-forward network in a forward propagation mode; calculating an error from the output of the feed-forward network and the true tag, the error being used to adjust feed-forward network parameters and generate a feedback signal; generating a feedback signal for adjusting a learning rate adjustment parameter according to the size and the change trend of the error; according to the intensity of the feedback signal, dynamically adjusting the adjustment rule of the weight and the bias; and adjusting the intensity of the feedback signal according to the effect of the current iteration. According to the method, the optimization speed and direction can be automatically adjusted, the problems of gradient disappearance, local optimal solution and training oscillation are effectively avoided, and the convergence speed and stability of the model are improved, especially when complex and nonlinear data are trained.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a feedback-enhanced adaptive feedforward network optimization method and a customized large model. Background Art

[0002] At present, there are still many problems in intelligent dialogue systems and related technical fields, which make it difficult to meet the increasingly complex user needs and application scenarios. With the rapid development of artificial intelligence technology, intelligent dialogue systems are widely used in customer service, education, medical care, finance and other fields. For example, the digitalization and intelligentization process in the field of industrial equipment continues to advance, and the automated operation, remote monitoring and fault diagnosis of equipment have become industry trends. However, the data of industrial equipment such as operation manuals, maintenance records, fault logs, etc. are numerous and complex, involving not only real-time data of machine operation, but also a large amount of natural language text processing, such as equipment fault reports, technical documents, etc. How to make the model effectively understand this data and provide intelligent decision support and fault solutions has become a major challenge in current technology.

[0003] However, the traditional neural network training process is susceptible to problems such as gradient vanishing and gradient exploding, which leads to unstable training, slow convergence, and easy to fall into local optimal solutions. The existing technology lacks effective optimization algorithms to dynamically adjust the optimization direction and speed, and is prone to oscillation and learning rate selection problems commonly seen in traditional gradient descent methods. In addition, traditional large models are usually general-purpose and perform poorly in specific industry applications. The existing technology lacks specialized customized large models in the field of industrial equipment, resulting in a weak understanding of professional knowledge in this field and difficulty in adapting to the natural language processing needs related to industrial equipment.

[0004] In view of this, it is necessary to provide a feedback-enhanced adaptive feedforward network optimization method and a customized large model to overcome the above-mentioned defects. Summary of the invention

[0005] The purpose of the present invention is to provide a feedback-enhanced adaptive feedforward network optimization method and a customized large model, aiming to solve the problems of unstable training, slow convergence, and easy falling into local optimal solutions in the traditional neural network training process, and avoid the common oscillation and learning rate selection problems in the traditional gradient descent method.

[0006] In order to achieve the above object, the present invention provides a feedback-enhanced adaptive feedforward network optimization method, comprising:

[0007] Step S11: Initialize the parameters of the feedforward network of the Transformer network model, including weights and biases;

[0008] Step S12: input the feature matrix of the feedforward network, and calculate the output of each layer of the feedforward network by forward propagation;

[0009] Step S13: Calculate the error based on the output of the feedforward network and the true label, and the error is used to adjust the feedforward network parameters and generate a feedback signal;

[0010] Step S14: generating a feedback signal according to the size and change trend of the error to characterize the direction and strength of optimization, wherein the generation of the feedback signal includes an evaluation of the error change rate and is used to adjust the learning rate adjustment parameter;

[0011] Step S15: dynamically adjusting the weight and bias adjustment rules according to the strength of the feedback signal; wherein, if the feedback signal indicates that the error is reduced but the adjustment speed is slowed down, increasing the adjustment amplitude to accelerate convergence;

[0012] Step S16: adjusting the strength of the feedback signal according to the effect of the current iteration. If the error reduction effect is not obvious or the error increases after multiple consecutive iterations, reducing the feedback strength to avoid over-adjustment;

[0013] Step S17: Repeat the above steps S12-S16 until the preset stop iteration condition is met, indicating that the Transformer network model training is completed.

[0014] In a preferred embodiment, in step S11, the initialization method is expressed as:

[0015]

[0016] Where W p is the weight of the feedforward network, b p is the bias of the feedforward network, W p,i is the i-th weight parameter of the feedforward network, b p,i is the i-th bias parameter of the feedforward network, ~ indicates that it obeys a specific distribution; N() indicates a normal distribution; σ p represents the standard deviation of the feedforward network when it is initialized.

[0017] In a preferred embodiment, in step S12, the propagation mode through each layer of the feedforward network is expressed as:

[0018] Z p =f p (W p X p +b p ),

[0019] In the formula, let X p is the input feedforward network feature matrix, matrix Z pis the output of the feedforward network, f p () is the activation function of the feedforward network;

[0020] Among them, let x be the input of the function, and the calculation method of the activation function of the feedforward network is expressed as:

[0021]

[0022] In the formula, γ p is the adaptive control parameter.

[0023] In a preferred embodiment, in step S13, the error function used to calculate the error adopts the cross entropy with a dynamic adjustment term, and the calculation method is expressed as:

[0024]

[0025] In the formula, is the predicted output of the feedforward network, y is the true label; p is the dynamic adjustment factor; E p is the training error of the feedforward network.

[0026] In a preferred embodiment, as the number of iterations increases, the penalty term is gradually increased to prevent overfitting. The calculation method of the dynamic adjustment factor is expressed as:

[0027] λ p =λ0(1-exp(-k p int(t))),

[0028] Where λ0 is the initial adjustment strength, k p is the rate constant of adjustment, int(t) is the current iteration number.

[0029] In a preferred embodiment, in step S14, the generation of the feedback signal includes an evaluation of the error change rate, defined as ΔE p , then its generation method is expressed as:

[0030] ΔE p =E p -E p,prev ,

[0031]

[0032] In the formula, E p,prev is the training error of the feedforward network in the previous iteration; E p,prev2 is the training error of the feedforward network in the first two iterations; is the second-order change in the error of the feedforward network; β p is the feedback sensitivity parameter; α pis the learning rate adjustment parameter, ← is the parameter update operation; γ p is the sensitivity adjustment parameter.

[0033] In a preferred embodiment, in step S15, the method of adjusting the parameters of the feedforward network based on the adaptive method of error feedback is expressed as follows:

[0034]

[0035] In the formula, f adjust () is the adjustment function.

[0036] In a preferred embodiment, assuming x as the input of the function, the calculation method of the adjustment function is expressed as:

[0037]

[0038] In the formula, tr p is the adjustment function impact factor; x max is the preset maximum value of x; the adjustment function can provide the largest adjustment amplitude when x is close to its maximum value to increase the adaptability of the model.

[0039] In a preferred embodiment, in step S16, the feedback intensity is adjusted based on the long-term memory factor θ p , the long-term memory factor records and adjusts the feedback strength to maintain the stability of learning, and is based on the nonlinear decay function of historical performance to improve the flexibility of adjustment. Its update method is expressed as:

[0040]

[0041] In the formula, ρ p is the memory decay factor; cs is the decay rate of the historical error effect; mes is the number of historical iterations considered; Represents the training error of the j-th feedforward network.

[0042] The present invention also provides a customized large model, which is a model based on a Transformer network, including an encoder and a decoder; wherein the encoder is used to map input features into a series of context-aware vector representations, and the decoder is used to gradually decode the vector representations into target features;

[0043] The encoder includes N identical encoding layers, each encoding layer includes two sublayers, a multi-head attention mechanism sublayer and a feedforward network sublayer, and the encoder adds a residual connection and a normalization operation after each sublayer;

[0044] The decoder includes N identical decoding layers, each decoding layer includes two multi-head attention mechanism sublayers and one feedforward network sublayer, and the decoder adds residual connections and normalization operations after each sublayer;

[0045] The feedforward networks of the feedforward network sublayers are all trained using the feedback-enhanced adaptive feedforward network optimization method as described in any of the above implementations.

[0046] The feedback-enhanced adaptive feedforward network optimization method and customized large model provided by the present invention are inspired by the ecosystem feedback mechanism in nature, especially the negative feedback loops in the ecosystem, which can maintain the dynamic balance of the system. By simulating this negative feedback mechanism, the feedback-enhanced adaptive feedforward network adjustment algorithm can automatically adjust the optimization speed and direction when adjusting the weights and biases of the feedforward network, thereby avoiding the common oscillation and learning rate selection problems in the traditional gradient descent method. Therefore, by simulating the negative feedback mechanism in the ecosystem and dynamically adjusting the optimization direction and speed of the model, the gradient disappearance, local optimal solution and training oscillation problems are effectively avoided, and the convergence speed and stability of the model are improved, especially when training complex and nonlinear data. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A flowchart of the feedback-enhanced adaptive feedforward network optimization method;

[0049] Figure 2 This is the architecture diagram for a custom large model. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and beneficial technical effect of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described in this specification are only for explaining the present invention, not for limiting the present invention.

[0051] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0052] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] In an embodiment of the present invention, a feedback-enhanced adaptive feedforward network optimization method is provided, which dynamically adjusts the optimization direction and speed of the model by simulating the negative feedback mechanism in the ecosystem, thereby improving the convergence speed and stability of the Transformer network model during training, especially when training complex and nonlinear data.

[0054] like Figure 1 As shown, the feedback-enhanced adaptive feedforward network optimization method includes steps S11-S17.

[0055] Step S11: Initialize the parameters of the feedforward network of the Transformer network model, including weights and biases.

[0056] Specifically, the initialization method is expressed as:

[0057]

[0058] Where W p is the weight of the feedforward network, b p is the bias of the feedforward network, W p,i is the i-th weight parameter of the feedforward network, b p,i is the i-th bias parameter of the feedforward network, ~ indicates that it obeys a specific distribution; N() indicates a normal distribution; σ p represents the standard deviation of the feedforward network when it is initialized. Preferably, σ p Set to 0.05.

[0059] Step S12: Input the feature matrix of the feedforward network, and calculate the output of each layer of the feedforward network by forward propagation.

[0060] Specifically, the propagation through each layer of the feedforward network is expressed as:

[0061] Z p =f p (W p X p +b p ),

[0062] In the formula, let X p is the input feedforward network feature matrix (the existing feature matrix can be used), Z p is the output of the feedforward network, f p () is the activation function of the feedforward network.

[0063] Among them, let x be the input of the function, and the calculation method of the activation function of the feedforward network is expressed as:

[0064]

[0065] In the formula, γ p is an adaptive control parameter. Preferably, γ p Set to 2.

[0066] Step S13: Calculate the error based on the output of the feedforward network and the true label, and the error is used to adjust the feedforward network parameters and generate a feedback signal.

[0067] In one embodiment, the error function used to calculate the error adopts the cross entropy with a dynamic adjustment term, and the calculation method is expressed as:

[0068]

[0069] In the formula, is the predicted output of the feedforward network, y is the true label; p is the dynamic adjustment factor; E p is the training error of the feedforward network.

[0070] Furthermore, as the number of iterations increases, the penalty term is gradually strengthened to prevent overfitting. The calculation method of the dynamic adjustment factor is expressed as:

[0071] λ p =λ0(1-exp(-k p int(t))),

[0072] Where λ0 is the initial adjustment strength, k p is the rate constant of adjustment, int(t) is the current iteration number.

[0073] Step S14: Based on the size and change trend of the error, a feedback signal is generated to characterize the direction and strength of the optimization, wherein the generation of the feedback signal includes an evaluation of the error change rate and is used to adjust the learning rate adjustment parameter. Therefore, the generation of the feedback signal not only considers the error change rate, but also includes the second-order derivative of the error to capture the acceleration of the change and enhance the model's sensitivity to the error change.

[0074] Specifically, the generation of the feedback signal includes the evaluation of the rate of change of the error, defined as ΔE p , then its generation method is expressed as:

[0075] ΔE p =E p -E p,prev ,

[0076]

[0077] In the formula, E p,prev is the training error of the feedforward network in the previous iteration; E p,prev2 is the training error of the feedforward network in the first two iterations; is the second-order change in the error of the feedforward network; β p is the feedback sensitivity parameter; α p is the learning rate adjustment parameter, ← is the parameter update operation; γ p is a sensitivity adjustment parameter. Preferably, β p Set to 0.1.

[0078] Step S15: dynamically adjusting the weight and bias adjustment rules according to the strength of the feedback signal; wherein, if the feedback signal indicates that the error is reduced but the adjustment speed is slowed down (for example, the adjustment speed is less than a threshold), the adjustment amplitude is increased to speed up convergence.

[0079] Specifically, the method of adjusting the parameters of the feedforward network based on the adaptive method of error feedback is expressed as:

[0080]

[0081] In the formula, f adjust () is the adjustment function.

[0082] Furthermore, let x be the input of the function, and the calculation method of the adjustment function can be expressed as:

[0083]

[0084] In the formula, tr p is the adjustment function impact factor; x max is the preset maximum value of x; the adjustment function can provide the largest adjustment amplitude when x is close to its maximum value to increase the adaptability of the model.

[0085] Step S16: Adjust the strength of the feedback signal according to the effect of the current iteration. If the error reduction effect is not obvious after multiple consecutive iterations (for example, the error reduction amplitude is less than the threshold) or the error increases, reduce the feedback strength to avoid over-adjustment.

[0086] Specifically, the feedback strength is adjusted based on the long-term memory factor θ p , the long-term memory factor records and adjusts the feedback strength to maintain the stability of learning, and is based on the nonlinear decay function of historical performance to improve the flexibility of adjustment. Its update method is expressed as:

[0087]

[0088] In the formula, ρp is the memory decay factor; cs is the decay rate of the historical error, which makes the update of the memory factor not only depend on the current error, but also affected by the previous error performance, improving the adaptability of the model to long-term trends; mes is the number of historical iterations considered; represents the training error of the jth feedforward network. Preferably, ρ p Set to 0.9.

[0089] Step S17: Repeat the above steps S12-S16 until the preset stop iteration condition is met, indicating that the Transformer network model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0090] The present invention also provides a customized large model. Figure 2 As shown in FIG. 1 , the customized large model is a Transformer network-based model, including an encoder and a decoder. The encoder is used to map the input features into a series of context-aware vector representations, and the decoder is used to gradually decode the vector representations into target features.

[0091] The encoder consists of N identical encoding layers, each of which includes two sublayers: a multi-head attention mechanism sublayer and a feedforward network sublayer, and the encoder adds residual connections and normalization operations after each sublayer.

[0092] The decoder consists of N identical decoding layers, each of which consists of two multi-head attention mechanism sublayers and one feedforward network sublayer, and the decoder adds residual connections and normalization operations after each sublayer.

[0093] It should be noted that the multi-head attention mechanism sublayer is implemented by a multi-head self-attention mechanism, which is the core module of the encoder and is responsible for learning the importance of each position from the input features and combining the information of all positions to obtain a global context representation, thereby improving the performance and generalization ability of the model. Specifically, the multi-head self-attention mechanism is implemented in that the input features pass through three different linear transformation layers to generate "query", "key" and "value" vectors respectively, and the linear transformation is implemented by multiplying the weight matrix. Further, the generated query, key and value vectors are divided into multiple smaller parts, namely "heads", so that the model processes multiple different representation subspaces in parallel, thereby capturing the diversity information in the input data. Further, for each head, the dot product of the query and key vectors is calculated respectively to obtain an attention score, which reflects the relative importance of different positions in the sequence of input data. In order to make the numerical stability of the attention score, the attention score is divided by a scaling factor, which can be the square root of the dimension of the key vector.

[0094] Furthermore, in both the encoder and decoder, the output of each sub-layer is the result of layer normalization and residual connection, which is expressed as:

[0095] y cgy =LN c (x trans +Sublayer(x trans )),

[0096] In the formula, LN c ( ) is the layer normalization function, Sublayer( ) represents the output of the residual connected sublayer, x trans Represents the input of the sublayer; y cgy is the feature output after layer normalization.

[0097] The feedforward network sublayer consists of two fully connected layers and an activation function, which is used to perform nonlinear transformation on the vector at each position. In this sublayer, the input vector first passes through the first fully connected layer for linear transformation, then passes through a ReLU activation function for nonlinear transformation, and finally passes through the second fully connected layer for linear transformation to improve the expressiveness and generalization ability of the model, so that the model can better handle complex input data.

[0098] It should be noted that the feedforward network of the feedforward network sublayer is a standard network, which is trained using the feedback-enhanced adaptive feedforward network optimization method as described in any of the above implementations.

[0099] In this embodiment, the customized large model is a natural language large model in the field of industrial equipment. The training data for training based on the feedback-enhanced adaptive feedforward network optimization method includes:

[0100] (1) Equipment operation manual and technical documentation: Detailed records of equipment use, maintenance, and technical parameters help the model understand equipment operation specifications, extract technical key points, and implement automated document parsing and troubleshooting suggestions;

[0101] (2) Maintenance records and repair reports: These contain descriptions of equipment failures, repair procedures, and spare parts used, from which common failure modes can be extracted and maintenance needs can be predicted;

[0102] (3) Fault log and alarm information: short text or structured data records abnormalities in equipment operation, which can be used for fault mode identification and alarm optimization;

[0103] (4) Industry standards and regulatory documents: including compliance requirements for equipment operation. By parsing regulatory documents, it helps companies ensure that equipment operations comply with regulations.

[0104] Therefore, the natural language large model in the field of industrial equipment has a stronger understanding of the professional knowledge in the field of industrial equipment, and is thus more adaptable to the natural language processing needs related to industrial equipment.

[0105] In summary, the feedback-enhanced adaptive feedforward network optimization method and customized large model provided by the present invention are inspired by the ecosystem feedback mechanism in nature, especially the negative feedback loops in the ecosystem, which can maintain the dynamic balance of the system. By simulating this negative feedback mechanism, the feedback-enhanced adaptive feedforward network adjustment algorithm can automatically adjust the optimization speed and direction when adjusting the weights and biases of the feedforward network, thereby avoiding the common oscillation and learning rate selection problems in the traditional gradient descent method. Therefore, by simulating the negative feedback mechanism in the ecosystem, the optimization direction and speed of the model are dynamically adjusted, which effectively avoids the problems of gradient vanishing, local optimal solution and training oscillation, and improves the convergence speed and stability of the model, especially when training complex and nonlinear data.

[0106] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0107] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0109] In the embodiments provided by the present invention, it should be understood that the disclosed systems or devices / terminal equipment and methods can be implemented in other ways. For example, the system or device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system or unit can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0112] The present invention is not limited to what is described in the specification and implementation modes, and therefore additional advantages and modifications can be easily realized by those skilled in the art. Therefore, without departing from the spirit and scope of the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details, representative devices, and illustrative examples shown and described herein.

Claims

1. A feedback-enhanced adaptive feedforward network optimization method, characterized in that: include: Step S11: Initialize the parameters of the feedforward network of the Transformer network model, including weights and biases; Step S12: input the feature matrix of the feedforward network, and calculate the output of each layer of the feedforward network by forward propagation; Step S13: Calculate the error based on the output of the feedforward network and the true label, and the error is used to adjust the feedforward network parameters and generate a feedback signal; Step S14: generating a feedback signal according to the size and change trend of the error to characterize the direction and strength of optimization, wherein the generation of the feedback signal includes an evaluation of the error change rate and is used to adjust the learning rate adjustment parameter; Step S15: dynamically adjusting the weight and bias adjustment rules according to the strength of the feedback signal; wherein, if the feedback signal indicates that the error is reduced but the adjustment speed is slowed down, increasing the adjustment amplitude to accelerate convergence; Step S16: adjusting the strength of the feedback signal according to the effect of the current iteration. If the error reduction effect is not obvious or the error increases after multiple consecutive iterations, reducing the feedback strength to avoid over-adjustment; Step S17: Repeat the above steps S12-S16 until the preset stop iteration condition is met, indicating that the Transformer network model training is completed.

2. The feedback-enhanced adaptive feedforward network optimization method according to claim 1, characterized in that: In step S11, the initialization method is expressed as: Where W p is the weight of the feedforward network, b p is the bias of the feedforward network, W p,i is the i-th weight parameter of the feedforward network, b p,i is the i-th bias parameter of the feedforward network, ~ indicates that it obeys a specific distribution; N() indicates a normal distribution; σ p represents the standard deviation of the feedforward network when it is initialized.

3. The feedback-enhanced adaptive feedforward network optimization method according to claim 2, characterized in that: In step S12, the propagation mode through each layer of the feedforward network is expressed as: Z p =f p (W p X p +b p ), In the formula, let X p is the input feedforward network feature matrix, Z p is the output of the feedforward network, f p () is the activation function of the feedforward network; Among them, let x be the input of the function, and the calculation method of the activation function of the feedforward network is expressed as: In the formula, γ p is an adaptive control parameter.

4. The feedback-enhanced adaptive feedforward network optimization method according to claim 3, characterized in that: In step S13, the error function used to calculate the error adopts the cross entropy with a dynamic adjustment term, and the calculation method is expressed as: In the formula, is the predicted output of the feedforward network, and y is the true label; λ p is the dynamic adjustment factor; E p is the training error of the feedforward network.

5. The feedback-enhanced adaptive feedforward network optimization method according to claim 4, characterized in that: As the number of iterations increases, the penalty term gradually increases to prevent overfitting. The calculation method of the dynamic adjustment factor is expressed as: λ p =λ0(1-exp(-k p ·int(t))), Where λ0 is the initial adjustment strength, k p is the rate constant of adjustment, int(t) is the current iteration number.

6. The feedback-enhanced adaptive feedforward network optimization method according to claim 5, characterized in that: In step S14, the generation of the feedback signal includes an evaluation of the rate of change of the error, defined as ΔE p , then its generation method is expressed as: ΔE p =E p -E p,prev , In the formula, E p,prev is the training error of the feedforward network in the previous iteration; E p,prev2 is the training error of the feedforward network in the first two iterations; is the second-order change in the error of the feedforward network; β p is the feedback sensitivity parameter; α p is the learning rate adjustment parameter, ← is the parameter update operation; γ p is the sensitivity adjustment parameter.

7. The feedback-enhanced adaptive feedforward network optimization method according to claim 6, characterized in that: In step S15, the method of adjusting the parameters of the feedforward network based on the adaptive method of error feedback is expressed as: In the formula, f adjust () is the adjustment function.

8. The feedback-enhanced adaptive feedforward network optimization method according to claim 7, characterized in that: Let x be the input of the function, and the calculation method of the adjustment function can be expressed as: In the formula, tr p is the adjustment function impact factor; x max is the preset maximum value of x; the adjustment function can provide the largest adjustment amplitude when x is close to its maximum value to increase the adaptability of the model.

9. The feedback-enhanced adaptive feedforward network optimization method according to claim 8, characterized in that: In step S16, the feedback intensity is adjusted based on the long-term memory factor θ p , the long-term memory factor records and adjusts the feedback strength to maintain the stability of learning, and is based on the nonlinear decay function of historical performance to improve the flexibility of adjustment. Its update method is expressed as: In the formula, ρ p is the memory decay factor; ζ cs is the decay rate of the historical error effect; mes is the number of historical iterations considered; Represents the training error of the j-th feedforward network.

10. A customized large model, characterized in that: The customized large model is a model based on a Transformer network, including an encoder and a decoder; wherein the encoder is used to map input features into a series of context-aware vector representations, and the decoder is used to gradually decode the vector representations into target features; The encoder includes N identical encoding layers, each encoding layer includes two sublayers, a multi-head attention mechanism sublayer and a feedforward network sublayer, and the encoder adds a residual connection and a normalization operation after each sublayer; The decoder includes N identical decoding layers, each decoding layer includes two multi-head attention mechanism sublayers and one feedforward network sublayer, and the decoder adds residual connections and normalization operations after each sublayer; The feedforward networks of the feedforward network sublayer are all trained using the feedback-enhanced adaptive feedforward network optimization method as described in any one of claims 1 to 9.

Images