Natural language data processing method based on semantic feedback loop and recursive optimization

Abstract

The invention relates to natural language processing, in particular to a natural language data processing method based on semantic feedback loop and recursive optimization, which comprises the following steps: generating an input vector according to a text feature vector input by a user; performing initialization processing on the input vector by using the initialization model to obtain an output vector; performing semantic evaluation on the output vector to generate a semantic feedback vector; the input vector is optimized and adjusted according to the semantic feedback vector and the learning rate; s2, judging whether a preset iteration termination condition is met, if the preset iteration termination condition is met, outputting a final result, and otherwise, returning to S2; according to the technical scheme provided by the invention, the defect that the convergence speed is difficult to effectively improve while the accuracy is ensured in the prior art can be effectively overcome.

Description

Technical Field

[0001] The present invention relates to natural language processing, and in particular to a natural language data processing method based on semantic feedback loop and recursive optimization. Background Art

[0002] With the development of science and technology and the continuous advancement of artificial intelligence technology, the ability to process natural language is also increasing. However, there are still disadvantages such as high computational complexity, slow convergence, sensitivity to initial conditions and difficulty in processing noisy data. These disadvantages are particularly prominent when facing complex problems and large-scale data, which limits the widespread application of the algorithm.

[0003] When a program processes natural language, due to the complexity, diversity and possible ambiguity of natural language, the program will go through multiple iterations. Especially when faced with complex, multi-dimensional optimization problems, the algorithm may gradually slow down in the process of approaching the optimal solution, resulting in slow convergence.

[0004] In the prior art, when faced with complex, multi-dimensional optimization problems, the following technical solutions are mainly used:

[0005] A. Evolutionary Algorithms

[0006] The problem solution is gradually optimized by simulating the process of natural selection and biological evolution. A large number of candidate solutions are generated and selected, crossed, and mutated to find the optimal solution. Since a large number of candidate solutions need to be processed, the computing resource requirements are high. At the same time, the algorithm relies on a large number of iterations for progressive optimization, resulting in a slow convergence speed. If the population diversity is insufficient, it is easy to fall into the local optimal solution.

[0007] B. Simulated Annealing

[0008] It imitates the physical annealing process, exploring possible solutions by searching the solution space extensively in the early stage, and then converges to the optimal solution as the temperature gradually decreases. This algorithm is very sensitive to temperature parameters. Incorrect temperature control may lead to inefficient algorithms or trapping in local optimal solutions. At the same time, due to its random search characteristics, it converges slowly in complex problems.

[0009] C. Monte Carlo Tree Search (MCTS)

[0010] By performing a large number of random simulations at each decision point to predict possible outcome paths, the decision-making process is optimized. Although this method can find better paths in complex strategy games and decision-making processes, its reliance on a large number of simulations leads to high computational complexity, especially in large-scale decision trees. In addition, due to the presence of randomness, the convergence performance is relatively unstable. Summary of the invention

[0011] 1. Technical issues to be resolved

[0012] In view of the above-mentioned shortcomings of the prior art, the present invention provides a natural language data processing method based on semantic feedback loop and recursive optimization, which can effectively overcome the defects of the prior art that it is difficult to effectively improve the convergence speed while ensuring accuracy.

[0013] (II) Technical solution

[0014] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0015] The method for processing natural language data based on semantic feedback loop and recursive optimization includes the following steps:

[0016] S1, generating an input vector according to a text feature vector input by a user;

[0017] S2, using the initialization model to initialize the input vector to obtain an output vector;

[0018] S3, evaluate the output vector at the semantic level and generate a semantic feedback vector;

[0019] S4, optimizing and adjusting the input vector according to the semantic feedback vector and the learning rate;

[0020] S5. Determine whether the preset iteration termination condition is met. If the preset iteration termination condition is met, output the final result, otherwise return to S2.

[0021] Preferably, generating an input vector according to the text feature vector input by the user in S1 includes:

[0022] The input vector is generated based on the text feature vector input by the user using the following formula:

[0023] X 1 =f inint (p);

[0024] Among them, p is the text feature vector input by the user, X 1 is the input vector of the first iteration, with dimension n, f inint For the initialization function.

[0025] Preferably, in S2, the input vector is initialized using the initialization model to obtain the output vector, including:

[0026] The input vector is initialized using the initialization model, and the output vector is expressed as follows:

[0027] Y t =M(X t );

[0028] Among them, X t is the input vector of the tth iteration, Y t is the output vector of the tth iteration, the output vector Y t With the input vector X t The dimensions are the same or different, M is the initialization model, which is used to convert the input vector X t Mapped to the output vector Y t , t is the number of iterations.

[0029] Preferably, in S3, the output vector is evaluated at a semantic level to generate a semantic feedback vector, including:

[0030] The output vector is evaluated at the semantic level, and the generated semantic feedback vector is expressed as follows:

[0031] S t =F feedback (Y t ,Y target );

[0032] Among them, S t is the semantic feedback vector of the tth iteration, Y target is the target output vector, F feedback is the semantic feedback evaluation function, used to calculate the output vector Y t With the target output vector Y target The difference between.

[0033] Preferably, in S4, optimizing and adjusting the input vector according to the semantic feedback vector and the learning rate includes:

[0034] The optimization adjustment of the input vector according to the semantic feedback vector and the learning rate is expressed as follows:

[0035]

[0036] Among them, X t+1 is the input vector of the t+1th iteration, η is the learning rate, which is used to control the optimization adjustment step size, η 0 is the initial learning rate, λ is the learning rate decay constant, and λ·t is used to control the degree to which the learning rate η decays as the number of iterations t increases. is the semantic feedback vector S t For the input vector X t The gradient of , represents the semantic feedback vector S t The sensitivity with respect to each input variable.

[0037] Preferably, in S5, it is determined whether a preset iteration termination condition is met, and if so, the final result is output, otherwise, the process returns to S2, including:

[0038] When the semantic feedback vector S t When the rate of change of satisfies the convergence condition, or reaches the preset number of iterations, the preset iteration termination condition is met and the final result is output, otherwise it returns to S2;

[0039] The convergence condition is expressed as follows:

[0040]

[0041] In the above formula, S t-1 is the semantic feedback vector of the t-1th iteration, |·| represents the calculation modulus, ε is a constant close to 0, which is used to define the convergence threshold of the algorithm, and α is an adjustment parameter used to adjust the convergence threshold as the number of iterations t increases.

[0042] (III) Beneficial effects

[0043] Compared with the prior art, the natural language data processing method based on semantic feedback loop and recursive optimization provided by the present invention can significantly improve the convergence speed by dynamically adjusting the learning rate, so that the algorithm can flexibly adapt to changes in different iteration stages, avoiding excessive or excessively small control optimization adjustment step sizes affecting the convergence speed. At the same time, dynamically judging whether to stop iteration based on the rate of change or cumulative change of the semantic feedback vector can not only avoid invalid calculations, but also effectively improve the convergence efficiency of the algorithm. The combination of these two methods can enable the algorithm to quickly approach the optimal solution while ensuring accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Natural language data processing methods based on semantic feedback loops and recursive optimization, such as Figure 1 As shown, S1, generating an input vector according to the text feature vector input by the user, specifically including:

[0048] The input vector is generated based on the text feature vector input by the user using the following formula:

[0049] X 1 =f inint (p);

[0050] Among them, p is the text feature vector input by the user, X 1 is the input vector of the first iteration, with dimension n, f inint For the initialization function.

[0051] S2. Initialize the input vector using the initialization model to obtain an output vector, specifically including:

[0052] The input vector is initialized using the initialization model, and the output vector is expressed as follows:

[0053] Y t =M(X t );

[0054] Among them, X t is the input vector of the tth iteration, Y t is the output vector of the tth iteration, the output vector Y t With the input vector X t The dimensions are the same or different, M is the initialization model, which is used to convert the input vector X t Mapped to the output vector Y t , t is the number of iterations.

[0055] S3. Evaluate the output vector at the semantic level and generate a semantic feedback vector, including:

[0056] The output vector is evaluated at the semantic level, and the generated semantic feedback vector is expressed as follows:

[0057] S t =F feedback (Y t,Y target );

[0058] Among them, S t is the semantic feedback vector of the tth iteration, Y target is the target output vector, F feedback is the semantic feedback evaluation function, used to calculate the output vector Y t With the target output vector Y target The difference between.

[0059] S4. Optimize and adjust the input vector according to the semantic feedback vector and the learning rate, including:

[0060] The optimization adjustment of the input vector according to the semantic feedback vector and the learning rate is expressed as follows:

[0061]

[0062] Among them, X t+1 is the input vector of the t+1th iteration, η is the learning rate, which is used to control the optimization adjustment step size, η 0 is the initial learning rate, λ is the learning rate decay constant, and λ·t is used to control the degree to which the learning rate η decays as the number of iterations t increases. is the semantic feedback vector S t For the input vector X t The gradient of , represents the semantic feedback vector S t The sensitivity with respect to each input variable.

[0063] In the above technical solution, λ·t is used to control the degree of attenuation of the learning rate η as the number of iterations t increases. The learning rate η is gradually reduced as the number of iterations t increases. A larger learning rate η is used at the beginning of the iteration to quickly approach the optimal solution. Then, fine-tuning is performed by gradually reducing the learning rate η to avoid premature stagnation.

[0064] S5: Determine whether the preset iteration termination condition is met. If the preset iteration termination condition is met, output the final result. Otherwise, return to S2. Specifically, it includes:

[0065] When the semantic feedback vector S t When the rate of change of satisfies the convergence condition, or reaches the preset number of iterations, the preset iteration termination condition is met and the final result is output, otherwise it returns to S2;

[0066] The convergence condition is expressed as follows:

[0067]

[0068] In the above formula, S t-1is the semantic feedback vector of the t-1th iteration, |·| represents the calculation modulus, ε is a constant close to 0, which is used to define the convergence threshold of the algorithm, and α is an adjustment parameter used to adjust the convergence threshold as the number of iterations t increases.

[0069] In the technical solution of the present application, the convergence speed can be significantly improved by dynamically adjusting the learning rate, so that the algorithm can flexibly adapt to changes in different iteration stages, avoiding excessive or small control optimization adjustment step sizes affecting the convergence speed. At the same time, dynamically judging whether to stop iteration is based on the rate of change or cumulative change of the semantic feedback vector, which can not only avoid invalid calculations, but also effectively improve the convergence efficiency of the algorithm. The combination of these two methods can enable the algorithm to quickly approach the optimal solution while ensuring accuracy.

[0070] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A natural language data processing method based on semantic feedback loop and recursive optimization, characterized in that: The following steps are involved: S1, generating an input vector according to a text feature vector input by a user; S2, using the initialization model to initialize the input vector to obtain an output vector; S3, evaluate the output vector at the semantic level and generate a semantic feedback vector; S4, optimizing and adjusting the input vector according to the semantic feedback vector and the learning rate; S5. Determine whether the preset iteration termination condition is met. If the preset iteration termination condition is met, output the final result, otherwise return to S2.

2. The method for natural language data processing based on semantic feedback loop and recursive optimization according to claim 1, characterized in that: In S1, an input vector is generated according to the text feature vector input by the user, including: The input vector is generated based on the text feature vector input by the user using the following formula: X1=f inint (p); Among them, p is the text feature vector input by the user, X1 is the input vector of the first iteration, the dimension is n, and f inint For the initialization function.

3. The method for natural language data processing based on semantic feedback loop and recursive optimization according to claim 2, characterized in that: In S2, the input vector is initialized using the initialization model to obtain an output vector, including: The input vector is initialized using the initialization model, and the output vector is expressed as follows: Y t =M(X t ); Among them, X t is the input vector of the tth iteration, Y t is the output vector of the tth iteration, the output vector Y t With the input vector X t The dimensions are the same or different, M is the initialization model, which is used to convert the input vector X t Mapped to the output vector Y t , t is the number of iterations.

4. The method for natural language data processing based on semantic feedback loop and recursive optimization according to claim 3, characterized in that: S3 evaluates the output vector at the semantic level and generates a semantic feedback vector, including: The output vector is evaluated at the semantic level, and the generated semantic feedback vector is expressed as follows: S t =F feedback (Y t ,Y target ); Among them, S t is the semantic feedback vector of the tth iteration, Y target is the target output vector, F feedback is the semantic feedback evaluation function, used to calculate the output vector Y t With the target output vector Y target The difference between.

5. The method for natural language data processing based on semantic feedback loop and recursive optimization according to claim 4, characterized in that: In S4, the input vector is optimized and adjusted according to the semantic feedback vector and the learning rate, including: The optimization adjustment of the input vector according to the semantic feedback vector and the learning rate is expressed as follows: Among them, X t+1 is the input vector of the t+1th iteration, η is the learning rate, which is used to control the optimization adjustment step size, η0 is the initial learning rate, λ is the learning rate decay constant, and λ·t is used to control the degree to which the learning rate η decays as the number of iterations t increases. is the semantic feedback vector S t For the input vector X t The gradient of , represents the semantic feedback vector S t The sensitivity with respect to each input variable.

6. The method for natural language data processing based on semantic feedback loop and recursive optimization according to claim 5, characterized in that: In S5, it is determined whether the preset iteration termination condition is met. If the preset iteration termination condition is met, the final result is output, otherwise, it returns to S2, including: When the semantic feedback vector S t When the rate of change of satisfies the convergence condition, or reaches the preset number of iterations, the preset iteration termination condition is met and the final result is output, otherwise it returns to S2; The convergence condition is expressed as follows: In the above formula, S t-1 is the semantic feedback vector of the t-1th iteration, |·| represents the calculation modulus, ε is a constant close to 0, which is used to define the convergence threshold of the algorithm, and α is an adjustment parameter used to adjust the convergence threshold as the number of iterations t increases.

Images