Natural language knowledge extraction method and device based on large and small model collaboration

Abstract

The present invention provides a natural language knowledge extraction method and device based on the collaboration of large and small models, which relates to the field of data processing technology. After converting a natural language text file into a vector format, it is input into a pre-constructed small model, and a recognition result is output, thereby determining the event indicated by the text file. Among them, the small model optimizes the parameters of the neural network based on the circular energy flow optimization algorithm, which enhances the adaptability and stability of the network at different training stages. In addition, it is constructed using an extreme learning machine classification algorithm based on a dynamic weight adjustment mechanism to improve the accuracy of classification. The training sample set for training the small model is constructed after event extraction from the initial sample set by a large natural language model. The present invention uses a large model to perform preliminary event extraction and refined training of the small model, which can efficiently process a large amount of data while maintaining high accuracy, especially in the extraction of specific events and entities.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a natural language knowledge extraction method and device based on large and small model collaboration. Background Art

[0002] With the rapid development of information technology, a large amount of text data has been generated on the Internet, including news reports, academic papers, blogs, and social media content. These data contain rich information and knowledge, and are of great value for applications such as information retrieval, data mining, sentiment analysis, and automatic summarization. However, due to the diversity and complexity of natural languages, traditional natural language processing methods face many challenges in knowledge extraction, such as data sparsity, insufficient semantic understanding, and high computing resource requirements for processing large-scale data sets. These challenges limit the accuracy, efficiency, and generalization ability of the model, making it a difficult task to effectively extract knowledge from complex natural language texts. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a natural language knowledge extraction method and device based on the collaboration of large and small models, which can efficiently process large amounts of data, improve the efficiency, accuracy and generalization ability of knowledge extraction, while maintaining high accuracy, especially in the extraction of specific events and entities, and can also reduce costs and resource consumption.

[0004] In the first aspect, an embodiment of the present invention provides a natural language knowledge extraction method based on the collaboration of large and small models, wherein the method includes: obtaining a pre-collected natural language text file; converting the natural language text file into a vector format to generate vector data; inputting the vector data into a pre-constructed small model and outputting a recognition result; the small model optimizes the parameters of the neural network based on a circular energy flow optimization algorithm, and is constructed using an extreme learning machine classification algorithm based on a dynamic weight adjustment mechanism; a training sample set for training the small model is constructed after event extraction from an initial sample set using a large natural language model; and based on the recognition result, determining the event indicated by the natural language text file.

[0005] In combination with the first aspect, an embodiment of the present invention provides a first implementation manner of the first aspect, wherein the small model includes a feature extraction model and a classifier model; the step of inputting vector data into a pre-built small model and outputting a recognition result includes: extracting features from the vector data through the feature extraction model of the small model, and outputting key features in the vector data; wherein the parameters of the feature extraction model are assigned energy values, and the energy values ​​are adjusted based on a circular energy flow optimization algorithm; the key features are input into the classifier model of the small model, and the event type recognition result is output; wherein the classifier model is based on an extreme learning machine as a classifier algorithm, and a dynamic weight adjustment mechanism is used to adjust the weights of the extreme learning machine.

[0006] In combination with the first aspect, an embodiment of the present invention provides a second implementation of the first aspect, wherein a method for constructing a classifier model includes: randomly initializing hidden layer weight parameters and biases of an extreme learning machine; inputting a training sample set through the hidden layer of the extreme learning machine, calculating the hidden layer output using a Sigmoid nonlinear activation function, and generating an output layer output; calculating adaptive noise as part of the weight update, and calculating the weight update amount of the output layer of the extreme learning machine using the partial derivative of the loss function; wherein the adaptive noise is determined based on the loss change rate and adjustment coefficient of the current iteration; based on the weight update amount, updating the output layer weight of the extreme learning machine, and adjusting the learning rate of the extreme learning machine according to error feedback; until the extreme learning machine meets the preset stop iteration condition, a classifier model is constructed based on the extreme learning machine.

[0007] In combination with the first aspect, an embodiment of the present invention provides a third implementation of the first aspect, wherein a method for constructing a feature extraction model includes: allocating initial energy to the parameters of the neural network according to the importance of the parameters of the neural network; inputting the training sample set into the neural network, performing forward propagation, monitoring and calculating the energy efficiency of the parameters; adjusting the energy of the parameters based on the energy efficiency and the loss energy of the neural network; controlling the iteration of the neural network through energy conservation checks until preset conditions are met, and constructing a feature extraction model based on the neural network.

[0008] In combination with the first aspect, an embodiment of the present invention provides a fourth implementation of the first aspect, wherein the step of adjusting the energy of parameters based on energy efficiency and loss energy of a neural network includes: converting and optimizing energy based on loss energy and energy efficiency; wherein the conversion rules are dynamically adjusted according to changes in a training sample set; and adjusting the energy distribution of parameters through the back propagation process of loss energy.

[0009] In combination with the first aspect, an embodiment of the present invention provides a fifth implementation of the first aspect, wherein the method further includes: converting the key features into a low-dimensional quantum state representation through an encoder; wherein the low-dimensional quantum state representation is generated using quantum logic gates; and passing the low-dimensional quantum state representation to a decoder to reconstruct a high-dimensional representation of the key features.

[0010] In combination with the first aspect, an embodiment of the present invention provides a sixth implementation of the first aspect, wherein the parameters of the encoder and the parameters of the decoder use an overall loss function to calculate the total loss and are updated and determined through a back-propagation algorithm.

[0011] In combination with the first aspect, an embodiment of the present invention provides a seventh implementation of the first aspect, wherein a method for constructing a training sample set includes: obtaining a target text from a natural language processing data set to generate an initial sample set; the natural language processing data set includes news articles, academic papers, and website text content; inputting the initial sample set into a pre-built natural language generation model, and outputting event data corresponding to the initial sample set; annotating the event data to generate data labels; the data labels are used to characterize the event type of the event data; using an embedding layer to generate a semantic vector for the data label, and inputting the semantic vector and the random noise vector into a generative adversarial network to generate an expanded sample; and constructing a training sample set based on the expanded sample and the event data.

[0012] In combination with the first aspect, an embodiment of the present invention provides an eighth implementation of the first aspect, wherein the method also includes: during the training process of the generative adversarial network, quality assessment of data generated by the generator of the generative adversarial network is performed based on external indicators by the discriminator of the generative adversarial network; based on the result of the quality assessment, adjusting the training strategy of the generative adversarial network; wherein the training strategy includes: updating the parameters of the generator and the discriminator according to the minimum deviation loss function, and dynamically adjusting the learning rates of the generator and the discriminator.

[0013] In the second aspect, an embodiment of the present invention provides a natural language knowledge extraction device based on the collaboration of large and small models, wherein the device includes: a data acquisition module, used to acquire a pre-collected natural language text file; a data processing module, used to convert the natural language text file into a vector format and generate vector data; an execution module, used to input the vector data into a pre-constructed small model and output a recognition result; the small model optimizes the parameters of the neural network based on a circular energy flow optimization algorithm, and is constructed using an extreme learning machine classification algorithm based on a dynamic weight adjustment mechanism; a training sample set for training the small model is constructed after event extraction of the initial sample set by the natural language large model; an output module, used to determine the event indicated by the natural language text file based on the recognition result.

[0014] The embodiments of the present invention bring the following beneficial effects: The embodiments of the present invention provide a method and device for natural language knowledge extraction based on the collaboration of large and small models, which uses a large model for preliminary event extraction to construct training samples for a small model. The generated samples are of higher quality and can better reflect the actual data distribution. Through the collaboration of large and small models, the system can effectively process large-scale data while ensuring high accuracy, thereby optimizing model performance and efficiently using resources. It is particularly accurate in the recognition of specific events and entities, and performs well in the extraction of specific events and entities.

[0015] The small model optimizes the parameters of the neural network based on the circular energy flow optimization algorithm, aiming to solve the problem of gradient disappearance or explosion, and improve the stability and efficiency of training. The extreme learning machine with dynamic weight adjustment mechanism is more flexible than the classic extreme learning machine, allowing dynamic adjustment of weights during network training to accelerate convergence. The above improved parameter optimization algorithm and data processing strategy reduce the demand for computing resources, reduce training costs, and improve the accuracy of knowledge extraction.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of a natural language knowledge extraction method based on large and small model collaboration provided by the present invention;

[0020] Figure 2 A flowchart of another natural language knowledge extraction method based on large and small model collaboration provided by the present invention;

[0021] Figure 3 A flowchart of a method for constructing a training sample set provided by the present invention;

[0022] Figure 4A schematic diagram of the structure of a natural language knowledge extraction device based on large and small model collaboration provided by the present invention;

[0023] Figure 5 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0025] The embodiments of the present invention provide a natural language knowledge extraction method and device based on large and small model collaboration, which can improve the efficiency, accuracy and generalization ability of knowledge extraction while reducing costs and resource consumption.

[0026] For ease of understanding, a natural language knowledge extraction method based on large and small model collaboration provided by an embodiment of the present invention is first described in detail. Figure 1 A flowchart of a natural language knowledge extraction method based on large and small model collaboration provided by an embodiment of the present invention is shown. Figure 1 As shown, the method comprises the following steps:

[0027] Step S102, obtaining a pre-collected natural language text file.

[0028] Step S104, converting the natural language text file into a vector format to generate vector data.

[0029] Step S106, input the vector data into the pre-built small model and output the recognition result.

[0030] Step S108, determining the event indicated by the natural language text file based on the recognition result.

[0031] Among them, the training sample set for training the small model is constructed after event extraction of the initial sample set through the natural language large model. In the specific implementation, the text data set of the target field is first collected. Secondly, the text file is converted into a format suitable for model processing using a preprocessing script, and the data is converted using a selected vectorization technology (such as word embedding). In one embodiment, the embodiment of the present invention uses a pre-trained word embedding model to realize the conversion between words and vectors, and maps the words to a high-dimensional vector space. The fixed-dimensional vector obtained by converting each word through the word embedding model can retain the semantic relationship between words, thereby converting the entire text into a sequence of word vectors.

[0032] Assume that the natural language text is T={t1,t2,…,t i ,…,t nys}, where t i is the i-th word in the text, t nys is the number of words in the text. Further, through the word embedding model , each word t i is mapped to a vector space of fixed dimension, namely: i =E(t i ). In the formula, is the word embedding model, t i is the word in the i-th text, v i is the i-th converted word vector. Based on this, the text T is converted into a vector sequence {v1,v2,…,v i ,…,v nys}.

[0033] Next, the above data can use the large model to perform preliminary event extraction to construct a training set, and then use this training set to train the small model. Throughout the process, it is necessary to monitor the performance of the model training, which may include indicators such as accuracy and loss function value, and adjust the model parameters or optimization strategies as needed. After the training is completed, the small model is deployed to practical applications, such as information retrieval systems, text classifiers, etc. During actual operation, the model performance needs to be continuously evaluated, and fine-tuned and optimized based on feedback. Among them, the embodiment of the present invention uses a large model for preliminary event extraction to construct training samples for the small model, which is more efficient than relying on manual annotation or simple data amplification, and the generated samples are of higher quality and can better reflect the actual data distribution.

[0034] Furthermore, the small model of the embodiment of the present invention optimizes the parameters of the neural network based on the circular energy flow optimization algorithm, aiming to solve the problem of gradient disappearance or explosion, and improve the stability and efficiency of training. In addition, the extreme learning machine classification algorithm based on the dynamic weight adjustment mechanism is constructed. The extreme learning machine using the dynamic weight adjustment mechanism is more flexible than the classic extreme learning machine, allowing dynamic adjustment of weights during network training to accelerate convergence. The improved parameter optimization algorithm and data processing strategy of the embodiment of the present invention reduce the demand for computing resources, reduce training costs, and improve the accuracy of knowledge extraction.

[0035] In one embodiment, the data of the embodiment of the present invention is derived from a public natural language processing data set, including news articles, academic papers, and website text content. The collected natural language text data is stored as a UTF-8 encoded text file. In one embodiment, taking the event extraction of news report text as an example, for example, for the text "Zhang San attended a science and technology exchange meeting in Shanghai", the event "attended a science and technology exchange meeting" is extracted through the natural language generation model. The generation model is a natural language model based on the Transformer architecture, which is trained on a large amount of data and has basic event extraction capabilities. Further, a training sample set is constructed for training a small model. The samples of the training sample set are events extracted by the large model. In this embodiment, it is "Zhang San attended a science and technology exchange meeting in Shanghai." The event is labeled. In this embodiment, the labeling category is "attending a meeting." Correspondingly, the small model determines the event indicated by the natural language text file based on the category indicated by the identified input data.

[0036] Among them, "big model" and "small model" generally refer to the scale of the model, including the complexity of the model (such as the number of parameters) and the size of the training data set. Large models often have billions or more parameters, which enables them to capture more complex patterns and more subtle language structures, helping to improve the generalization ability of the model on a variety of tasks. Small models have fewer parameters and are usually designed for specific tasks, such as sentiment analysis, keyword extraction, etc. Small models are faster to train and infer, and are more suitable for edge devices or resource-constrained environments. In the embodiments of the present invention, through the collaboration of large and small models, the system can effectively process large-scale data while ensuring high accuracy, especially in the recognition of specific events and entities.

[0037] Furthermore, based on the above embodiment, the embodiment of the present invention also provides another natural language knowledge extraction method based on the collaboration of large and small models. Figure 2 A flowchart corresponding to an embodiment of the present invention is shown, referring to Figure 2 , the method comprises the following steps:

[0038] Step S202, obtaining a pre-collected natural language text file.

[0039] Step S204, converting the natural language text file into a vector format to generate vector data.

[0040] Step S206, extracting features from the vector data using the feature extraction model of the small model, and outputting key features in the vector data.

[0041] Step S208, input the key features into the classifier model of the small model, and output the event type recognition result.

[0042] In specific implementation, the small model of the embodiment of the present invention includes a feature extraction model and a classifier model. Correspondingly, the embodiment of the present invention first extracts features from the vector data through the feature extraction model of the small model, and then classifies it through the classifier model, so as to determine the recognition result corresponding to the vector data.

[0043] Some solutions in the prior art use neural networks for feature extraction. Traditional neural networks are often subject to the problems of gradient disappearance, gradient explosion or falling into local optimal solutions in parameter optimization and training stability, which is more obvious in deep network training, affecting the stability of training and the performance of the model. In an embodiment of the present invention, a 3-layer fully connected neural network is used for feature extraction. In addition, the parameters of the feature extraction model of the embodiment of the present invention are assigned energy values. In the embodiment of the present invention, energy values ​​are assigned to the parameters according to the initial importance of the parameters. A larger energy value means that the parameter has a greater potential impact on the output when the model is initialized. In addition, the energy value is adjusted based on the circular energy flow optimization algorithm. In specific implementation, the embodiment of the present invention adopts a neural network parameter optimization method based on circular energy flow optimization. Inspired by the energy circulation system in nature and the circulation of matter and energy in the ecosystem, the circular energy flow optimization algorithm optimizes the weights and biases of the neural network through a closed-loop feedback mechanism. Each update of the network is not only a simple parameter adjustment, but involves a process of energy calculation and energy conversion, thereby enhancing the adaptability and robustness of the network in the face of different training stages.

[0044] Specifically, the method for constructing the feature extraction model of the embodiment of the present invention is as follows:

[0045] 1) Assign initial energy to the parameters of the neural network according to their importance.

[0046] Let the i-th parameter of the neural network be p i , the initial energy value corresponding to the i-th parameter is E i , for each parameter p in the network i (including weights and biases) assign an initial energy value E iThe energy distribution is based on the initial importance of the parameters, which is set proportional to the normalized value of the parameters and can be expressed as:

[0047]

[0048] In the formula, and are the mean and standard deviation of the neural network parameters, is the neural network initialization weight hyperparameter. Preferably, the parameter is set to 5.

[0049] 2) Input the training sample set into the neural network, perform forward propagation, monitor and calculate the energy efficiency of the parameters.

[0050] When the network is forward propagating, each parameter The energy usage of is monitored, and the ratio of the energy contributed by each parameter to the energy consumed is calculated. Let the energy efficiency of the i-th parameter be , that is, calculate the energy efficiency of each parameter The calculation method can be expressed as:

[0051]

[0052] In the formula, is the loss function, is a hyperparameter for adjusting the energy transfer efficiency. Preferably, is set to 0.1.

[0053] Furthermore, the partial derivatives of the parameters Characterized as parameters Network loss The sensitivity of is the rate at which the loss changes due to parameter changes. In one embodiment, the loss function is the mean square error loss function, then The calculation method can be expressed as:

[0054]

[0055] In the formula, is the network prediction value, obtained by the preset Softmax classification function; is the actual label value, is the network output parameter The partial derivative of .

[0056] Furthermore, the loss energy is calculated based on the difference between the network output and the true label , which is used to guide subsequent energy feedback and redistribution, can be expressed as:

[0057]

[0058] In the formula, is the conversion factor of energy to loss ratio. Preferably, Set to 10.

[0059] 3) Adjust the energy of parameters based on energy efficiency and loss energy of neural network.

[0060] Specifically, the energy distribution of the parameters is adjusted through the back propagation process of the lost energy. If the energy utilization efficiency of a parameter is low, energy will be extracted from the parameter energy pool to supplement it. The calculation method of the new energy value can be expressed as:

[0061]

[0062] In the formula, is the number of parameters, is the parameter after energy adjustment The new energy value of is the energy feedback factor. Preferably, is set to 0.05.

[0063] Furthermore, the embodiment of the present invention also converts and optimizes energy based on loss energy and energy efficiency. In each iteration, parameter energy is converted and optimized, including energy synthesis and decomposition, to achieve optimal network performance, which can be expressed as:

[0064]

[0065] In the formula, is the parameter after energy conversion The final energy value of is the parameter after energy adjustment The new energy value, is the jth parameter of the neural network; is the self-energy conservation coefficient, is the energy transfer coefficient from the jth parameter to the ith parameter; is the energy efficiency adjustment factor.

[0066] Furthermore, the energy transfer coefficient The calculation method can be expressed as:

[0067]

[0068] In the formula, is the distance between the i-th parameter and the j-th parameter in the network structure, is the distance between the i-th parameter and the k-th parameter in the network structure, and in one embodiment, is represented by the difference in the number of layers; is the temperature parameter that controls the transfer sensitivity and is calculated by the simulated annealing algorithm.

[0069] Furthermore, the energy efficiency adjustment coefficient uses the energy response characteristics of each node in the network to dynamically adjust the energy transfer efficiency. The calculation method can be expressed as:

[0070]

[0071] In the formula, is the sensitivity adjustment factor, which controls the degree of influence of the loss on energy efficiency; is the loss function About parameters Preferably, Set to 0.3.

[0072] In the embodiment of the present invention, the conversion rule is dynamically adjusted according to the changes in the training sample set to adapt to different training environments and data characteristics. The environmental adaptability adjustment method can be expressed as:

[0073]

[0074]

[0075] In the formula, is the updated self-energy conservation coefficient, is the updated energy transfer coefficient from the jth parameter to the ith parameter, is the environmental adaptation rate, is the energy value change between this iteration and the previous iteration, is the change in loss function between this iteration and the previous iteration, It is adjusted according to energy and loss changes The function of is a function that adjusts the energy transfer coefficient.

[0076] 4) Control the iteration of the neural network through energy conservation check until the preset conditions are met, and build a feature extraction model based on the neural network.

[0077] Repeat the above steps and perform energy conservation checks at the same time to ensure that the overall network energy is effectively utilized. The energy conservation check method can be expressed as:

[0078]

[0079] In the formula, is the total number of neural network parameters.

[0080] When the energy conservation equation holds true, the iteration stops, indicating that the model training is completed.

[0081] In summary, the circular energy flow optimization algorithm adopted in the neural network training proposed in the present invention is inspired by the energy circulation system in nature. It optimizes parameters through energy calculation and conversion, enhances the adaptability and stability of the network in different training stages, and effectively solves the problem of gradient disappearance or explosion in traditional neural networks.

[0082] Furthermore, the classifier model of the embodiment of the present invention is based on the extreme learning machine as the classifier algorithm, and is constructed using a dynamic weight adjustment mechanism. The model can adaptively adjust the weights according to the characteristics of the data during the training process in the hope of obtaining better classification performance. In the specific implementation, a dynamic weight adjustment mechanism is used to achieve faster convergence speed and higher classification accuracy.

[0083] Specifically, the method for constructing the classifier model of the embodiment of the present invention is as follows:

[0084] 1) Randomly initialize the hidden layer weight parameters and bias of the extreme learning machine.

[0085] Specifically, the hidden layer weight parameters and bias of the extreme learning machine are randomly initialized through Gaussian distribution, which can be expressed as:

[0086]

[0087] In the formula, is the weight parameter of the hidden layer of the extreme learning machine, is the initialization standard deviation, is a normal distribution. Preferably, Set to 0.01.

[0088] 2) Input the training sample set through the hidden layer of the extreme learning machine, use the Sigmoid nonlinear activation function to calculate the hidden layer output, and generate the output layer output.

[0089] Specifically, the input feature vector is transformed through the hidden layer using the Sigmoid nonlinear activation function to generate the output of the hidden layer, which can be expressed as:

[0090]

[0091]

[0092] In the formula, is the output of the hidden layer, and are the weights and biases of the hidden layers, respectively. is the Sigmoid activation function, is the input feature vector, is the output of the output layer, is the weight of the output layer.

[0093] Among them, the input feature vector represents the original information of the data. These vectors can be obtained from various types of raw data (such as text word vectors, etc.) through preprocessing. These feature vectors are then passed to the hidden layer. The neurons in the hidden layer will be weighted and summed with the input features, plus a bias term. The output of each hidden layer neuron will be nonlinearly transformed through an activation function to introduce the complexity of the model so that it can learn and express richer data relationships. The output range of the Sigmoid function is (0,1), which can map continuous inputs to outputs close to 0 or 1, which is easy to interpret as probabilities. In the specific implementation, the input feature vector is a training sample of the classifier model, which refers to the data composition corresponding to the above-mentioned natural language text file.

[0094] 3) Calculate the adaptive noise as part of the weight update, and use the partial derivative of the loss function to calculate the weight update of the output layer of the extreme learning machine.

[0095] The above adaptive noise is determined based on the loss change rate and adjustment coefficient of the current iteration. Define adaptive noise It is calculated by the loss change rate of the current iteration, and the calculation method can be expressed as:

[0096]

[0097] In the formula, Indicates the amount of change in noise level; is the adjustment factor; represents the rate of change of the loss function. Preferably, Set to 0.35.

[0098] Furthermore, the weight update amount is calculated by taking the partial derivative of the loss function, and then the output layer weight is optimized and adjusted, which can be expressed as:

[0099]

[0100]

[0101] In the formula, is the weight update amount, and They are respectively the output layer weights after optimization update and the output layer weights before optimization update, is the learning rate of the extreme learning machine, is the loss function of the extreme learning machine, specifically the mean square error loss function.

[0102] Furthermore, the loss function The partial derivative of The calculation method can be expressed as:

[0103]

[0104] Where Nsa is the number of training samples input in the current batch, It is The model output of samples is It is The true labels of samples; After being processed by the activation function, The hidden layer output of samples.

[0105] 4) Based on the weight update amount, update the output layer weight of the extreme learning machine, and adjust the learning rate of the extreme learning machine according to the error feedback.

[0106] The above weight update amount is used to optimize and adjust the output layer weight. In one embodiment, the way to adjust the learning rate can be expressed as:

[0107]

[0108] In the formula, is the learning rate of the adjusted extreme learning machine, is the learning rate of the extreme learning machine before adjustment, is the attenuation factor of the learning rate of the extreme learning machine, is the current iteration number, is the total number of iterations. Preferably, Set to 0.95.

[0109] 5) Until the extreme learning machine meets the preset stop iteration condition, a classifier model is built based on the extreme learning machine.

[0110] Repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0111] In summary, the embodiment of the present invention combines dynamic weight adjustment and back propagation algorithm to build a classifier model, improves the classification performance and learning speed of the extreme learning machine, makes the model converge faster, and improves the classification accuracy. In addition, in combination with the above-mentioned feature extraction model, the embodiment of the present invention improves the computing efficiency and resource utilization optimization through improved algorithm optimization parameters, reduces training costs, and improves the accuracy of knowledge extraction.

[0112] Furthermore, the embodiment of the present invention also performs feature dimensionality reduction on the output of the feature extraction model to reduce the number of features, speed up the algorithm training speed, and reduce storage requirements. In addition, the data is visualized and the relationship between features is understood to improve the model interpretability. The classification boundaries are made clearer, and the accuracy of classification is improved.

[0113] Existing dimensionality reduction technologies such as PCA and t-SNE often have difficulty maintaining the key features of the data when processing extremely large-scale data, which affects the performance of the final model. The embodiment of the present invention uses the quantum state conversion self-encoder algorithm as a dimensionality reduction model to effectively constrain the dimension of the feature vector obtained after feature extraction. Among them, the key features are converted into low-dimensional quantum state representations through the encoder; the low-dimensional quantum state representations are passed to the decoder to reconstruct the high-dimensional representation of the key features.

[0114] Specifically, the low-dimensional quantum state representation of the embodiment of the present invention is generated by quantum logic gates. The parameters of the encoder and the decoder use the overall loss function to calculate the total loss, and are updated and determined by the back-propagation algorithm. The traditional autoencoder compresses the input data into a low-dimensional representation through an encoder, and then reconstructs the data through a decoder. The present invention uses a quantum state conversion autoencoder to treat the encoded data as a quantum state, and uses quantum state conversion technology to process the data to improve the quality of data compression, so that the model can maintain the important features of the data in a very low dimension.

[0115] Furthermore, the embodiments of the present invention define an overall loss function to evaluate the difference between the reconstructed data and the original data, and use quantum variational methods or other hybrid quantum classical optimization strategies to adjust the parameters of the encoder and decoder through the back-propagation concept to minimize the loss function.

[0116] Specifically, the training process of the quantum state conversion self-encoder algorithm is as follows:

[0117] 1) Initialize the weights and biases of the encoder and decoder in the quantum state conversion self-encoder model. In one embodiment, the parameter initialization is set in a random initialization manner, which can be expressed as:

[0118]

[0119] In the formula, is the parameter update operation, are the parameters of the encoder, are the parameters of the decoder, is a parameter random initialization function, is the initialization standard deviation. Preferably, Set to 0.01.

[0120] 2) Pass the input feature vector to the encoder. Let the feature vector input to the encoder be , the encoder takes as input the feature vector Converted to a low-dimensional quantum state representation through an encoder , which can be expressed as:

[0121]

[0122] In the formula, is the encoded low-dimensional representation, is the encoder function, are the parameters of the encoder.

[0123] Specifically, in the quantum state conversion stage, the embodiment of the present invention uses quantum logic gates to Processing is performed to obtain the converted quantum state , which can be expressed as:

[0124]

[0125] In the formula, is the processed quantum state, Quantum logic gate operations that represent quantum state transitions, is the control parameter.

[0126] Furthermore, the calculation method of quantum logic gate operation can be expressed as:

[0127]

[0128] In the formula, represents the unitary transformation of the quantum state, is the Hamiltonian, is a control parameter. Preferably, Set to .

[0129] 3) The data processed by the quantum state is then passed to the decoder, and the decoder receives the processed quantum state , reconstructing the high-dimensional representation of the original input data can be expressed as:

[0130]

[0131] In the formula, is the reconstructed data after decoding, is the decoder function, Decoder parameters.

[0132] 4) The embodiment of the present invention adopts the overall loss function Calculate the total loss of the autoencoder and update all network parameters through the back-propagation algorithm. In one embodiment, the overall loss function Including reconstruction loss and quantum state fidelity loss, the calculation method is:

[0133]

[0134] In the formula, for and The reconstruction error, is the quantum state fidelity loss function, is the regularization parameter. Preferably, Set to 0.4.

[0135] Furthermore, the quantum state fidelity loss The calculation method can be expressed as:

[0136]

[0137] In the formula, Representing quantum states Original encoding state The square of the magnitude of the inner product between .

[0138] Furthermore, the parameters of the autoencoder are based on the loss function For any encoder or decoder parameter, the update method can be expressed as:

[0139]

[0140] In the formula, and Respectively represent the values ​​before and after the parameter update, is the learning rate, Represents the loss function About parameters Preferably, Set to 0.01.

[0141] 5) Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0142] In summary, the present invention utilizes the quantum state conversion strategy to perform feature dimensionality reduction of data, which not only improves the quality of data compression, but also maintains the data features in extremely low dimensions.

[0143] Step S210, determining the event indicated by the natural language text file based on the recognition result.

[0144] In summary, after processing the data using the feature extraction model, classifier model, and data dimensionality reduction method constructed in the above manner, a recognition result is obtained. The event indicated by the text file is determined based on the recognition result. In one embodiment, the recognition result is the content recognized by the classifier model, such as giving a speech, business meeting, webinar, workshop participation, social activities, business trips, writing reports, participating in meeting preparations, customer visits, and team building.

[0145] In summary, another natural language knowledge extraction method and device based on the collaboration of large and small models provided by the embodiment of the present invention achieves optimization of model performance and efficient use of resources by performing preliminary event extraction with a large model and fine-tuning training with a small model. Through the collaborative use of large and small models, the system can efficiently process large amounts of data while maintaining high accuracy, especially in the extraction of specific events and entities.

[0146] Furthermore, based on the above embodiment, the embodiment of the present invention also provides another natural language knowledge extraction method based on the collaboration of large and small models, which is to explain the steps of constructing a training sample set, wherein the training sample set of the embodiment of the present invention is used to train a small model, such as training a feature extraction model and a classifier model. In specific implementation, the feature extraction model can be trained by the training sample set, and the extracted data can be processed by the above dimensionality reduction method after being output, and the processed data can be used as the training sample set of the classifier model. Figure 3 A flowchart of a method for constructing a training sample set provided by an embodiment of the present invention is shown. Figure 3 , including the following steps:

[0147] Step S10, obtaining target text from a natural language processing dataset to generate an initial sample set.

[0148] Step S11, input the initial sample set into a pre-built natural language generation model, and output the event data corresponding to the initial sample set.

[0149] Specifically, this step refers to the description of the above embodiment. The embodiment of the present invention performs preliminary data extraction through a large model, which not only improves the efficiency of data processing, but also provides a strong foundation for deeper data analysis and decision support.

[0150] Step S12: annotate the event data and generate data labels.

[0151] In one embodiment, the natural language processing dataset includes news articles, academic papers, and website text content; data labels are used to characterize event types of event data, such as giving a speech, business meeting, webinar, workshop participation, social activities, business trips, writing reports, participating in meeting preparations, customer visits, and team building.

[0152] Step S13, using the embedding layer to generate a semantic vector for the data label, inputting the semantic vector and the random noise vector into the generative adversarial network to generate an expanded sample.

[0153] In the task of natural language knowledge extraction, the collection, acquisition, annotation and preprocessing of training data are time-consuming and labor-intensive, and insufficient training samples can easily lead to poor model generalization ability and affect the accuracy of the model. Traditional methods rely on simple copying or manual annotation to expand data, which lacks efficient methods for generating new data with practical value. To solve this problem, the embodiment of the present invention uses a semantically enhanced generative adversarial network to generate samples to achieve data expansion.

[0154] The generative adversarial network consists of two parts: a generator (G) and a discriminator (D). The generator is responsible for generating data similar to the real data, while the discriminator is responsible for distinguishing the generated data from the real data. In order to optimize the performance of the generative adversarial network and reduce the risk of mode collapse, the present invention uses the minimum deviation loss function to accurately quantify the difference between the generated data and the real data, and improves the fidelity and semantic relevance of the generated data by optimizing the semantic consistency in the generation process.

[0155] The steps of generating an extended sample in the embodiment of the present invention are as follows:

[0156] a-First initialize the generator and the discriminator Specifically, the parameters of the generator and the discriminator are set to and , the initialization process can be expressed as:

[0157]

[0158] In the formula, Represents the parameters of the generator; represents the parameters of the discriminator; It means randomly selecting an initial value in the range from 0 to 0.1.

[0159] b-Furthermore, the embedding layer is used to generate semantic vectors for the data labels, and the semantic vectors and random noise vectors are input into the generative adversarial network to generate augmented samples.

[0160] Specifically, the embedding layer is used for Generate a semantic vector from each label , the semantic vector is embedded through the function Obtained, can be expressed as:

[0161]

[0162] In the formula, For the The semantic vector corresponding to the label; is the embedded function.

[0163] c- In each iteration, the generator receives random noise and semantic vector , generate data , which can be expressed as:

[0164]

[0165] In the formula, is a generator function, Represents random noise of the input Represents fake data output by the generator.

[0166] d-Furthermore, the real data samples are fed into the discriminator together with the generated fake data, and the discriminator evaluates the real data at the same time and fake data , output authenticity score and ,and, is the discriminator function, represents the real data sample, Fake data samples generated for the generator.

[0167] In which, during the training process of the generative adversarial network, the embodiment of the present invention uses the discriminator of the generative adversarial network to perform quality assessment on the data generated by the generator of the generative adversarial network based on external indicators; and, based on the results of the quality assessment, adjust the training strategy of the generative adversarial network.

[0168] The quality of generated data is evaluated using external indicators. The external indicators are calculated through the quality evaluation function, and the training strategy is adjusted accordingly. It can be expressed as:

[0169]

[0170] In the formula, is the quality assessment function; To generate data; A function that adjusts the training strategy based on the quality evaluation.

[0171] In one embodiment, the quality assessment function This is achieved by calculating the similarity between the generated data and the data in the real data set, which can be expressed as:

[0172]

[0173] in, Represents the similarity measurement function, and the measurement method is implemented based on the Euclidean distance calculation method; Indicates the first samples, The number of training samples input to the generative adversarial network for the current batch.

[0174] Among them, the training strategy of the embodiment of the present invention includes: updating the parameters of the generator and the discriminator according to the minimum deviation loss function, and dynamically adjusting the learning rate of the generator and the discriminator.

[0175] The specific implementation includes the following steps:

[0176] 1) Update the parameters of the generator and discriminator according to the loss function. The update method can be expressed as:

[0177]

[0178]

[0179] In the formula, and They are the parameters of the generator and discriminator before updating; and are the parameters of the updated generator and discriminator respectively; represents the loss function of the generative adversarial network; and Represent the learning rates of the generator and the discriminator respectively; and represents the gradient with respect to the respective parameter. Preferably, and It is set in a dynamic adjustment mode, and its initial values ​​are set to 0.01 and 0.05 respectively.

[0180] Furthermore, the loss function of the generative adversarial network is specifically expressed as the minimum deviation loss function. The minimum deviation loss function focuses on the difference in statistical distribution between the generated data and the real data, such as the variance difference. The calculation method can be expressed as:

[0181]

[0182] In the formula, is a factor that is dynamically adjusted based on the deviation of the data, represents the loss function of the generator, Represents the loss function obtained by the discriminator.

[0183] Furthermore, the factor representation loss function dynamically adjusted based on the deviation of the data achieves the minimum deviation of the generated data, which depends on the statistical characteristics of the generated data and the real data, and the calculation method is expressed as:

[0184]

[0185]

[0186] In the formula, The number of samples entered for the current batch; is the joint variance, which represents the overall distribution variance of the real data and the generated data; The current batch A real sample, The current batch The noise input to the generator; is the variance calculation function.

[0187] Furthermore, the generator loss This is achieved through log-likelihood loss, which can be expressed as:

[0188]

[0189] In the formula, Express expectations, Indicates noise The distribution of Indicates The distribution of semantic vectors corresponding to labels, Indicates that it obeys a specific distribution.

[0190] Furthermore, the discriminator loss goal is to distinguish between real data and generated fake data, which is achieved using cross entropy loss and can be expressed as:

[0191]

[0192] In the formula, is the probability distribution of the real data; represents random noise.

[0193] 2) Automatically adjust the learning rates of the generator and discriminator according to the efficiency of the discriminator in distinguishing true and false data. The adjustment method can be expressed as:

[0194]

[0195] In the formula, and is the adjusted learning rate; is the function for adjusting the learning rate.

[0196] In one embodiment, the learning rate adjustment mechanism is dynamically adjusted based on the feedback of the loss, and the adjustment method can be expressed as:

[0197]

[0198]

[0199] In the formula, and is the learning rate adjustment factor, which is based on the loss The rate of change is defined as:

[0200]

[0201]

[0202] In the formula, is a hyperparameter that controls the sensitivity of learning rate changes. Preferably, Set to 0.01.

[0203] e-Further, the above steps are repeated until a preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0204] Step S14: construct a training sample set based on the expanded samples and event data.

[0205] Another natural language knowledge extraction method based on large and small model collaboration provided by an embodiment of the present invention uses a generative adversarial network for data expansion, and especially uses semantic vectors to enhance the semantic consistency of generated data, which can effectively improve the quality and applicability of generated data and reduce the difference between generated samples and actual data.

[0206] In combination with the above embodiments, the embodiments of the present invention are based on data expansion of generative adversarial networks and network optimization of circular energy flows, so that the model has better generalization ability, can adapt to diverse data inputs, and reduce overfitting. By combining the collaborative work of large models and small models, data expansion of generative adversarial networks, neural network training methods optimized by circular energy flows, feature dimensionality reduction of quantum state conversion strategies, and extreme learning machine improvement strategies with dynamic weight adjustment, multiple challenges faced by traditional technologies are effectively solved, the performance of natural language processing tasks is improved, and all-round optimization from data processing to model training is achieved.

[0207] Furthermore, an embodiment of the present invention also provides a natural language knowledge extraction device based on large and small model collaboration, Figure 4 FIG. 4 shows a schematic diagram of the structure of a natural language knowledge extraction device based on large and small model collaboration provided by an embodiment of the present invention. Figure 4 As shown, the device includes: a data acquisition module 100, used to acquire a pre-collected natural language text file; a data processing module 200, used to convert the natural language text file into a vector format and generate vector data; an execution module 300, used to input the vector data into a pre-constructed small model and output a recognition result; the small model optimizes the parameters of the neural network based on a circular energy flow optimization algorithm, and is constructed using an extreme learning machine classification algorithm based on a dynamic weight adjustment mechanism; a training sample set for training the small model is constructed after event extraction from an initial sample set by a natural language large model; an output module 400, used to determine the event indicated by the natural language text file based on the recognition result.

[0208] A natural language knowledge extraction device based on large and small model collaboration provided in an embodiment of the present invention has the same technical features as a natural language knowledge extraction method based on large and small model collaboration provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0209] Furthermore, based on the above embodiments, the embodiments of the present invention also provide another natural language knowledge extraction device based on the collaboration of large and small models, wherein the small model includes a feature extraction model and a classifier model; the above execution module 300 is also used to extract features from vector data through the feature extraction model of the small model, and output key features in the vector data; wherein the parameters of the feature extraction model are assigned energy values, and the energy values ​​are adjusted based on the circular energy flow optimization algorithm; the key features are input into the classifier model of the small model, and the event type recognition result is output; wherein the classifier model is based on the extreme learning machine as the classifier algorithm, and a dynamic weight adjustment mechanism is used to adjust the weight of the extreme learning machine.

[0210] Furthermore, the above-mentioned execution module 300 is also used to construct a classifier model, including: randomly initializing the hidden layer weight parameters and bias of the extreme learning machine; inputting a training sample set through the hidden layer of the extreme learning machine, calculating the hidden layer output using the Sigmoid nonlinear activation function, and generating an output layer output; calculating adaptive noise as part of the weight update, and using the partial derivative of the loss function to calculate the weight update amount of the output layer of the extreme learning machine; wherein the adaptive noise is determined according to the loss change rate and adjustment coefficient of the current iteration; based on the weight update amount, updating the output layer weight of the extreme learning machine, and adjusting the learning rate of the extreme learning machine according to error feedback; until the extreme learning machine meets the preset stop iteration condition, a classifier model is constructed based on the extreme learning machine.

[0211] Furthermore, the above-mentioned execution module 300 is also used to construct a feature extraction model, including: allocating initial energy to the parameters of the neural network according to the importance of the parameters of the neural network; inputting the training sample set into the neural network, performing forward propagation, monitoring and calculating the energy efficiency of the parameters; adjusting the energy of the parameters based on the energy efficiency and the loss energy of the neural network; controlling the iteration of the neural network through energy conservation checks until the preset conditions are met, and constructing a feature extraction model based on the neural network.

[0212] The execution module 300 is also used to convert and optimize energy based on loss energy and energy efficiency; wherein the conversion rules are dynamically adjusted according to changes in the training sample set; and the energy distribution of the parameters is adjusted through the back propagation process of the loss energy.

[0213] The execution module 300 is also used to convert the key features into low-dimensional quantum state representations through an encoder; wherein the low-dimensional quantum state representations are generated using quantum logic gates; and the low-dimensional quantum state representations are passed to a decoder to reconstruct the high-dimensional representations of the key features. The parameters of the encoder and the parameters of the decoder are calculated using an overall loss function to calculate the total loss, and are updated and determined through a back-propagation algorithm.

[0214] Furthermore, the above-mentioned execution module 300 is also used to construct a training sample set, including: obtaining target text from a natural language processing data set to generate an initial sample set; the natural language processing data set includes news articles, academic papers, and website text content; inputting the initial sample set into a pre-built natural language generation model, and outputting event data corresponding to the initial sample set; annotating the event data to generate data labels; the data labels are used to characterize the event type of the event data; using an embedding layer to generate a semantic vector for the data label, and inputting the semantic vector and the random noise vector into a generative adversarial network to generate an expanded sample; and constructing a training sample set based on the expanded sample and the event data.

[0215] The above-mentioned execution module 300 is also used to perform quality assessment on the data generated by the generator of the generative adversarial network based on external indicators through the discriminator of the generative adversarial network during the training process of the generative adversarial network; based on the results of the quality assessment, adjust the training strategy of the generative adversarial network; wherein the training strategy includes: updating the parameters of the generator and the discriminator according to the minimum deviation loss function, and dynamically adjusting the learning rates of the generator and the discriminator.

[0216] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above Figures 1 to 3 The steps of the method are shown.

[0217] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor. Figures 1 to 3 The steps of the method are shown.

[0218] The embodiment of the present invention also provides a schematic diagram of the structure of an electronic device, such as Figure 5 FIG. 5 is a schematic diagram of the structure of the electronic device, wherein the electronic device includes a processor 51 and a memory 50, the memory 50 stores computer executable instructions that can be executed by the processor 51, and the processor 51 executes the computer executable instructions to implement the above Figures 1 to 3 The method shown.

[0219] exist Figure 5 In the illustrated embodiment, the electronic device further includes a bus 52 and a communication interface 53 , wherein the processor 51 , the communication interface 53 and the memory 50 are connected via the bus 52 .

[0220] Among them, the memory 50 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 52 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc., and can also be an AMBA (Advanced Microcontroller Bus Architecture, on-chip bus standard) bus, wherein AMBA defines three types of buses, including APB (Advanced Peripheral Bus) bus, AHB (Advanced High-performance Bus) bus and AXI (Advanced eXtensible Interface) bus. The bus 52 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0221] The processor 51 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 51. The above processor 51 can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 51 reads the information in the memory and combines its hardware to complete the above Figures 1 to 3 Any of the methods shown.

[0222] A computer program product of a natural language knowledge extraction method and apparatus based on large and small model collaboration provided in an embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0223] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0224] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0225] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A natural language knowledge extraction method based on large and small model collaboration, characterized in that: include: Obtaining a pre-collected natural language text file; Converting the natural language text file into a vector format to generate vector data; The vector data is input into a pre-built small model, and a recognition result is output; the small model optimizes the parameters of the neural network based on a circular energy flow optimization algorithm, and is constructed using an extreme learning machine classification algorithm based on a dynamic weight adjustment mechanism; The training sample set for training the small model is constructed by extracting events from the initial sample set using a large natural language model; Based on the recognition result, determining the event indicated by the natural language text file; Wherein, the small model includes a feature extraction model, the parameters of the feature extraction model are assigned energy values, and the energy values ​​are adjusted based on a circular energy flow optimization algorithm; The step of adjusting the energy value based on the circular energy flow optimization algorithm comprises: During the training process of the feature extraction model, monitoring and calculating the energy efficiency of the parameters of the feature extraction model; Adjusting the energy of a parameter based on the energy efficiency and the loss energy of a neural network of the feature extraction model; The step of adjusting the energy of the parameter based on the energy efficiency and the loss energy of the neural network of the feature extraction model comprises: Based on the loss energy and the energy efficiency, the energy is converted and optimized; wherein the conversion rule is dynamically adjusted according to the change of the training sample set; Through the back propagation process of the lost energy, the energy distribution of the parameters is adjusted.

2. The method according to claim 1, characterized in that The small model also includes a classifier model; the step of inputting the vector data into the pre-built small model and outputting the recognition result includes: Performing feature extraction on the vector data by using the feature extraction model of the small model, and outputting key features in the vector data; The key features are input into the classifier model of the small model, and the event type recognition result is output; wherein the classifier model is based on an extreme learning machine as a classifier algorithm, and a dynamic weight adjustment mechanism is used to adjust the weight of the extreme learning machine.

3. The method according to claim 2, characterized in that The method for constructing the classifier model comprises: Randomly initialize the hidden layer weight parameters and bias of the extreme learning machine; Input a training sample set through the hidden layer of the extreme learning machine, calculate the hidden layer output using the Sigmoid nonlinear activation function, and generate an output layer output; Calculating adaptive noise as part of the weight update, and calculating the weight update amount of the output layer of the extreme learning machine using the partial derivative of the loss function; wherein the adaptive noise is determined according to the loss change rate and adjustment coefficient of the current iteration; Based on the weight update amount, updating the output layer weight of the extreme learning machine, and adjusting the learning rate of the extreme learning machine according to error feedback; Until the extreme learning machine meets a preset stop iteration condition, a classifier model is constructed based on the extreme learning machine.

4. The method according to claim 1, characterized in that: The method for constructing the feature extraction model comprises: assigning initial energy to the parameters of the neural network according to the importance of the parameters of the neural network; Inputting the training sample set into the neural network, performing forward propagation, monitoring and calculating the energy efficiency of the parameters; Adjusting the energy of a parameter based on the energy efficiency and the loss energy of the neural network; The iteration of the neural network is controlled by energy conservation check until a preset condition is met, and a feature extraction model is constructed based on the neural network.

5. The method according to claim 2, characterized in that: The method further comprises: Converting the key feature into a low-dimensional quantum state representation through an encoder; wherein the low-dimensional quantum state representation is generated using a quantum logic gate; The low-dimensional quantum state representation is passed to a decoder to reconstruct a high-dimensional representation of the key feature.

6. The method according to claim 5, characterized in that The parameters of the encoder and the parameters of the decoder are determined by calculating the total loss using an overall loss function and updating through a back-propagation algorithm.

7. The method according to claim 1, characterized in that The method for constructing the training sample set includes: Obtaining target text from a natural language processing dataset to generate an initial sample set; the natural language processing dataset includes news articles, academic papers, and website text content; Inputting the initial sample set into a pre-built natural language generation model, and outputting event data corresponding to the initial sample set; Annotating the event data to generate a data label; the data label is used to characterize the event type of the event data; Generate a semantic vector for the data label using an embedding layer, input the semantic vector and a random noise vector into a generative adversarial network to generate an expanded sample; A training sample set is constructed based on the augmented samples and the event data.

8. The method according to claim 7, characterized in that The method further comprises: During the training process of the generative adversarial network, the quality of the data generated by the generator of the generative adversarial network is evaluated based on external indicators by the discriminator of the generative adversarial network; Based on the result of the quality assessment, the training strategy of the generative adversarial network is adjusted; wherein the training strategy includes: updating the parameters of the generator and the discriminator according to the minimum deviation loss function, and dynamically adjusting the learning rates of the generator and the discriminator.

9. A natural language knowledge extraction device based on large and small model collaboration, characterized in that: The device comprises: A data acquisition module, used to acquire pre-collected natural language text files; A data processing module, used for converting the natural language text file into a vector format to generate vector data; An execution module is used to input the vector data into a pre-built small model and output a recognition result; the small model optimizes the parameters of the neural network based on a circular energy flow optimization algorithm, and is constructed using an extreme learning machine classification algorithm based on a dynamic weight adjustment mechanism; a training sample set for training the small model is constructed by extracting events from an initial sample set through a natural language large model; An output module, configured to determine the event indicated by the natural language text file based on the recognition result; The small model includes a feature extraction model, and the parameters of the feature extraction model are assigned energy values, and the energy values ​​are adjusted based on a circular energy flow optimization algorithm; the step of adjusting the energy values ​​based on the circular energy flow optimization algorithm includes: during the training process of the feature extraction model, monitoring and calculating the energy efficiency of the parameters of the feature extraction model; adjusting the energy of the parameters based on the energy efficiency and the loss energy of the neural network of the feature extraction model; The step of adjusting the energy of parameters based on the energy efficiency and the loss energy of the neural network of the feature extraction model comprises: converting and optimizing the energy based on the loss energy and the energy efficiency; wherein the conversion rule is dynamically adjusted according to the change of the training sample set; and adjusting the energy distribution of the parameters through the back propagation process of the loss energy.

Images