Large power model construction method and system based on natural language processing
By building a timing attention module and a gating mechanism, the shortcomings of the existing power big model in dealing with time characteristics and semantic connections are solved, and more efficient power problem identification and answer generation are achieved.
Patent Information
- Application Number
- CN202411608400.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-04-11
AI Technical Summary
When using the multi-head attention mechanism to extract attention characteristics of dialogue text, existing power big models focus on spatial characteristics, ignore temporal characteristics, and lack consideration of semantic connections between multiple problem statements.
By collecting timing power questions and real power answers, a question encoding module, feature extraction module, timing attention module, feature filtering module and gating mechanism are built to generate fusion features and output power answer categories, enhancing consideration of time characteristics and semantic connections.
It improves the time feature processing capability of dialogue text, enhances the semantic connection between multiple problem statements, and improves the overall performance of the power big model.
Smart Images
Figure CN119807336A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for constructing a large electric power model based on natural language processing. Background Art
[0002] The process of constructing a large power model based on natural language processing refers to the process of using natural language processing technology to perform text processing on the power question corpus and the power answer corpus in the power knowledge question and answer process, and constructing a neural network model of the power knowledge question and answer from the processed text.
[0003] At present, the power big model is mainly used to identify the user's power problem category and assign the corresponding power answer to the power problem category. The existing power big model is mainly composed of an embedding layer, a multi-head attention mechanism, a normalization layer, and a feedforward neural network. When the existing technology uses the multi-head attention mechanism to extract the attention features of the dialogue text, it mainly focuses on the spatial features of the dialogue text, and does not consider the temporal features enough. However, the meanings expressed by dialogues of different orders in actual scenarios will also be different. Secondly, the existing technology recognizes dialogue texts mostly by identifying keywords or identifying the meaning of sentences, but does not consider the semantic connection between multiple question sentences enough. Therefore, the existing power big model construction method does not consider the temporal features and semantic connections enough. Summary of the invention
[0004] Purpose of the invention: The purpose of the present invention is to provide a method and system for constructing a large power model based on natural language processing, increase the consideration of time characteristics and semantic connections, and solve the problems existing in the background technology.
[0005] Technical solution: The method for constructing a large power model based on natural language processing described in the present invention comprises the following steps:
[0006] (1) collecting the user's time-series power questions and the real power answers corresponding to the time-series power questions, constructing a question encoding module for the time-series power questions, and converting the time-series power questions into question vectors through the question encoding module;
[0007] (2) constructing a feature extraction module of the question vector using a preset long short-term memory network, extracting features from the question vector using the feature extraction module to obtain question features, constructing a temporal attention module of the question features using a preset attention mechanism, and outputting attention features corresponding to the question features using the temporal attention module;
[0008] (3) constructing a feature filtering module of the attention feature, using the feature filtering module to filter irrelevant attention features in the attention feature to obtain the remaining attention features, constructing a gating mechanism for the remaining attention features, and based on the gating mechanism, performing feature fusion on the remaining attention features to obtain the fused features;
[0009] (4) constructing an answer output module of the fused features, and outputting the power answer category corresponding to the fused features according to the answer output module, and combining the question encoding module, feature extraction module, temporal attention module, feature filtering module, gating mechanism and answer output module into a pre-trained power model of the user;
[0010] Calculate the answer loss value between the real power answer and the power answer category, train the pre-trained power big model based on the answer loss value, obtain the trained power big model, and use the trained power big model as the power big model construction result of the user.
[0011] Furthermore, step (1) constructs a problem coding module for the time series power problem, including:
[0012] Construct a word-level segmentation component for the sequential power problem;
[0013] Constructing a character level segmentation component of the word level segmentation component;
[0014] Constructing a missing value interpolation component of the character level word segmentation component;
[0015] Constructing a word segmentation scoring mechanism for the missing value interpolation component;
[0016] Constructing a scoring and screening mechanism for the word segmentation scoring mechanism;
[0017] The word level segmentation component, the character level segmentation component, the missing value interpolation component, the word segmentation scoring mechanism and the scoring screening mechanism are combined into the question encoding module.
[0018] Furthermore, in step (2), a feature extraction module of the question vector is constructed using a preset long short-term memory network, including:
[0019] Obtaining a forget gate, an input gate, and an output gate in the long short-term memory network;
[0020] Based on the forget gate, the input gate and the output gate, a feature extraction module of the question vector is constructed.
[0021] Furthermore, a preset attention mechanism is used to construct a temporal attention module of the problem feature, including: using the attention mechanism as the temporal attention module.
[0022] Furthermore, constructing a feature filtering module of the attention feature includes: constructing a feature filtering module of the attention feature.
[0023] Furthermore, a gating mechanism for the residual attention feature is constructed, including: constructing a feature mapping layer for the residual attention feature; constructing a feature fusion layer between the residual attention feature and the feature mapping layer; and combining the feature mapping layer and the feature fusion layer into the gating mechanism.
[0024] Further, calculating the answer loss value between the real power answer and the power answer category includes: calculating the answer loss value between the real power answer and the power answer category.
[0025] Furthermore, the pre-trained power model is trained based on the answer loss value to obtain a trained power model, including: calculating the gradient value of the answer loss value; calculating the gradient descent distance in the pre-trained power model based on the gradient value; judging whether the gradient descent distance is less than a preset distance; when the gradient descent distance is not less than the preset distance, updating the weights and biases in the pre-trained power model using the following formula to obtain updated weights and updated biases; completing the training of the pre-trained power model using the updated weights and updated biases to obtain a first trained power model; when the gradient descent distance is less than the preset distance, determining a second trained power model corresponding to the pre-trained power model.
[0026] The present invention provides a system for constructing a large electric power model based on natural language processing, comprising:
[0027] A question conversion module, used to collect the user's time-series power questions and the real power answers corresponding to the time-series power questions, construct a question encoding module for the time-series power questions, and convert the time-series power questions into question vectors through the question encoding module;
[0028] A feature output module, used to construct a feature extraction module of the question vector using a preset long short-term memory network, extract features of the question vector using the feature extraction module to obtain question features, construct a temporal attention module of the question features using a preset attention mechanism, and output an attention feature corresponding to the question features using the temporal attention module;
[0029] A feature fusion module is used to construct a feature filtering module of the attention feature, use the feature filtering module to filter irrelevant attention features in the attention feature to obtain remaining attention features, construct a gating mechanism for the remaining attention features, and perform feature fusion on the remaining attention features based on the gating mechanism to obtain fused features;
[0030] A model combination module is used to construct an answer output module of the fused features, and output the power answer category corresponding to the fused features according to the answer output module, and combine the question encoding module, the feature extraction module, the temporal attention module, the feature filtering module, the gating mechanism and the answer output module into a pre-trained power model of the user;
[0031] The model training module is used to calculate the answer loss value between the real power answer and the power answer category, train the pre-trained power model based on the answer loss value, obtain the trained power model, and use the trained power model as the power model construction result of the user.
[0032] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any one of the methods for constructing a large electric power model based on natural language processing is implemented.
[0033] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the temporal power problem of the text is represented by a vector, a temporal attention module of the problem feature is constructed by utilizing a preset attention mechanism, so as to increase the consideration of the multi-head attention mechanism for the temporal features of the dialogue text, a feature filtering module of the attention feature is constructed, so as to make the neural network focus on important attention features and remove unimportant attention features, and a gating mechanism of the remaining attention features is constructed to strengthen the semantic connection between different attention features through the feature fusion layer in the gating mechanism; finally, the consideration of the connection between temporal features and semantics is increased. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a schematic diagram of the method flow of the present invention;
[0035] Figure 2 It is a system schematic diagram of the present invention. DETAILED DESCRIPTION
[0036] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0037] like Figure 1 As shown, an embodiment of the present invention provides a method for constructing a large power model based on natural language processing, comprising the following steps:
[0038] S1. Collect the user's time-series power problems and the real power answers corresponding to the time-series power problems, construct a question encoding module for the time-series power problems, and convert the time-series power problems into question vectors through the question encoding module.
[0039] The time-series power questions refer to power-related questions asked by users, and the real power answers refer to replies to power questions determined by experts.
[0040] By constructing the problem encoding module of the time-series power problem, the time-series power problem of the text is represented by a vector.
[0041] Constructing the problem encoding module of the time series power problem includes: constructing a word-level segmentation component of the time series power problem; constructing a character-level segmentation component of the word-level segmentation component; constructing a missing value interpolation component of the character-level segmentation component; and constructing a segmentation scoring mechanism of the missing value interpolation component using the following formula:
[0042] P(e1,e2,…,e j )=P(e1)P(e2∣e1)P(e3∣e1,e2)…P(e j |e1,…,e j-1 )
[0043] P(e j |e1,…,e j-1 )=P(e j ∣e j-n+1 ,…,e j-1 )
[0044] P(e1,e2,…,e j )=P(e1)P(e2∣e1)P(e3∣e1,e2)…P(e j ∣e j-n+1 ,…,e j-1 )
[0045] Among them, P(e1,e2,…,e j )=P(e1)P(e2∣e1)P(e3∣e1,e2)…P(e j ∣e j-n+1 ,…,e j-1 ) represents the word segmentation scoring mechanism, e1, e2,…, e j represents the word segmentation sequence output by the missing value interpolation component, j represents the number of word segmentation sequences, P(e1) represents the marginal probability of e1 word segmentation, P(e2|e1) represents the conditional probability of e2 word segmentation occurring under the premise of e1 word segmentation event, P(e3|e1,e2) represents the conditional probability of e3 word segmentation occurring under the premise of e1 and e2 word segmentation events, and P(e j |e1,…,e j-1 ) indicates that e1,…,e j-1 The premise of the participle event occurs jThe conditional probability of word segmentation, e1,…,e j-1 Indicates e j All the participles before the participle, e j-n+1 ,…,e j-1 Indicates e j The n-1 participles before the participle, P(e j ∣e j-n+1 ,…,e j-1 ) indicates that e occurs j-n+1 ,…,e j-1 The premise of the participle event occurs j The conditional probability of word segmentation, P(e j |e1,…,e j-1 )=P(e j ∣e j-n+1 ,…,e j-1 ) means to convert a huge number of e1,…,e j-1 The corresponding conditional probability is simplified to a relatively small number of e j-n+1 ,…,e j-1 The corresponding conditional probability, P(e1,e2,…,e j ) indicates that the word segmentation scoring mechanism output occurs at the same time e1,…,e j Probability of word segmentation events;
[0046] The scoring and screening mechanism of the word segmentation scoring mechanism is constructed using the following formula:
[0047]
[0048] Among them, scoring screening mechanism represents the scoring screening mechanism, P[r] represents the r probabilities output by the word segmentation scoring mechanism, and P max(r) represents the r largest probabilities output by the word segmentation scoring mechanism, w k Indicates P k The corresponding weight, P k represents the kth probability output by the word segmentation scoring mechanism, x k P represents the output of the missing value interpolation component received by the word segmentation scoring mechanism k The probability of the corresponding word, X k represents the word segmentation output by the word segmentation scoring mechanism;
[0049] The word level segmentation component, the character level segmentation component, the missing value interpolation component, the word segmentation scoring mechanism and the scoring screening mechanism are combined into the question encoding module.
[0050] Among them, the word-level segmentation component refers to a segmentation algorithm that segments the timing power problem into words according to the space positions and commas in the sentence, and the character-level segmentation component refers to an algorithm that segments the words output by the word-level segmentation component in the character sense, for example, a sentence composed of 5 consecutive characters separated by space positions is divided into two-by-two segmentations, single segmentations, and segmentations composed of three characters, etc. The missing value interpolation component refers to the operation of filling in 0 for the segmentation when the dimension of the segmentation does not reach the preset dimension.
[0051] S2. Use a preset long short-term memory network to construct a feature extraction module for the question vector, use the feature extraction module to extract features from the question vector to obtain question features, use a preset attention mechanism to construct a temporal attention module for the question features, and use the temporal attention module to output attention features corresponding to the question features.
[0052] The feature extraction module of the question vector is constructed by using a preset long short-term memory network, including: obtaining a forget gate, an input gate, and an output gate in the long short-term memory network; based on the forget gate, the input gate, and the output gate, the feature extraction module of the question vector is constructed by using the following formula:
[0053]
[0054] Among them, feature extractor represents the feature extraction module, f t represents the output of the forget gate, sigmoid represents the activation function, W if represents the weight of the forget gate, x t represents the problem vector, b if represents the bias of the forget gate, h t-1 represents the output result of the feature extraction module at time t-1, W hf Indicates h in the forget gate t-1 The weight of b hf Indicates h in the forget gate t-1 The bias, i t , g t Represents the two output results of the input gate, W ii , W ig represents the weight of the input gate, b ii 、b ig represents the bias of the input gate, W hi , W hg Indicates the input gate h t-1 The weight of b hi 、b hg Indicates h in the forget gate t-1 The bias, c t Represents the memory of the current state, ct-1 represents the memory of the state at time t-1, i represents the serial number of the input gate, W io represents the weight of the output gate, b io Represents the bias of the output gate, W ho Indicates the output gate h t-1 The weight of b ho Indicates the output gate h t-1 The bias, o t Represents the output result of the output gate, h t Represents the output result of the feature extraction module at time t.
[0055] A temporal attention module of the question feature is constructed by utilizing a preset attention mechanism, so as to increase the consideration of the temporal features of the dialogue text by the multi-head attention mechanism.
[0056] The attention mechanism refers to a timing graph attention mechanism.
[0057] The temporal attention module of the problem feature is constructed using a preset attention mechanism, including: the attention mechanism includes:
[0058]
[0059] Wherein, attention mechanism represents the attention mechanism, d(t) represents the entity-time feature matrix, represents the vector of the 0th node of the l-1th layer in the attention mechanism at the t0th time, Indicates mapping t-t0 to d T A function of a low-dimensional vector of dimension, t represents the time variable, represents the vector of the first node of the l-1th layer in the attention mechanism at the t1th time, Indicates mapping t-t1 to d T A function of a low-dimensional vector of dimension Indicates that the Nth node in the l-1th layer of the attention mechanism is in the tth N The vector at the moment, Indicates that tt N Mapping to d T A function of a low-dimensional vector of dimension w Q 、w K 、w V represents the weight in the attention mechanism, Q(t) represents the query vector, K(t) represents the key vector, V(t) represents the value vector, IN represents InstanceNormalization, Instance Normalization represents the normalization layer, Softmax represents the activation function, Conv represents the convolutional layer, dT ′ represents the dimension of K(t), FFN represents feedforward neural network, h t represents the output result of the feature extraction module at time t, represents the vector of the node of the lth layer output by the attention mechanism at time t, and g(t) represents the hidden vector output by the attention mechanism;
[0060] The attention mechanism is used as the temporal attention module.
[0061] S3. Construct a feature filtering module for the attention feature, use the feature filtering module to filter irrelevant attention features in the attention feature to obtain the remaining attention features, construct a gating mechanism for the remaining attention features, and based on the gating mechanism, perform feature fusion on the remaining attention features to obtain fused features.
[0062] By constructing a feature filtering module of the attention feature, the neural network is made to focus on important attention features and remove unimportant attention features.
[0063] Constructing the feature filtering module of the attention feature, including: constructing the feature filtering module of the attention feature using the following formula:
[0064]
[0065] Among them, feature extraction represents the feature filtering module, represents the s attention features output by the temporal attention module, represents the s largest attention features output by the temporal attention module, M u Indicates g u The weight, g u represents the u-th attention feature output by the temporal attention module, G u Represents the remaining attention features output by the feature filtering module.
[0066] A gating mechanism for the residual attention features is constructed to strengthen the semantic connection between different attention features through a feature fusion layer in the gating mechanism.
[0067] Constructing a gating mechanism for the residual attention feature includes: constructing a feature mapping layer of the residual attention feature using the following formula:
[0068] G u ′=sigmoid(w G *G u )+b G
[0069] Among them, G u ′=sigmoid(w G (G u ))+b G represents the feature mapping layer, G u ′ represents the output result of the feature mapping layer, w G represents the weight of the feature mapping layer, b G represents the bias of the feature map layer, G u represents the remaining attention features output by the feature filtering module;
[0070] The feature fusion layer between the residual attention feature and the feature mapping layer is constructed using the following formula:
[0071] G u ″=w G ′Hadamard (G u ′,G u )+b G '
[0072] Among them, G u ″ represents the output result of the feature fusion layer, w G ′ represents the weight of the feature fusion layer, b G ′ represents the bias of the feature fusion layer, G u ′ represents the output result of the feature mapping layer, G u represents the remaining attention features output by the feature filtering module, and Hadamard() represents the Hadamard product;
[0073] The feature mapping layer and the feature fusion layer are combined as the gating mechanism.
[0074] S4. Construct an answer output module of the fused features, and output the power answer category corresponding to the fused features according to the answer output module, and combine the question encoding module, the feature extraction module, the temporal attention module, the feature filtering module, the gating mechanism and the answer output module into a pre-trained power model of the user.
[0075] Constructing the answer output module of the fused features, including: constructing the answer output module of the fused features using the following formula:
[0076] H = softmax(MLP(Hadamard(G u ″,w x x k )))
[0077] Among them, H represents the answer category probability output by the answer output module, softmax represents the activation function, MLP represents the multi-layer perceptron, Hadamard() represents the Hadamard product, and w x represents the weight in the answer output module, x k P represents the output of the missing value interpolation component received by the word segmentation scoring mechanism k The probability of the corresponding word, G u ″ represents the output result of the feature fusion layer.
[0078] S5. Calculate the answer loss value between the real power answer and the power answer category, train the pre-trained power big model based on the answer loss value to obtain the trained power big model, and use the trained power big model as the power big model construction result of the user.
[0079] Calculating the answer loss value between the real power answer and the power answer category includes: calculating the answer loss value between the real power answer and the power answer category using the following formula:
[0080]
[0081] Among them, L(θ,θ1,θ2) represents the answer loss value, θ represents the power answer category, θ1 represents the word segmentation in the real power answer, θ2 represents the sentence in the real power answer, and U I represents the Ith participle in the electricity answer category, I′ represents the number of participles in the electricity answer category, V J represents the Jth sentence in the power answer category, J′ represents the number of sentences in the power answer category, p(U I |θ,θ1) represents the output of the pre-trained power model U I The probability of belonging to θ1, p(V J |θ,θ2) represents the output of the pre-trained power model V J The probability of belonging to θ2.
[0082] In one embodiment of the present invention, the pre-trained power model is trained based on the answer loss value to obtain the trained power model, including: calculating the gradient value of the answer loss value using the following formula:
[0083]
[0084] in, represents the gradient value, L represents the answer loss value, w represents any weight in the pre-trained power model, and b represents any bias in the pre-trained power model;
[0085] Based on the gradient value, the gradient descent distance in the pre-trained power model is calculated using the following formula:
[0086]
[0087] Among them, ε represents the gradient descent distance, τ represents the learning rate, Represents the gradient value;
[0088] Determine whether the gradient descent distance is less than a preset distance; when the gradient descent distance is not less than the preset distance, update the weights and biases in the pre-trained power model using the following formula to obtain updated weights and updated biases:
[0089] w′=w+ε
[0090] b′=b+ε
[0091] Wherein, w′ represents the updated weight, b′ represents the updated bias, ε represents the gradient descent distance, w represents any weight in the pre-trained power model, and b represents any bias in the pre-trained power model;
[0092] The training of the pre-trained power model is completed by using the updated weight and the updated bias to obtain a first trained power model; when the gradient descent distance is less than a preset distance, a second trained power model corresponding to the pre-trained power model is determined.
[0093] It should be noted that the preset distance is a manually set parameter and can be set according to specific scenarios.
[0094] like Figure 2 As shown, the present invention provides a power large model construction system 200 based on natural language processing, comprising:
[0095] It includes a question conversion module 201, a feature output module 202, a feature fusion module 203, a model combination module 204 and a model training module 205. The module described in the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.
[0096] The question conversion module 201 is used to collect the user's time-series power questions and the real power answers corresponding to the time-series power questions, construct a question encoding module for the time-series power questions, and convert the time-series power questions into question vectors through the question encoding module;
[0097] The feature output module 202 is used to construct a feature extraction module of the question vector using a preset long short-term memory network, extract features from the question vector using the feature extraction module to obtain question features, construct a temporal attention module of the question features using a preset attention mechanism, and output an attention feature corresponding to the question features using the temporal attention module;
[0098] The feature fusion module 203 is used to construct a feature filtering module for the attention feature, use the feature filtering module to filter irrelevant attention features in the attention feature to obtain the remaining attention features, construct a gating mechanism for the remaining attention features, and perform feature fusion on the remaining attention features based on the gating mechanism to obtain a fused feature;
[0099] The model combination module 204 is used to construct the answer output module of the fused feature, and output the power answer category corresponding to the fused feature according to the answer output module, and combine the question encoding module, the feature extraction module, the temporal attention module, the feature filtering module, the gating mechanism and the answer output module into a pre-trained power model of the user;
[0100] The model training module 205 is used to calculate the answer loss value between the real power answer and the power answer category, train the pre-trained power big model based on the answer loss value, obtain the trained power big model, and use the trained power big model as the power big model construction result of the user.
[0101] An embodiment of the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, the method for constructing a large electric power model based on natural language processing as described in any one of the above is implemented.
Claims
1. A method for constructing a large power model based on natural language processing, characterized in that: The following steps are involved: (1) collecting the user's time-series power questions and the real power answers corresponding to the time-series power questions, constructing a question encoding module for the time-series power questions, and converting the time-series power questions into question vectors through the question encoding module; (2) constructing a feature extraction module of the question vector using a preset long short-term memory network, extracting features from the question vector using the feature extraction module to obtain question features, constructing a temporal attention module of the question features using a preset attention mechanism, and outputting attention features corresponding to the question features using the temporal attention module; (3) constructing a feature filtering module of the attention feature, using the feature filtering module to filter irrelevant attention features in the attention feature to obtain the remaining attention features, constructing a gating mechanism for the remaining attention features, and based on the gating mechanism, performing feature fusion on the remaining attention features to obtain the fused features; (4) constructing an answer output module of the fused features, and outputting the power answer category corresponding to the fused features according to the answer output module, and combining the question encoding module, feature extraction module, temporal attention module, feature filtering module, gating mechanism and answer output module into a pre-trained power model of the user; Calculate the answer loss value between the real power answer and the power answer category, train the pre-trained power big model based on the answer loss value, obtain the trained power big model, and use the trained power big model as the power big model construction result of the user.
2. According to claim 1, a method for constructing a large power model based on natural language processing is characterized in that: Step (1) constructs a problem coding module for the time series power problem, including: Construct a word-level segmentation component for the sequential power problem; Constructing a character level segmentation component of the word level segmentation component; Constructing a missing value interpolation component of the character level word segmentation component; Constructing a word segmentation scoring mechanism for the missing value interpolation component; Constructing a scoring and screening mechanism for the word segmentation scoring mechanism; The word level segmentation component, the character level segmentation component, the missing value interpolation component, the word segmentation scoring mechanism and the scoring screening mechanism are combined into the question encoding module.
3. According to claim 1, a method for constructing a large power model based on natural language processing is characterized in that: In step (2), a feature extraction module of the question vector is constructed using a preset long short-term memory network, including: Obtaining a forget gate, an input gate, and an output gate in the long short-term memory network; Based on the forget gate, the input gate and the output gate, a feature extraction module of the question vector is constructed.
4. According to claim 1, a method for constructing a large power model based on natural language processing is characterized in that: A temporal attention module of the problem feature is constructed using a preset attention mechanism, including: using the attention mechanism as the temporal attention module.
5. According to claim 1, a method for constructing a large power model based on natural language processing is characterized in that: Constructing a feature filtering module of the attention feature, including: constructing a feature filtering module of the attention feature.
6. According to claim 1, a method for constructing a large power model based on natural language processing is characterized in that: Constructing a gating mechanism for the residual attention feature, including: constructing a feature mapping layer for the residual attention feature; constructing a feature fusion layer between the residual attention feature and the feature mapping layer; and combining the feature mapping layer and the feature fusion layer into the gating mechanism.
7. According to claim 1, a method for constructing a large power model based on natural language processing is characterized in that: Calculating the answer loss value between the real power answer and the power answer category includes: calculating the answer loss value between the real power answer and the power answer category.
8. According to claim 1, a method for constructing a large power model based on natural language processing is characterized in that: The pre-trained power model is trained based on the answer loss value to obtain a trained power model, including: calculating the gradient value of the answer loss value; calculating the gradient descent distance in the pre-trained power model based on the gradient value; judging whether the gradient descent distance is less than a preset distance; when the gradient descent distance is not less than the preset distance, updating the weights and biases in the pre-trained power model using the following formula to obtain updated weights and updated biases; completing the training of the pre-trained power model using the updated weights and updated biases to obtain a first trained power model; when the gradient descent distance is less than the preset distance, determining a second trained power model corresponding to the pre-trained power model.
9. A power large model construction system based on natural language processing, characterized in that: The system comprises: A question conversion module, used to collect the user's time-series power questions and the real power answers corresponding to the time-series power questions, construct a question encoding module for the time-series power questions, and convert the time-series power questions into question vectors through the question encoding module; A feature output module, used to construct a feature extraction module of the question vector using a preset long short-term memory network, extract features of the question vector using the feature extraction module to obtain question features, construct a temporal attention module of the question features using a preset attention mechanism, and output an attention feature corresponding to the question features using the temporal attention module; A feature fusion module is used to construct a feature filtering module of the attention feature, use the feature filtering module to filter irrelevant attention features in the attention feature to obtain remaining attention features, construct a gating mechanism for the remaining attention features, and perform feature fusion on the remaining attention features based on the gating mechanism to obtain fused features; A model combination module is used to construct an answer output module of the fused features, and output the power answer category corresponding to the fused features according to the answer output module, and combine the question encoding module, the feature extraction module, the temporal attention module, the feature filtering module, the gating mechanism and the answer output module into a pre-trained power model of the user; The model training module is used to calculate the answer loss value between the real power answer and the power answer category, train the pre-trained power model based on the answer loss value, obtain the trained power model, and use the trained power model as the power model construction result of the user.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into the processor, it implements the method for constructing a large power model based on natural language processing according to any one of claims 1 to 8.