Part-of-speech prediction method and device, computer equipment and storage medium

Abstract

The invention relates to a part-of-speech prediction method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a text vocabulary sequence, and extracting semantic features corresponding to vocabularies in the text vocabulary sequence; the text vocabulary sequence comprises at least two vocabularies; for each vocabulary, according to the semantic feature corresponding to the vocabulary, extracting the hidden state feature of the current time step where the vocabulary is located; according to the accumulative increment memory feature under the previous time step and the hidden state feature of the vocabulary, generating an accumulative increment memory feature under the current time step; according to the accumulative increment memory feature under the current time step and the hidden state feature corresponding to the vocabulary, determining a time sequence feature corresponding to the vocabulary; extracting content features corresponding to the vocabularies according to the time sequence features corresponding to the vocabularies; and predicting the part-of-speech of the corresponding vocabulary according to the semantic feature, the time sequence feature and the content feature corresponding to each vocabulary in the text vocabulary sequence. By adopting the method, the accuracy of the part-of-speech prediction result of the vocabulary can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a method, apparatus, computer device and storage medium for predicting a part of speech. Background Art

[0002] In the field of Natural Language Processing (NLP), when using deep learning models to process text information, it is usually necessary to capture the contextual relationship between words through a self-attention mechanism in order to determine the part of speech of each word. Therefore, how to accurately capture the contextual relationship between words is crucial.

[0003] In traditional technologies, commonly used deep learning models include the Bertrand model (BERT), Long Short-Term Memory (LSTM) and Graph Convolutional Networks (GCN). These deep learning models can capture the relationship between words in text information, but they cannot accurately capture word relationships, which affects the accuracy of the part-of-speech prediction results of words in the text. Summary of the invention

[0004] Based on this, it is necessary to provide a part-of-speech prediction method, device, computer equipment and storage medium that can accurately predict the part-of-speech of a vocabulary in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for predicting a part of speech, comprising:

[0006] Obtaining a text word sequence and extracting semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0007] For each word, extract the hidden state features of the word at the current time step based on the semantic features corresponding to the word;

[0008] Generate the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary;

[0009] Determine the time series features corresponding to the vocabulary according to the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary;

[0010] Extracting content features corresponding to the vocabulary based on the temporal features corresponding to the vocabulary;

[0011] According to the semantic features, temporal features and content features corresponding to each word in the text word sequence, the part of speech of the corresponding word is predicted.

[0012] In one embodiment, the cumulative incremental memory features at the current time step are generated based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary, including: generating the newly added memory features at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary; obtaining the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the newly added memory features at the current time step.

[0013] In one embodiment, a newly added memory feature at the current time step is generated based on the accumulated incremental memory features at the previous time step and the hidden state features of the vocabulary, including: determining the context-related hidden features of the vocabulary based on the correlation degree data between the context vocabulary and the vocabulary in the text vocabulary sequence, and the hidden state features of the vocabulary; generating the newly added memory feature at the current time step based on the context-related hidden features of the vocabulary and the accumulated incremental memory features at the previous time step.

[0014] In one of the embodiments, semantic features corresponding to each word in a text vocabulary sequence are extracted, including: extracting linear features of each word in the text vocabulary sequence at different grammatical dimensions to obtain basic grammatical vectors at the corresponding grammatical dimensions; for each word, generating contextual features of the word at the corresponding grammatical dimensions based on the basic grammatical vectors of the word at different grammatical dimensions and the basic grammatical vectors of the context words of the word in the text vocabulary sequence at the corresponding grammatical dimensions; generating semantic features corresponding to the word based on the contextual features of the word at different grammatical dimensions and the basic grammatical vectors of the word.

[0015] In one embodiment, based on the basic grammatical vectors of the vocabulary at different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence at the corresponding grammatical dimensions, context features of the vocabulary at the corresponding grammatical dimensions are generated, including: based on the basic grammatical vectors of the vocabulary at different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence at the corresponding grammatical dimensions, determining the attention weight of the context vocabulary to the vocabulary at the corresponding grammatical dimension; for each grammatical dimension, weighting the basic grammatical vector of the vocabulary at the corresponding grammatical dimension according to the attention weight of the context vocabulary to the vocabulary at the grammatical dimension, to obtain the context features of the vocabulary at the corresponding grammatical dimension.

[0016] In one of the embodiments, content features corresponding to the words are extracted based on the temporal features corresponding to the words, including: extracting features of the temporal features corresponding to the words based on dynamic convolution weights to obtain content features corresponding to the words; wherein the dynamic convolution weights are generated based on nonlinear feature fusion of the temporal features corresponding to each word in a text vocabulary sequence.

[0017] In one of the embodiments, the part of speech of the corresponding word is predicted based on the semantic features, temporal features and content features corresponding to each word in a text vocabulary sequence, including: for each word, the semantic features, temporal features and content features corresponding to the word are fused to obtain the fused features of the word; based on the fused features of different words in the text vocabulary sequence, a semantic graph of the text vocabulary sequence is constructed; based on the semantic graph, the part of speech of each word in the text vocabulary sequence is predicted.

[0018] In a second aspect, the present application also provides a part-of-speech prediction device, comprising:

[0019] A first extraction module is used to obtain a text word sequence and extract semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0020] The second extraction module is used to extract the hidden state features of each word at the current time step according to the semantic features corresponding to the word;

[0021] A first generation module is used to generate the cumulative incremental memory features at the current time step according to the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary;

[0022] A first determination module is used to determine the time series feature corresponding to the vocabulary according to the accumulated incremental memory feature at the current time step and the hidden state feature corresponding to the vocabulary;

[0023] The third extraction module is used to extract the content features corresponding to the vocabulary according to the temporal features corresponding to the vocabulary;

[0024] The prediction module is used to predict the part of speech of the corresponding word according to the semantic features, temporal features and content features corresponding to each word in the text word sequence.

[0025] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0026] Obtaining a text word sequence and extracting semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0027] For each word, extract the hidden state features of the word at the current time step based on the semantic features corresponding to the word;

[0028] Generate the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary;

[0029] Determine the time series features corresponding to the vocabulary according to the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary;

[0030] Extracting content features corresponding to the vocabulary based on the temporal features corresponding to the vocabulary;

[0031] According to the semantic features, temporal features and content features corresponding to each word in the text word sequence, the part of speech of the corresponding word is predicted.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0033] Obtaining a text word sequence and extracting semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0034] For each word, extract the hidden state features of the word at the current time step based on the semantic features corresponding to the word;

[0035] Generate the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary;

[0036] Determine the time series features corresponding to the vocabulary according to the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary;

[0037] Extracting content features corresponding to the vocabulary based on the temporal features corresponding to the vocabulary;

[0038] According to the semantic features, temporal features and content features corresponding to each word in the text word sequence, the part of speech of the corresponding word is predicted.

[0039] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0040] Obtaining a text word sequence and extracting semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0041] For each word, extract the hidden state features of the word at the current time step based on the semantic features corresponding to the word;

[0042] Generate the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary;

[0043] Determine the time series features corresponding to the vocabulary according to the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary;

[0044] Extracting content features corresponding to the vocabulary based on the temporal features corresponding to the vocabulary;

[0045] According to the semantic features, temporal features and content features corresponding to each word in the text word sequence, the part of speech of the corresponding word is predicted.

[0046] The above-mentioned part-of-speech prediction method, device, computer equipment and storage medium obtain a text vocabulary sequence and extract the semantic features corresponding to each vocabulary in the text vocabulary sequence; wherein the text vocabulary sequence includes at least two words; for each word, according to the semantic features corresponding to the word, extract the hidden state features of the word in the current time step; according to the cumulative incremental memory features at the previous time step and the hidden state features of the word, generate the cumulative incremental memory features at the current time step; according to the cumulative incremental memory features at the current time step and the hidden state features corresponding to the word, determine the time series features corresponding to the word; according to the time series features corresponding to the word, extract the content features corresponding to the word; according to the semantic features, time series features and content features corresponding to each word in the text vocabulary sequence, predict the part of speech of the corresponding word. In the above-mentioned technical scheme, by introducing the cumulative incremental memory features, extracting the time series features corresponding to the words, the omission of key information in the time series features is avoided, thereby improving the richness and comprehensiveness of the time series features. Furthermore, content features are extracted based on temporal features, and the part of speech of the corresponding vocabulary is comprehensively predicted based on the temporal features, content features and semantic features of the vocabulary, so that the obtained part of speech can fully consider the lexical information of different dimensions carried in the text vocabulary from the semantic dimension, temporal dimension and content dimension, thereby improving the richness and diversity of the extracted features. Therefore, the part of speech of the corresponding vocabulary is comprehensively predicted based on the temporal features, content features and semantic features of the vocabulary, thereby improving the accuracy of vocabulary prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] FIG1 is a schematic diagram of a process flow of a method for predicting a part of speech in one embodiment;

[0049] FIG2 is a schematic flow chart of a cumulative incremental memory feature determination step in one embodiment;

[0050] FIG3 is a schematic flow chart of a semantic feature determination step in one embodiment;

[0051] FIG4 is a schematic diagram of a process flow of a part-of-speech prediction step in one embodiment;

[0052] FIG5 is a block diagram of a part-of-speech prediction device in one embodiment;

[0053] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] In an exemplary embodiment, as shown in FIG1 , a method for predicting a part of speech is provided, and the method is applied to a terminal for example. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server, and the present application does not make any limitation on this. In this embodiment, the method includes the following steps:

[0056] S110, obtaining a text vocabulary sequence, and extracting semantic features corresponding to each vocabulary in the text vocabulary sequence; wherein the text vocabulary sequence includes at least two vocabulary.

[0057] The text word sequence can be understood as a sequence composed of at least two words arranged in a specific order. The semantic feature can be understood as a feature that represents the semantic meaning of the words in the text word sequence.

[0058] In one of the optional embodiments, the semantic features corresponding to each word in the text vocabulary sequence can be extracted in the following manner: the text vocabulary sequence is input into the BERT model to obtain the semantic features corresponding to each word in the text vocabulary sequence.

[0059] In another optional embodiment, extracting the semantic features corresponding to each word in the text word sequence may include: performing word embedding processing on the text word sequence to obtain the initial word vector corresponding to each word; performing linear transformation on each initial word vector through mapping matrices of different grammatical dimensions to obtain the grammatical vector under the corresponding grammatical dimension; for each word, performing feature fusion on the grammatical vectors corresponding to the word under different grammatical dimensions to obtain the semantic features corresponding to the word. The different grammatical dimensions may include at least one of the part-of-speech dimension, the syntactic dimension, and the pragmatic dimension.

[0060] It should be noted that the feature fusion in this embodiment includes but is not limited to feature splicing fusion, feature weighted fusion, feature linear fusion or other feature synthesis methods.

[0061] S120, for each word, extracting the hidden state feature of the word at the current time step according to the semantic feature corresponding to the word.

[0062] Here, a time step corresponds to a word in the text vocabulary sequence.

[0063] In an optional embodiment, for each word, the semantic features corresponding to the word can be input into the LSTM model to obtain the forward features and backward features of the word; the forward features and the corresponding backward features are fused to obtain a feature fusion result, and the feature fusion result is used as the hidden state feature of the word at the current time step.

[0064] For example, the forward feature and the corresponding backward feature can be concatenated or added together using formula (1-1) to obtain a feature fusion result, which is used as the hidden state feature of the vocabulary at the current time step.

[0065] (1-1)

[0066] Among them, h t Represents the hidden state features of the vocabulary at the current time step t, The forward feature representing the current time step t of the vocabulary, Represents the backward features of the current time step t of the vocabulary.

[0067] S130, generating the cumulative incremental memory features at the current time step according to the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary.

[0068] Among them, the cumulative incremental memory feature at the current time step can be understood as at least integrating the unforgotten features of all words from the initial time step to the current time step, that is, the memory features, and the features of the association relationship between the words, so that the cumulative incremental memory feature can carry the content information and time step sequence information of all words from the initial time step to the current time step.

[0069] In an optional embodiment, for each word, the time step corresponding to the word is taken as the current time step, and the cumulative incremental memory feature of the previous time step is determined; the cumulative incremental memory feature of the previous time step and the hidden state feature of the word are fused, and the feature fusion result is taken as the cumulative incremental memory feature of the current time step.

[0070] S140, determining the time series features corresponding to the vocabulary according to the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary.

[0071] Among them, the temporal feature can be understood as the feature that represents the order of words in the text word sequence.

[0072] In an optional embodiment, for each word, the time step corresponding to the word can be taken as the current time step to determine the cumulative incremental memory feature at the current time step; the cumulative incremental memory feature at the current time step and the hidden state feature corresponding to the word are fused, and the feature fusion result is used as the temporal feature corresponding to the word.

[0073] For example, formula (1-2) can be used to perform feature weighted fusion on the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary, and the feature weighted fusion result can be used as the temporal feature corresponding to the vocabulary.

[0074] (1-2)

[0075] in, Represents the temporal features corresponding to the current time step t of the vocabulary, h t Represents the hidden state features of the vocabulary at the current time step t, M t The hidden state features corresponding to the vocabulary, The temporal weights corresponding to the characters are trainable parameters.

[0076] S150, extracting content features corresponding to the vocabulary according to the temporal features corresponding to the vocabulary;

[0077] Among them, content features can be understood as features that characterize the meaning of lexical content in a text lexical sequence.

[0078] In one of the optional embodiments, the temporal features corresponding to the vocabulary can be input into a preset convolutional network to extract the content features corresponding to the vocabulary.

[0079] In another optional embodiment, extracting content features corresponding to the vocabulary based on the temporal features corresponding to the vocabulary may include: extracting features of the temporal features corresponding to the vocabulary based on dynamic convolution weights to obtain content features corresponding to the vocabulary; wherein the dynamic convolution weights are generated based on nonlinear feature fusion of the temporal features corresponding to each vocabulary in the text vocabulary sequence.

[0080] Exemplarily, for each word in the text vocabulary sequence, the initial convolution kernel corresponding to the word can be determined according to the time series features corresponding to the word; the initial convolution kernels corresponding to each word are fused (e.g., weighted summation, etc.) to obtain the dynamic convolution weight corresponding to the text vocabulary sequence; the dynamic convolution weight corresponding to the text vocabulary sequence and the time series features corresponding to each word in the text vocabulary sequence are fused to obtain a content feature set containing the content features corresponding to each word. For example, an initial convolution kernel suitable for the city type is generated for the word "Beijing", and an initial convolution kernel suitable for the character type is generated for the word "Zhang San". Finally, the initial convolution kernel suitable for the city type and the initial convolution kernel suitable for the character type are fused to obtain the dynamic convolution weight. The advantage of this setting is that, through the dynamic convolution weight, the hierarchical information of the time series features corresponding to different words can be adapted, thereby improving the adaptability and versatility of the dynamic convolution weight.

[0081] In a specific implementation, the initial convolution kernel corresponding to the vocabulary can be determined by the following formula (1-3):

[0082] (1-3)

[0083] in, Represents the initial convolution kernel corresponding to word i, represents the temporal features corresponding to word i, W represents the initial convolution weight, and b represents the bias vector. The initial convolution weight and the bias vector are both trainable parameters. ReLU ( ) is an activation function that introduces nonlinear features.

[0084] In a specific implementation, the dynamic convolution weight corresponding to the text vocabulary sequence can be determined by the following formula (1-4):

[0085] (1-4)

[0086] Among them, K final Dynamic convolution weights corresponding to the characterization of text vocabulary sequences, Represents the initial convolution kernel corresponding to the i-th word, w i Represents the convolution kernel weight corresponding to the i-th word in the text vocabulary sequence, and n represents the total number of words in the text vocabulary sequence.

[0087] In a specific implementation, the content feature set may be determined by the following formula (1-5):

[0088] (1-5)

[0089] Among them, V cnn Characterize the content feature set, V biLSTMCharacterize the temporal features corresponding to the text word sequence, K final Characterize the dynamic convolution weights corresponding to the text word sequence. Among them, the time series features corresponding to the text word sequence , Represents the temporal features corresponding to the i-th word. Among them, the content feature set , Represents the content features corresponding to the i-th word.

[0090] S160, predicting the part of speech of the corresponding word according to the semantic features, temporal features and content features corresponding to each word in the text word sequence.

[0091] In an optional embodiment, for each word, the semantic features, temporal features and content features corresponding to the word are fused to obtain the fused features of the word; the fused features of each word are input into the conditional random field to predict the part of speech of the corresponding word. Exemplarily, the part of speech of the predicted word can be the probability of different candidate parts of speech to which the word belongs, and the final part of speech of the corresponding word is determined from each candidate part of speech according to the probability.

[0092] The above-mentioned part-of-speech prediction method obtains a text vocabulary sequence and extracts the semantic features corresponding to each vocabulary in the text vocabulary sequence; wherein the text vocabulary sequence includes at least two vocabulary words; for each vocabulary word, according to the semantic features corresponding to the vocabulary word, extract the hidden state features of the vocabulary word at the current time step; according to the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary word, generate the cumulative incremental memory features at the current time step; according to the cumulative incremental memory features at the current time step and the hidden state features corresponding to the vocabulary word, determine the time series features corresponding to the vocabulary word; according to the time series features corresponding to the vocabulary word, extract the content features corresponding to the vocabulary word; according to the semantic features, time series features and content features corresponding to each vocabulary word in the text vocabulary sequence, predict the part of speech of the corresponding vocabulary word. In the above-mentioned technical scheme, by introducing the cumulative incremental memory features to extract the time series features corresponding to the vocabulary word, the omission of key information in the time series features is avoided, thereby improving the richness and comprehensiveness of the time series features. Furthermore, content features are extracted based on temporal features, and the part of speech of the corresponding vocabulary is comprehensively predicted based on the temporal features, content features and semantic features of the vocabulary, so that the obtained part of speech can fully consider the lexical information of different dimensions carried in the text vocabulary from the semantic dimension, temporal dimension and content dimension, thereby improving the richness and diversity of the extracted features. Therefore, the part of speech of the corresponding vocabulary is comprehensively predicted based on the temporal features, content features and semantic features of the vocabulary, thereby improving the accuracy of vocabulary prediction.

[0093] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the step of determining the cumulative increment memory feature in S130 is refined.

[0094] Referring to the cumulative incremental memory feature determination steps shown in FIG2 , the steps include:

[0095] S210, generating a new memory feature at the current time step based on the accumulated incremental memory feature at the previous time step and the hidden state feature of the vocabulary.

[0096] In one of the optional embodiments, for each word, the time step corresponding to the word is taken as the current time step, and the accumulated incremental memory features of the previous time step are determined; the accumulated incremental memory features of the previous time step and the hidden state features of the word are weightedly fused, and the weighted fusion result is taken as the newly added memory features of the current time step.

[0097] In another optional embodiment, generating a new memory feature at the current time step based on the accumulated incremental memory features at the previous time step and the hidden state features of the vocabulary can include: determining the context-related hidden features of the vocabulary based on the association degree data between the context vocabulary and the vocabulary in the text vocabulary sequence, and the hidden state features of the vocabulary; generating a new memory feature at the current time step based on the context-related hidden features of the vocabulary and the accumulated incremental memory features at the previous time step.

[0098] Among them, the association degree data is used to characterize the correlation between a word and its corresponding context word; the context-related hidden features are used to characterize the hidden features related to the word carried in the mined context words; the newly added memory features are used to characterize the newly added non-forgotten features, that is, memory features, as time goes by.

[0099] Exemplarily, for each word, the context word corresponding to the word can be selected from the text word sequence according to the preset context length; and the correlation degree data between the word and the context word corresponding to the word can be determined according to the hidden state feature corresponding to the word and the hidden state feature corresponding to the context word. The matrix dimension of the correlation degree data is related to the preset context length, and the context length can be customized.

[0100] In a specific implementation, the following formula (1-6) may be used to determine the correlation degree data between different words:

[0101] A t =similarity(h t ,h j ) (1-6)

[0102] Among them, h t Represents the tth word x t The corresponding hidden state feature; h jis the tth word x t The corresponding context word x j The corresponding hidden state feature, A t For vocabulary x t With vocabulary x j The degree of association data between them; similarity() is the preset similarity determination function.

[0103] Exemplarily, for each word, the context word of the word can be fused with the word (such as dot product or cosine similarity, etc.), and the fusion result can be used as the context-related hidden feature of the word; the correlation degree data and the hidden state feature of the word can be fused to obtain the context-related hidden feature of the word.

[0104] Exemplarily, the context-related hidden features of the vocabulary and the accumulated incremental memory features at the previous time step may be weightedly fused, and the weighted fusion result may be used as the newly added memory features at the current time step.

[0105] In a specific implementation, the newly added memory feature at the current time step is determined by the following formula (1-7):

[0106] (1-7)

[0107] Among them, f(M t-1， h t， A t ) represents the vocabulary x at the current time step t The corresponding new memory feature, M t-1 Characterizes the cumulative incremental memory characteristics of the previous time step, h t Representation vocabulary x t The hidden state features of A t Represents the degree of association between a word in a text word sequence and its context word, W m Characterize the first weight matrix, W m’ represents the second weight matrix, b represents the bias vector; σ( ) is a preset new memory determination function. Among them, the first weight matrix, the second weight matrix and the bias vector are all trainable parameters.

[0108] S220, obtaining the cumulative incremental memory feature at the current time step according to the cumulative incremental memory feature at the previous time step and the newly added memory feature at the current time step.

[0109] In an optional embodiment, feature weighted fusion is performed on the accumulated incremental memory features at the previous time step and the newly added memory features at the current time step, and the feature weighted fusion result is used as the accumulated incremental memory features at the current time step.

[0110] In a specific implementation, the cumulative incremental memory feature at the current time step is determined by the following formula (1-8):

[0111] (1-8)

[0112] Among them, M t Characterizes the cumulative incremental memory characteristics at the current time step, f(M t-1， h t， A t ) represents the vocabulary x at the current time step t The corresponding new memory feature, M t-1 Characterizes the cumulative incremental memory characteristics of the previous time step, h t Hidden state features representing vocabulary, A t Represents the word x in the text word sequence t The data on the degree of association between the vocabulary and the context vocabulary of the vocabulary, α represents the memory feature weight as a trainable parameter.

[0113] The embodiment of the present application introduces cumulative incremental memory features, which can more robustly extract the cumulative incremental memory features at the current time step, avoid omission of key information in the feature extraction process, and improve the accuracy of the cumulative incremental memory features at the current time step, thereby improving the accuracy of subsequent determination of the corresponding temporal features of the vocabulary.

[0114] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the semantic feature determination step in S110 is refined.

[0115] Referring to the semantic feature determination step shown in FIG3 , the steps include:

[0116] S310, extracting linear features of each word in the text word sequence under different grammatical dimensions, and obtaining basic grammatical vectors under corresponding grammatical dimensions.

[0117] In an optional embodiment, word embedding processing can be performed on the text vocabulary sequence to obtain the initial word vector corresponding to each vocabulary; a mapping matrix of different grammatical dimensions can be obtained; for each initial word vector, the initial word vector and the mapping matrix of different grammatical dimensions can be fused to obtain the basic grammatical vector under the corresponding grammatical dimension.

[0118] Exemplarily, performing word embedding processing on a text word sequence to obtain an initial word vector corresponding to each word may include: inputting the text word sequence into a word embedding model to generate an initial word vector corresponding to each word. For example, the text "I went to Beijing" includes the word "I", the word "went" and the word "Beijing", that is, the text word sequence X={x 1， x2， x3}={I went to Beijing}. Then the text word sequence is input into the word embedding model to generate the word vector e1 corresponding to the word x1 "I", the word vector e2 corresponding to the word x2 "went", and the word vector e3 corresponding to the word x3 "Beijing", that is, the initial word vector E={e 1， e 2， e3}.

[0119] In a specific implementation, the basic grammar vectors under different grammar dimensions are determined by the following formula (1-9):

[0120] (1-9)

[0121] in, Represents the i-th word x i The basic grammar vector in the kth grammar dimension, M k Represents the mapping matrix under the kth grammatical dimension, e i Represent the word vector corresponding to the i-th word.

[0122] S320, for each word, according to the basic grammatical vectors of the word in different grammatical dimensions and the basic grammatical vectors of the context words of the word in the text word sequence in the corresponding grammatical dimension, generate the context features of the word in the corresponding grammatical dimension.

[0123] In one of the optional embodiments, for each word, the basic grammatical vector of the context words of the word in the text word sequence under the corresponding grammatical dimension is obtained; the basic grammatical vectors of the word under different grammatical dimensions and the basic grammatical vectors of the context words corresponding to the word under the corresponding grammatical dimension are subjected to feature weighted fusion; and the result of the feature weighted fusion is used as the context feature of the word under the corresponding grammatical dimension.

[0124] In another optional embodiment, generating context features of words in corresponding grammatical dimensions based on basic grammatical vectors of words in different grammatical dimensions and basic grammatical vectors of context words in a text lexical sequence in corresponding grammatical dimensions may include: determining the attention weight of context words on words in corresponding grammatical dimensions based on basic grammatical vectors of words in different grammatical dimensions and basic grammatical vectors of context words in a text lexical sequence in corresponding grammatical dimensions; for each grammatical dimension, weighting the basic grammatical vectors of words in corresponding grammatical dimensions based on the attention weight of context words on words in the grammatical dimension to obtain the context features of words in the corresponding grammatical dimension.

[0125] Exemplarily, the following method can be used to determine the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension based on the basic grammatical vectors of the vocabulary in different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence in the corresponding grammatical dimension: for each vocabulary, obtain the basic grammatical vector of the context vocabulary corresponding to the vocabulary in the text vocabulary sequence in the corresponding grammatical dimension; for each grammatical dimension, determine the dimension score of the context vocabulary to the vocabulary in the grammatical dimension based on the basic grammatical vector of the vocabulary in the grammatical dimension and the basic grammatical vector of the context vocabulary in the grammatical dimension; determine the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension based on the proportion of the dimension score under the grammatical dimension to the dimension scores under each grammatical dimension.

[0126] Optionally, determining the dimensional score of the context vocabulary to the vocabulary in the grammatical dimension based on the basic grammatical vector of the vocabulary in the grammatical dimension and the distance between the basic grammatical vectors of the context vocabulary in the grammatical dimension may include: determining the dimensional score of the context vocabulary to the vocabulary in the grammatical dimension based on the basic grammatical vector of the vocabulary in the grammatical dimension and the distance between the basic grammatical vectors of the context vocabulary in the grammatical dimension; or setting a scoring function according to preset rules, and determining the dimensional score of the context vocabulary to the vocabulary in the grammatical dimension through the preset scoring function, etc. The present application does not impose any limitation on this.

[0127] Exemplarily, the following method is used to determine the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension based on the proportion of the dimension score under the grammatical dimension to the dimension score under each grammatical dimension: determine the score and value of the context vocabulary to the vocabulary in each grammatical dimension; use the proportion of the dimension score under the grammatical dimension to the score and value as the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension.

[0128] In a specific implementation, the context word x can be determined by the following formula (1-10): j For vocabulary x i Attention weights under corresponding grammatical dimensions:

[0129] (1-10)

[0130] in, Represents the i-th word x i The context vocabulary x j The attention weight under the k-th grammatical dimension, Represents the basic grammatical vector of the i-th word in the k-th grammatical dimension, Representation of the jth word x jThe basic grammar vector under the kth grammar dimension, D represents the number of dimensions of the grammar dimension.

[0131] Exemplarily, for each grammatical dimension, a weighted summation may be performed on the attention weight of the context vocabulary on the vocabulary in the grammatical dimension and the basic grammatical vector of the vocabulary in the grammatical dimension, and the weighted summation result may be used as the context feature of the vocabulary in the grammatical dimension.

[0132] In a specific implementation, the contextual features of the vocabulary under the grammatical dimension can be determined by the following formula (1-11):

[0133] (1-11)

[0134] in, Represents the i-th word x i The contextual features under this grammatical dimension, Represents the i-th word x i The context vocabulary x j The attention weight under the k-th grammatical dimension, Representation of the jth word x j The basis grammar vector at the kth grammar dimension.

[0135] S330, generating semantic features corresponding to the vocabulary according to the context features of the vocabulary in different grammatical dimensions and the basic grammatical vector of the vocabulary.

[0136] In an optional embodiment, for each grammatical dimension, feature weighted fusion is performed on the context features of the vocabulary in the grammatical dimension and the basic grammatical vector of the vocabulary, and the result of the feature weighted fusion is used as the dimensional feature of the vocabulary in the grammatical dimension; for each vocabulary, feature weighted fusion is performed on all dimensional features of the vocabulary to obtain the semantic feature corresponding to the vocabulary.

[0137] In a specific implementation, the dimensional features of vocabulary under different grammatical dimensions can be determined by the following formula (1-12):

[0138] (1-12)

[0139] in, Represents the i-th word x i Dimensional features under the kth grammatical dimension, Represents the i-th word x i The contextual features under the kth grammatical dimension, Represents the i-th word x iIn the basic grammatical vector under the kth grammatical dimension, λ represents the dimension weight as a trainable parameter, which is used to control the mixing ratio of the vocabulary's own information and the vocabulary's corresponding context information.

[0140] In a specific implementation, the semantic features corresponding to the vocabulary can be determined by the following formula (1-13):

[0141] (1-13)

[0142] in, Represents the i-th word x i The corresponding semantic features, Represents the i-th word x i Dimensional features under the kth grammatical dimension, α k The grammatical weight representing the kth grammatical dimension is a trainable parameter.

[0143] In this embodiment, by introducing the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence under the corresponding grammatical dimension, the semantic feature generation process includes not only the semantic features of the vocabulary itself, but also the semantic information of the vocabulary under different grammatical dimensions, as well as the context information of the context corresponding to the vocabulary under different grammatical dimensions, thereby improving the accuracy and completeness of the generated semantic features and further increasing the diversity and richness of the semantic features corresponding to the vocabulary.

[0144] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the part-of-speech prediction step in S160 is refined.

[0145] Referring to FIG. 4 , the part-of-speech prediction step includes:

[0146] S410, for each word, the semantic feature, temporal feature and content feature corresponding to the word are fused to obtain a fusion feature of the word.

[0147] In an optional embodiment, for each word, a weighted fusion of the semantic features, temporal features and content features corresponding to the word is performed, and the weighted fusion result is used as the fusion feature of the word.

[0148] In a specific implementation, the fusion feature of the vocabulary is determined by the following formula (1-14):

[0149] (1-14)

[0150] Among them, v i Representation vocabulary x i The fusion characteristics of Representation vocabulary xi The semantic features of Representation vocabulary x i The timing characteristics of Representation vocabulary x i , α represents the first fusion weight, β represents the second fusion weight, and γ represents the third fusion weight. The first fusion weight, the second fusion weight, and the third fusion weight are all trainable parameters.

[0151] S420, constructing a semantic graph of the text vocabulary sequence according to the fusion features of different vocabulary in the text vocabulary sequence.

[0152] In an optional embodiment, for each word in a text vocabulary sequence, the fusion feature of the word is used as a graph node in a semantic graph; based on the fusion features of different graph nodes, the edge weights between the graph nodes are determined; and based on the graph nodes and edge weights, a semantic graph of the text vocabulary sequence is constructed.

[0153] In a specific implementation, the edge weights between the graph nodes are determined by the following formula (1-15):

[0154] (1-15)

[0155] Among them, a(v i ,v j ) represents the word x i Corresponding graph nodes and vocabulary x j The edge weights between corresponding graph nodes, Representation vocabulary x i The transposed vector corresponding to the fused feature, v j Representation vocabulary x j Corresponding to the fusion feature, d represents the dimension of the fusion feature, and W represents the graph weight as a trainable parameter.

[0156] S430, predicting the part of speech of each word in the text word sequence according to the semantic graph.

[0157] In an optional embodiment, the part of speech of each word in the text word sequence is predicted based on the semantic graph in the following manner: the semantic graph is input into a graph convolutional network to obtain the part of speech of each word in the text word sequence.

[0158] In a specific embodiment, the part of speech of each word in the text word sequence is predicted by the following formula (1-16):

[0159] V fusion =GCN(H) (1-16)

[0160] Among them, GCN () represents the graph convolutional network, V fusionA part-of-speech matrix representing each word in the text vocabulary sequence, and H represents the semantic graph of the text vocabulary sequence. Among them, the semantic graph can be represented by a feature matrix, and the size of the feature matrix is the number of words and the dimension of the word vector.

[0161] Exemplarily, a semantic graph is constructed based on the relationship between "Zhang San", "went to", and "Beijing", and the deep semantics in these relationships are extracted through a graph convolutional network. For example, there may be a semantic association of traveling between "Zhang San" and "Beijing", and the model can more accurately determine whether "Zhang San" is relevant to "Beijing", thereby improving the recognition accuracy of "Zhang San" as a personal name and "Beijing" as a place name.

[0162] In this embodiment, for each word, the semantic features, temporal features, and content features corresponding to the word are fused to obtain the fused features of the word, preserving the independence of the features corresponding to the word in multiple dimensions. Furthermore, based on the semantic graph constructed from the fused features corresponding to different words in the text vocabulary sequence, the part-of-speech of each word in the text vocabulary sequence is predicted, accurately capturing the complex relationships and semantic information between the words, avoiding the loss of word information caused by simple concatenation, and further improving the accuracy of the predicted part-of-speech of the words.

[0163] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps does not have a strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages, and these steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps in other steps.

[0164] Based on the same inventive concept, an embodiment of the present application also provides a part-of-speech prediction device for implementing the above-mentioned part-of-speech prediction method. The implementation solution provided by this device to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the part-of-speech prediction device provided below can refer to the limitations on the part-of-speech prediction method in the above text, and will not be repeated here.

[0165] In an exemplary embodiment, as shown in Figure 5, a part-of-speech prediction device is provided, including: a first extraction module 510, a second extraction module 520, a first generation module 530, a first determination module 540, a third extraction module 550, and a prediction module 560, where:

[0166] The first extraction module 510 is used to obtain a text word sequence and extract semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0167] The second extraction module 520 is used to extract the hidden state feature of the word at the current time step according to the semantic feature corresponding to the word for each word;

[0168] A first generating module 530, for generating a cumulative incremental memory feature at a current time step according to the cumulative incremental memory feature at a previous time step and the hidden state feature of the vocabulary;

[0169] A first determination module 540 is used to determine the time series feature corresponding to the vocabulary according to the accumulated incremental memory feature at the current time step and the hidden state feature corresponding to the vocabulary;

[0170] The third extraction module 550 is used to extract the content features corresponding to the vocabulary according to the temporal features corresponding to the vocabulary;

[0171] The prediction module 560 is used to predict the part of speech of the corresponding word according to the semantic features, temporal features and content features corresponding to each word in the text word sequence.

[0172] In one embodiment, the first generation module 530 includes: a newly added memory feature generation unit, which is used to generate a newly added memory feature at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary; a cumulative incremental memory feature generation unit, which is used to obtain the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the newly added memory features at the current time step.

[0173] In one of the embodiments, the newly added memory feature generation unit is also used to determine the context-related hidden features of the vocabulary based on the correlation degree data between the context vocabulary and the vocabulary in the text vocabulary sequence, and the hidden state features of the vocabulary; and generate the newly added memory features at the current time step based on the context-related hidden features of the vocabulary and the accumulated incremental memory features at the previous time step.

[0174] In one embodiment, the first extraction module 510 includes: a basic grammatical vector determination unit, which is used to extract the linear features of each word in the text vocabulary sequence at different grammatical dimensions to obtain the basic grammatical vectors at the corresponding grammatical dimensions; a context feature determination unit, which is used to generate the context features of the word at the corresponding grammatical dimension for each word according to the basic grammatical vectors of the word at different grammatical dimensions and the basic grammatical vectors of the context words of the word in the text vocabulary sequence at the corresponding grammatical dimension; and a semantic feature determination unit, which is used to generate the semantic features corresponding to the word according to the context features of the word at different grammatical dimensions and the basic grammatical vectors of the word.

[0175] In one of the embodiments, the context feature determination unit is further used to determine the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension based on the basic grammatical vectors of the vocabulary in different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence in the corresponding grammatical dimension; for each grammatical dimension, the basic grammatical vector of the vocabulary in the corresponding grammatical dimension is weighted according to the attention weight of the context vocabulary to the vocabulary in the grammatical dimension to obtain the context feature of the vocabulary in the corresponding grammatical dimension.

[0176] In one embodiment, the third extraction module 550 is also used to extract the temporal features corresponding to the vocabulary according to the dynamic convolution weights to obtain the content features corresponding to the vocabulary; wherein the dynamic convolution weights are generated by nonlinear feature fusion of the temporal features corresponding to each vocabulary in the text vocabulary sequence.

[0177] In one of the embodiments, the prediction module 560 is further used to fuse the semantic features, temporal features and content features corresponding to each word to obtain the fusion features of the word; construct a semantic graph of the text word sequence based on the fusion features of different words in the text word sequence; and predict the part of speech of each word in the text word sequence based on the semantic graph.

[0178] Each module in the above-mentioned part-of-speech prediction device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0179] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a part-of-speech prediction method is implemented.

[0180] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0181] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0182] Obtaining a text word sequence and extracting semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0183] For each word, extract the hidden state features of the word at the current time step based on the semantic features corresponding to the word;

[0184] Generate the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary;

[0185] Determine the time series features corresponding to the vocabulary according to the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary;

[0186] Extracting content features corresponding to the vocabulary based on the temporal features corresponding to the vocabulary;

[0187] According to the semantic features, temporal features and content features corresponding to each word in the text word sequence, the part of speech of the corresponding word is predicted.

[0188] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the accumulated incremental memory features at the previous time step and the hidden state features of the vocabulary, a newly added memory feature at the current time step is generated; based on the accumulated incremental memory features at the previous time step and the newly added memory features at the current time step, the accumulated incremental memory features at the current time step are obtained.

[0189] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining the context-related hidden features of the vocabulary based on the correlation degree data between the context vocabulary and the vocabulary in the text vocabulary sequence, and the hidden state features of the vocabulary; generating the newly added memory features at the current time step based on the context-related hidden features of the vocabulary and the accumulated incremental memory features at the previous time step.

[0190] In one embodiment, when the processor executes the computer program, the following steps are also implemented: extracting the linear features of each word in the text vocabulary sequence at different grammatical dimensions to obtain the basic grammatical vectors at the corresponding grammatical dimensions; for each word, generating the contextual features of the word at the corresponding grammatical dimension based on the basic grammatical vectors of the word at different grammatical dimensions and the basic grammatical vectors of the context words of the word in the text vocabulary sequence at the corresponding grammatical dimension; generating the semantic features corresponding to the word based on the contextual features of the word at different grammatical dimensions and the basic grammatical vectors of the word.

[0191] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the basic grammatical vectors of the vocabulary in different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence in the corresponding grammatical dimension, the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension is determined; for each grammatical dimension, according to the attention weight of the context vocabulary to the vocabulary in the grammatical dimension, the basic grammatical vector of the vocabulary in the corresponding grammatical dimension is weighted to obtain the contextual features of the vocabulary in the corresponding grammatical dimension.

[0192] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the dynamic convolution weight, the temporal features corresponding to the vocabulary are extracted to obtain the content features corresponding to the vocabulary; wherein the dynamic convolution weight is generated according to the nonlinear feature fusion of the temporal features corresponding to each vocabulary in the text vocabulary sequence.

[0193] In one embodiment, when the processor executes the computer program, the following steps are also implemented: for each word, the semantic features, temporal features and content features corresponding to the word are fused to obtain the fusion features of the word; based on the fusion features of different words in the text word sequence, a semantic graph of the text word sequence is constructed; based on the semantic graph, the part of speech of each word in the text word sequence is predicted.

[0194] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0195] Obtaining a text word sequence and extracting semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0196] For each word, extract the hidden state features of the word at the current time step based on the semantic features corresponding to the word;

[0197] Generate the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary;

[0198] Determine the time series features corresponding to the vocabulary according to the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary;

[0199] Extracting content features corresponding to the vocabulary based on the temporal features corresponding to the vocabulary;

[0200] According to the semantic features, temporal features and content features corresponding to each word in the text word sequence, the part of speech of the corresponding word is predicted.

[0201] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary, a newly added memory feature at the current time step is generated; based on the cumulative incremental memory features at the previous time step and the newly added memory features at the current time step, the cumulative incremental memory features at the current time step are obtained.

[0202] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: determining the context-related hidden features of the vocabulary based on the association degree data between the context vocabulary and the vocabulary in the text vocabulary sequence, and the hidden state features of the vocabulary; generating the newly added memory features at the current time step based on the context-related hidden features of the vocabulary and the accumulated incremental memory features at the previous time step.

[0203] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: extracting linear features of each word in a text vocabulary sequence at different grammatical dimensions to obtain basic grammatical vectors at the corresponding grammatical dimensions; for each word, generating contextual features of the word at the corresponding grammatical dimension based on the basic grammatical vectors of the word at different grammatical dimensions and the basic grammatical vectors of the context words of the word in the text vocabulary sequence at the corresponding grammatical dimension; generating semantic features corresponding to the word based on the contextual features of the word at different grammatical dimensions and the basic grammatical vectors of the word.

[0204] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the basic grammatical vectors of the vocabulary in different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence in the corresponding grammatical dimension, the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension is determined; for each grammatical dimension, the basic grammatical vector of the vocabulary in the corresponding grammatical dimension is weighted according to the attention weight of the context vocabulary to the vocabulary in the grammatical dimension to obtain the contextual features of the vocabulary in the corresponding grammatical dimension.

[0205] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: according to the dynamic convolution weight, the temporal features corresponding to the vocabulary are extracted to obtain the content features corresponding to the vocabulary; wherein the dynamic convolution weight is generated according to the nonlinear feature fusion of the temporal features corresponding to each vocabulary in the text vocabulary sequence.

[0206] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: for each word, the semantic feature, temporal feature and content feature corresponding to the word are fused to obtain the fusion feature of the word; according to the fusion features of different words in the text word sequence, a semantic graph of the text word sequence is constructed;

[0207] According to the semantic graph, predict the part of speech of each word in the text word sequence.

[0208] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0209] Obtaining a text word sequence and extracting semantic features corresponding to each word in the text word sequence; wherein the text word sequence includes at least two words;

[0210] For each word, extract the hidden state features of the word at the current time step based on the semantic features corresponding to the word;

[0211] Generate the cumulative incremental memory features at the current time step based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary;

[0212] Determine the time series features corresponding to the vocabulary according to the accumulated incremental memory features at the current time step and the hidden state features corresponding to the vocabulary;

[0213] Extracting content features corresponding to the vocabulary based on the temporal features corresponding to the vocabulary;

[0214] According to the semantic features, temporal features and content features corresponding to each word in the text word sequence, the part of speech of the corresponding word is predicted.

[0215] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the cumulative incremental memory features at the previous time step and the hidden state features of the vocabulary, a newly added memory feature at the current time step is generated; based on the cumulative incremental memory features at the previous time step and the newly added memory features at the current time step, the cumulative incremental memory features at the current time step are obtained.

[0216] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: determining the context-related hidden features of the vocabulary based on the association degree data between the context vocabulary and the vocabulary in the text vocabulary sequence, and the hidden state features of the vocabulary; generating the newly added memory features at the current time step based on the context-related hidden features of the vocabulary and the accumulated incremental memory features at the previous time step.

[0217] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: extracting linear features of each word in a text vocabulary sequence at different grammatical dimensions to obtain basic grammatical vectors at the corresponding grammatical dimensions; for each word, generating contextual features of the word at the corresponding grammatical dimension based on the basic grammatical vectors of the word at different grammatical dimensions and the basic grammatical vectors of the context words of the word in the text vocabulary sequence at the corresponding grammatical dimension; generating semantic features corresponding to the word based on the contextual features of the word at different grammatical dimensions and the basic grammatical vectors of the word.

[0218] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the basic grammatical vectors of the vocabulary in different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence in the corresponding grammatical dimension, the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension is determined; for each grammatical dimension, the basic grammatical vector of the vocabulary in the corresponding grammatical dimension is weighted according to the attention weight of the context vocabulary to the vocabulary in the grammatical dimension to obtain the contextual features of the vocabulary in the corresponding grammatical dimension.

[0219] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: according to the dynamic convolution weight, the temporal features corresponding to the vocabulary are extracted to obtain the content features corresponding to the vocabulary; wherein the dynamic convolution weight is generated according to the nonlinear feature fusion of the temporal features corresponding to each vocabulary in the text vocabulary sequence.

[0220] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: for each word, the semantic features, temporal features and content features corresponding to the word are fused to obtain the fusion features of the word; based on the fusion features of different words in the text word sequence, a semantic graph of the text word sequence is constructed; based on the semantic graph, the part of speech of each word in the text word sequence is predicted.

[0221] It should be noted that the data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0222] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0223] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0224] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for predicting part of speech, characterized in that: The method comprises: Acquire a text vocabulary sequence, and extract semantic features corresponding to each vocabulary in the text vocabulary sequence; wherein the text vocabulary sequence includes at least two vocabulary; For each word, according to the semantic features corresponding to the word, extract the hidden state features of the word at the current time step; Generate the cumulative incremental memory feature at the current time step according to the cumulative incremental memory feature at the previous time step and the hidden state feature of the vocabulary; Determine the time series feature corresponding to the vocabulary according to the accumulated incremental memory feature at the current time step and the hidden state feature corresponding to the vocabulary; Extracting content features corresponding to the vocabulary according to the temporal features corresponding to the vocabulary; The part of speech of the corresponding word is predicted according to the semantic features, temporal features and content features corresponding to each word in the text word sequence.

2. The method according to claim 1, characterized in that The step of generating the cumulative incremental memory feature at the current time step according to the cumulative incremental memory feature at the previous time step and the hidden state feature of the vocabulary comprises: Generate a newly added memory feature at the current time step according to the accumulated incremental memory feature at the previous time step and the hidden state feature of the vocabulary; The accumulated incremental memory feature at the current time step is obtained according to the accumulated incremental memory feature at the previous time step and the newly added memory feature at the current time step.

3. The method according to claim 2, characterized in that The step of generating the newly added memory feature at the current time step according to the accumulated incremental memory feature at the previous time step and the hidden state feature of the vocabulary includes: Determine the context-related hidden features of the vocabulary according to the data of the degree of association between the context vocabulary of the vocabulary in the text vocabulary sequence and the vocabulary, and the hidden state features of the vocabulary; A newly added memory feature at the current time step is generated according to the context-related hidden features of the vocabulary and the accumulated incremental memory features at the previous time step.

4. The method according to any one of claims 1 to 3, characterized in that: The extracting of semantic features corresponding to each word in the text word sequence includes: Extracting linear features of each word in the text vocabulary sequence under different grammatical dimensions to obtain basic grammatical vectors under corresponding grammatical dimensions; For each word, according to the basic grammatical vectors of the word in different grammatical dimensions and the basic grammatical vectors of the context words of the word in the text word sequence in the corresponding grammatical dimension, generate the context features of the word in the corresponding grammatical dimension; The semantic features corresponding to the vocabulary are generated according to the context features of the vocabulary in different grammatical dimensions and the basic grammatical vectors of the vocabulary.

5. The method according to claim 4, characterized in that The step of generating context features of the vocabulary under the corresponding grammatical dimension according to the basic grammatical vectors of the vocabulary under different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence under the corresponding grammatical dimension comprises: Determine the attention weight of the context vocabulary to the vocabulary in the corresponding grammatical dimension according to the basic grammatical vectors of the vocabulary in different grammatical dimensions and the basic grammatical vectors of the context vocabulary of the vocabulary in the text vocabulary sequence in the corresponding grammatical dimension; For each grammatical dimension, the basic grammatical vector of the vocabulary in the corresponding grammatical dimension is weighted according to the attention weight of the context vocabulary to the vocabulary in the grammatical dimension to obtain the context feature of the vocabulary in the corresponding grammatical dimension.

6. The method according to any one of claims 1 to 3, characterized in that: The step of extracting content features corresponding to the vocabulary according to the temporal features corresponding to the vocabulary includes: Extracting the temporal features corresponding to the vocabulary according to the dynamic convolution weights to obtain the content features corresponding to the vocabulary; The dynamic convolution weight is generated according to the nonlinear feature fusion of the temporal features corresponding to each word in the text word sequence.

7. The method according to any one of claims 1 to 3, characterized in that: The predicting the part of speech of the corresponding vocabulary according to the semantic features, temporal features and content features corresponding to each vocabulary in the text vocabulary sequence includes: For each word, the semantic features, temporal features and content features corresponding to the word are fused to obtain the fusion features of the word; Constructing a semantic graph of the text vocabulary sequence according to the fusion features of different words in the text vocabulary sequence; The part of speech of each word in the text word sequence is predicted according to the semantic graph.

8. A part-of-speech prediction device, characterized in that: include: A first extraction module is used to obtain a text vocabulary sequence and extract semantic features corresponding to each vocabulary in the text vocabulary sequence; wherein the text vocabulary sequence includes at least two vocabulary; A second extraction module is used to extract, for each word, the hidden state feature of the word at the current time step according to the semantic feature corresponding to the word; A first generating module, used for generating the cumulative incremental memory feature at the current time step according to the cumulative incremental memory feature at the previous time step and the hidden state feature of the vocabulary; A first determination module is used to determine the time series feature corresponding to the vocabulary according to the accumulated incremental memory feature at the current time step and the hidden state feature corresponding to the vocabulary; A third extraction module, used to extract content features corresponding to the vocabulary according to the temporal features corresponding to the vocabulary; The prediction module is used to predict the part of speech of the corresponding word according to the semantic features, temporal features and content features corresponding to each word in the text word sequence.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images