150 results about "Word representation" patented technology

Systems and processes for unified language modeling are provided. In accordance with one example, a method includes, at an electronic device with one or more processors and memory, receiving a character of a sequence of characters and determining a current character context based on the received character of the sequence of characters and a previous character context. The method further includes determining a current word representation based on the current character context and determining a current word context based on the current word representation and a previous word context. The method further includes determining a next word representation based on the current word context and providing the next word representation.

The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.

The present disclosure generally relates to integrated text conversion and prediction. In an example process, a current character input of a first writing system is received. A first current character context in the first writing system is determined based on the current character input and a first previous character context in the first writing system. A second current character context in a second writing system is determined based on the first current character context, a second previous character context in the second writing system, and a character representation in the second writing system. A current word context in the second writing system is determined based on the second current character context, a previous word context in the second writing system, and a word representation in the second writing system. Based on the current word context, a probability distribution over a word inventory in the second writing system is determined.

The invention provides a text classification method and device based on transfer learning, and the method comprises the steps: S1, employing an unlabeled text to train a BERT model, and obtaining a pre-trained word representation BERT model; S2, filtering links, forwarding symbols and user name contents in the to-be-classified text; S3, inputting the filtered text into the word representation BERTmodel trained in the step S1 to obtain a semantic file of the text; and S4, inputting the semantic file of the text into a convolutional neural network for processing to obtain a category label of the sentence in the semantic file. According to the method, transfer learning is used for text classification, and a BERT word representation model trained based on large-scale unlabeled corpora is provided. The word representation model has universality, does not depend on the specific text field, and can also be used for other tasks such as entity extraction and sentiment analysis.

The invention provides a fine-grained word representation model-based sequence labeling model, which is used for performing a sequence labeling task, and belongs to the field of computer application and natural language processing. The structure of the model is mainly composed of three parts including a feature representation layer, a BiLSTM layer and a CRF layer. When the sequence labeling task is performed by utilizing the model, firstly an attention mechanism-based character level word representation model Finger is proposed for fusing morphological information and character information ofwords; secondly the Finger and a BiLSTM-CRF model finish the sequence labeling task jointly; and finally a result with F1 of 91.09% is obtained in a CoNLL 2003 data set in end-to-end and no any feature engineering forms by a method. An experiment shows that the designed Finger model remarkably improves the recall rate of a sequence labeling system, so that the model identification capability is remarkably improved.

A system and method for speech recognition includes a speaker-independent set of stored word representations derived from speech of many users deemed to be typical speakers and for use by all users, and may further include speaker-dependent sets of stored word representations specific to each user. The speaker-dependent sets may be used to store custom commands, so that a user may replace default commands to customize and simplify use of the system. Utterances from a user which match stored words in either set according to the ordering rules are reported as words.

The invention relates to a paper title generation method capable of utilizing distributed semantic information, and belongs to the field of natural language processing. The paper title generation method comprises the following steps of: firstly, utilizing a TextRank algorithm to obtain the first k keywords of a paper abstract, training a GloVe (Global Vector for word representation) model to obtain a word vector, and carrying out vector initialization on the extracted keywords; then, utilizing a recurrent neural network title based on a long short-term memory unit to obtain the title; and finally, carrying out title construction. A deep learning method is used for mining the deep semantic information of the title, and the generated title exhibits high readability and conforms to the semantic rule of the title.

The invention discloses a video description method based on multi-feature fusion. The method is characterized in that (1) deep spatial-temporal features of a video are extracted by fusing traditional CNN features and SIFT flow features; (2) an S2VT sentence generation model with added average pooling features being video overall features is adopted to generate corresponding sentence description according to the deep spatial-temporal features extracted in the step (1); and (3) a word2vec word vector method is adopted to replace one-hot vector word representation to optimize the sentence generation model in the step (2). The method has the advantages that more robust spatial-temporal features can be better extracted through multi-feature fusion; meanwhile, the average pooling features are added into the sentence generation model, so that more relations are established between visual information and words; finally, the word2vec word vector method is adopted to replace one-hot vector word representation, so that more relations are established between words, and video description performance is effectively improved.

The invention belongs to the technical field of computer word representation, and discloses a deep dynamic context word representation model and method. The deep dynamic context word representation model is a masking language model formed by stacking multi-layer bidirectional Transformer encoders with layers of attention mechanisms. The network is a multi-layer neural network, and each layer of the network captures context information of each word in the input statement from different angles. Then different weights are given to each layer of the network through a layer attention mechanism. Finally, different word representations are synthesized according to the weights to form context representations of the words. Words generated by using the model represent three tasks of logic reasoning(MultiNLI), named entity recognition (CoNLL2003) and reading understanding (SQuAD) on a public data set, and the tasks are improved by 2.0%, 0.47% and 2.96% respectively compared with an existing model.

The invention relates to a social network short text recommendation method based on a word meaning topic model, which comprises the following specific steps of fusing the word representation learningbased on context attention mechanism of word meaning and synonym information into social network short text recommendation so as to enrich word level characteristics of a text; fusing the dirichlet multi-item mixed distribution short text topic modeling based on word meaning representation into social network short text recommendation so as to enrich text level characteristics; modeling the potential interest degree and tendency degree of the user evolved along with time by combining the social network user relationship, the short text topic characteristics of the user-related text based on word meaning representation and the potential relationship characteristics between the user and the text; predicting the potential tendency of the user to the text through a parameter estimation method,and selecting the text with the maximum tendency to recommend to the user to realize short text recommendation. According to the method, the word meaning information is fused into the short text topic modeling and the social network short text recommendation task, so that the accuracy of the social network short text recommendation task is improved.

The invention discloses a target sentiment analysis method and system based on an attention gated convolutional network, and the method comprises the steps: step 1, inputting a given context word vector and a corresponding target word vector, and enabling the given context word vector and the corresponding target word vector to serve as inputs for training; step 2, performing multi-head attentionmechanism interaction by utilizing the context words and the context sensing target words; step 3, enabling the sentiment feature vectors cintra and tinter generated by two channels to respectively pass through a gating convolution mechanism to generate a context word representation ai and a context word representation ui with context perception target word representations; step 4, pooling the emotion feature oi, and selecting the most representative feature; step 5, performing full connection on the pooled feature word vectors, and then performing classification through a Softmax classifier;and step 6, training and updating the attention gated convolutional network model by minimizing the cross entropy loss function. The accuracy can be effectively improved, the convergence time can be shortened, and the practicability is higher.

The invention discloses a method and device for determining an association relationship between entities, and equipment and a storage medium, and is applied to the technical field of natural languageprocessing, knowledge maps and deep learning. According to the specific implementation scheme, the method comprises the steps of obtaining target association information, wherein the target association information comprises a first word representing a first entity, a second word representing a second entity and a third word representing a target association relationship; determining a first semantic feature according to the target association relationship and similar text segments for the target association information in a preset text segment library; determining a description text for the target associated information according to the target associated information and a preset knowledge graph; determining a second semantic feature according to the target associated information and the description text; and according to the first semantic feature and the second semantic feature, determining a confidence coefficient of a target association relationship between the first entity and thesecond entity.

The invention provides an entity question answering method and device based on a neural network and a terminal, wherein the method comprises converting a word contained in a question and a candidate document into a word vector, and generating a corresponding question word vector sequence and a candidate document word vector sequence; inputting the problem word vector sequence and the candidate document word vector sequence into the long-short-term memory network model, and outputting the word coding sequence of the problem and the candidate document; matching the word encoding sequence of theproblem and the word encoding sequence of the candidate document to generate a candidate document representation based on matching information, wherein the candidate document representation comprisesa plurality of word representations; selecting start and end words from all word representations and generating entity answers based on the start and end words. The method reduces explicit computationand cumulative error, effectively utilizes semantic representation between questions and documents, and improves positioning accuracy of entity answers.

The invention provides a text attribute word sentiment classification method based on a deep learning network. The text attribute word sentiment classification method comprises the steps of obtaininga word vector corresponding to a target sentence in a text; inputting the word vector into a hidden information extraction network model to obtain a hidden state vector; extracting first syntax information in a syntax dependency tree corresponding to the target sentence based on the hidden state vector and the first syntax extraction neural network; based on the hidden state vector and a second syntax extraction neural network, extracting second syntax information in a local syntax dependency tree corresponding to the target sentence; denoising the first syntax information and the second syntax information to obtain context representation and attribute word representation; averaging and pooling the context representation and the attribute word representation, and then splicing to obtain feature representation corresponding to the target sentence; and inputting the feature representation into an emotion classification function to obtain an emotion classification result. Compared with the prior art, the relation between the target sentence and the syntax information and the relation between the attribute words and the syntax information are fully considered, and the accuracy of sentiment classification is improved.

The invention relates to a Mian language emotion analysis method and a device based on transfer learning, and belongs to the technical field of natural language processing. The method comprises the following steps: firstly, carrying out cross-language words representation on Burmese vocabularies to realize mapping from a Burmese text to an English text semantic space; pre-training an English emotion classification model based on a CNN and attention mechanism network; learning cross-language emotion features by sharing neural network layer parameters of the English emotion classification model,and migrating the cross-language emotion features into the Mian language emotion classification model to realize Mian language emotion classification; and carrying out model optimizing by using marked Mian language data to finally realize Mian language emotion classification. According to the Mian language emotion analysis device based on transfer learning, effective emotion analysis is achievedon Mian language sentences, and the problem of poor performance caused by lack of Mian language emotion mark data, is solved.

The invention provides a visual language task processing system, a visual language task processing method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The system comprises a target encoder, a text encoder and a text decoder, the target encoder and the text encoder are respectively connected with the text decoder, and the target encoder is used for inputting a predetermined image; performing coding processing on the predetermined image to obtain a target representation sequence; outputting a target representation sequence; the text encoder is used for inputting text description; encoding the text description to obtain a word representation sequence; outputting a word representation sequence; the text decoder is used for inputting a target representation sequence and a word representation sequence; decoding the target representation sequence and the word representation sequence to obtain a multi-modal representation sequence; and outputting a multi-modal representation sequence, wherein the multi-modal representation sequence is used for processing the visual language task. The system can improve the accuracy of processing the visual language task to a certain extent.

The invention provides a resume data information analysis and matching method and device, electronic equipment and a medium. According to the method, the retrieved resume can be preprocessed; obtaining a resume to be parsed, constructing a word segmentation directed acyclic graph according to a pre-constructed word segmentation dictionary so as to segment the resume to be analyzed; the word segmentation result of the to-be-analyzed resume can be quickly obtained; obtaining resume text, further constructing a co-occurrence matrix according to the resume text; determining keywords of the resumetext based on a co-occurrence matrix; obtaining a character sequence in the keyword; performing word representation processing on the character sequence by utilizing a word representation model, wordrepresentation of the character sequence is obtained. According to the method, the word representation is input into the resume label analysis model to obtain the resume label sequence, and the similarity between each label in the resume label sequence and the label of each post is further calculated to determine the resume matched with each post, so that the post and the resume can be quickly, accurately and intelligently matched.

The invention provides a remote supervision relation extraction method with entity perception based on a PCNN model. The method specifically comprises the following steps: combining word embedding with head entity and tail entity embedding and relative position embedding by using a multi-head self-attention mechanism to generate enhanced word semantic representation of a perceptible entity, whichcan capture semantic dependence between each word and an entity pair; introducing a global door, and combining the enhanced word representation perceived by each entity in the input sentence with theaverage value of the enhanced word representations to form a final word representation input by the PCNN, and in addition, in order to determine a key sentence segment in which the most important relationship classification information appears. According to the method, another gate mechanism is introduced, and different weights are allocated to each sentence segment, so the effect of key sentencesegments in the PCNN is highlighted. Experiments show that the remote supervision relationship extraction method provided by the invention can improve the prediction capability of the remote supervision relationship in the sentence.

The invention discloses a method and system for analyzing potential theme phrases of text data, and the method comprises the steps: collecting a text data set, carrying out the word segmentation of the text data set, and obtaining the word representation form of the text data set; extracting effective phrases formed after word matching according to the words in the text data set, and obtaining a mixed representation form of the words which are not matched into the effective phrases and the phrase set; carrying out word vector training on the text data set in the mixed representation form to obtain a corresponding word vector model; constructing DR-Phrase LDA and solving each parameter; training the DR-Phrase LDA, and outputting potential theme phrases of the text data according to a training result. According to the invention, a phrase topic model based on word vectors is adopted; according to the model, statistics information of phrases in model training is reasonably improved by means of a Chinese language law in probability topic model training, specifically, a word vector method is adopted for measuring the relation between phrase component words, the semantic relation of the words in text integrity and phrase local is quantitatively reflected, and the model precision is higher.

The invention discloses a text classification method and device. The method comprises the steps of preprocessing a text to obtain a to-be-classified text; inputting the to-be-classified text into a target depth classification model, the target depth classification model being determined by training and learning a training sample with an identified viewpoint category; performing vectorization representation on the to-be-classified text according to the target depth classification model, determining a word representation vector, a sentence representation vector and an article representation vector of the to-be-classified text; and classifying the to-be-classified text based on the word representation vector, the sentence representation vector and the article representation vector, and determining a classification result of the to-be-classified text, the classification result at least comprising a viewpoint category. The technical problem that the actual viewpoint category of the text cannot be accurately determined due to the fact that a traditional text classification scheme is poor in classification effect is solved.

The invention provides an implicit discourse relationship analysis method based on a semantic-syntax dual-path graph convolutional network, and belongs to the technical field of natural language processing application. The method comprises the following steps that of firstly, conducting word segmentation is conducted on corpora; constructing a word list and the like for preprocessing, and then initializing word vectors by utilizing GloVe; and respectively inputting the two initialized arguments into Bi-LSTM codes to obtain word representations of the Bi-LSTM codes fused with the context information. T; using the Bi-LSTM output is used to subsequently initialize representations of graph nodes in a binary channel graph convolutional network. , wWherein the dual-path graph convolutional network comprises two GCN networks, extracting corresponding features from the interaction structure graph and the syntax structure graph respectively, and splicing different features obtained by two pathsfor classification. And ; updating parameters in the model to convergence by adopting an Adam algorithm, and completing the analysis of the implicit discourse relationship by using the parameter withthe optimal performance.