320 results about "Word embedding" patented technology

The invention provides a full-text retrieval method and system fusing various types of documents, and relates to the technical field of information retrieval, and the method comprises the following steps: obtaining document representation through document content extraction and structure recognition, generating a cross-modal semantic vector by using word embedding and nonlinear transformation, constructing a hierarchical index and a cross-document association graph, and obtaining a full-text retrieval result; the basic correlation score is calculated after the query request is received, and the comprehensive score of the candidate content segments is calculated based on the association graph to determine the optimal retrieval result, so that unified representation and retrieval of heterogeneous documents are realized, the cross-document retrieval precision and relevance are improved, and the processing capability of a retrieval system on complex queries is enhanced.

The invention provides a global and local feature fusion SMPL-X action-to-text generation method, and belongs to the field of artificial intelligence. The method comprises the following steps: preprocessing and coding an input SMPL-X action sequence into a double-flow action feature; through a cross-modal mapping module, the double-flow action features are mapped to a pre-trained large language model through independent projection branches, and global conditions and local action prefix embedding are obtained; through a text generation module, a decoder of a pre-trained large language model is used as a trunk network, text cue word embedding is extracted based on a text instruction given by a user, local action prefix embedding and text cue word embedding are spliced and then input into the decoder, and global conditions are injected into each layer of the decoder through a cross attention mechanism. And generating a description text in an autoregression mode. According to the method, the description text which is consistent with action semantics and has sufficient details can be stably and accurately generated, and the generation stability and the cross-scene applicability are improved when disturbance exists in the action sequence.

The invention relates to the field of intelligent report generation, in particular to an intelligent insight report generation method and system based on an AI large model, and the method comprises the steps: inputting an insight demand, and generating an insight data package comprising insight contents, associated data and industry labels; extracting a basic keyword set of the insight data packet to form a mixed feature code; after mixed feature coding preprocessing, weight distribution is carried out; and after weight distribution of the AI large model, injecting a high-weight feature vector into a semantic understanding core layer, injecting a low-weight feature vector into a logical reasoning layer, and outputting analysis data to form an intelligent insight report. According to the method, word embedding parameters are optimized according to field semantic characteristics, a parameter verification mechanism is introduced, field text characteristics are adapted by adjusting vector dimensions, context windows and low-frequency vocabulary filtering threshold values, window parameter validity is verified through cosine similarity, word frequency threshold value reasonability is verified through standard deviation, and field text characteristic matching is achieved. And it is ensured that the feature vectors can accurately capture domain-specific semantics.

The embodiment of the invention provides an address data matching method and related equipment, and belongs to the technical field of geographic information services. The method comprises the following steps: constructing an address annotation corpus according to input address information data and a preset address database; the method comprises the following steps: generating a geographic information embedding vector according to a preset geographic information knowledge graph, performing address element analysis in combination with an address annotation corpus to obtain an address element sequence so as to construct a dictionary tree, and performing similarity screening through a spatial hierarchical matching algorithm to obtain a similar address set; generating an address embedding vector matrix through a preset word embedding vector model, and performing feature extraction through a preset semantic feature extraction model to obtain semantic-level similar features; according to input address information data, multi-dimensional character similarity matching is carried out to obtain character-level similar features, then weighted fusion is carried out in combination with semantic-level similar features, and target matching address data is determined according to a weighted fusion result. According to the embodiment of the invention, the address data matching accuracy and efficiency can be improved.

The present invention facilitates communication between sign language users and machines by translating sign language and text using AI models, deep learning computer vision, and word embeddings. Users interact via sign language, captured and processed through deep learning and NLP modules. The system converts sign language videos into text, constructs coherent sentences, and generates contextually appropriate responses using a Retrieve and Generate (RAG) model. Responses are translated back into sign language videos, spelling out words not found in the dictionary. If requested, a human agent can respond. Key features include high-accuracy recognition, context-aware response generation, dynamic vocabulary updates, and optional human interaction. The method ensures efficient processing with LLM, embedding techniques, and deep learning, optimizing translation accuracy and user experience. The system adapts to multiple languages and dialects by training on specific sign languages, making it applicable globally.

The application discloses an application method of a Transformer architecture based on a biological regulation mechanism, S1, original data is divided into basic units token, and then each discrete token is mapped into a continuous high-dimensional vector through a word embedding layer; S2, a prototype activation unit identifies an initial semantic core from the input embedded token, and scores an activation value based on the embedded vector of the token; S3, a multi-scale prototype aggregation module performs multi-scale pattern aggregation on the prototype candidate, and forms a prototype vector across tokens; S4, a regulation path diverter determines a main / auxiliary path distribution strategy of each token according to the context semantic weight and the prototype responsiveness of the token; S5, the main path performs a standard QKV calculation process; S6, a biological regulation loop module performs feedback regulation on the auxiliary path token by using the aggregated prototype vector in the S3 step; S7, two token expression streams generated by the main path and the auxiliary path are integrated, so that the final output of the Transformer layer is generated.

A method initializes and accelerates training of neural network based large language model, including by: (i) accessing a corpora for training a neural-network based large language model having word embeddings and word projections in respective word embedding and word projection layers and at least one hidden layer; (ii) counting raw token frequencies associated with content within the corpora; (iii) smoothing the raw token frequencies into a series of vector norms based on log or scaled log functions parameterized by maximum norm information; and (iv) injecting vector norm information into word embeddings and / or word projections based on norm-angle reparameterization to prepare the large language model for training.

The invention provides an embedded layer access collaborative acceleration system and method of a neural network model, a processing unit layer, a control layer and a routing layer cooperatively work, and the control layer can predict a candidate item of a next lexical element before a processing unit by querying a lexical element statistical table. A routing layer reads word embedding vectors of candidate lexical elements from a memory bank in advance and stores the word embedding vectors in a prefetching buffer area corresponding to the memory bank, and after a processing unit inferes a next lexical element by using a neural network model, if the next lexical element hits the candidate lexical element, the next lexical element is not hit by the candidate lexical element. If not, the word embedding vector of the hit candidate lexical element is read from the prefetching buffer area, and compared with the mode that the processing unit inferes the next lexical element and then reads the corresponding word embedding vector from the storage bank, lexical element prefetching processing is increased, and the access speed of an embedding layer of the neural network model can be increased.

The invention discloses a big and small model collaborative natural language processing method, system and device and a medium, which are applied to the field of language processing, and the method comprises the following steps: performing data preprocessing on to-be-processed text data to obtain word embedding vector data; obtaining the semantic complexity of the word embedding vector data and the input text length of the to-be-processed text data, and calculating the task complexity according to the semantic complexity and the input text length; splitting the text processing task into a plurality of sub-tasks according to the task complexity; according to a preset task complexity threshold, dynamically allocating each sub-task to the large model and the small model for processing to obtain a text reasoning result of each sub-task; and fusing the text reasoning results to obtain a text processing result. According to the method, by dynamically distributing the tasks to the large and small models and fusing the reasoning results of the large and small models, the reasoning efficiency and precision are effectively balanced, the accuracy and the real-time performance of the natural language processing tasks are improved, and meanwhile the dynamic adaptability of the system is enhanced.

The invention relates to an assembly type bridge modeling method based on a large model and an RAG technology. The method comprises the steps that a bridge standard component library is established based on an assembly type bridge standard image set; calling a large language model, and generating standardized knowledge statements based on the bridge standard component library; a word embedding method is adopted to convert the normalized knowledge statement into vectors, and the vectors are stored in a vector database; converting an input natural statement query into a vector by adopting a word embedding method, and retrieving in a vector database to obtain a normalized knowledge statement related to the query; performing parameter analysis and optimization on the retrieved knowledge statements based on a large language model by utilizing an RAG technology so as to generate component parameters; and calling a component modeling algorithm based on each component parameter to generate a component three-dimensional model, and assembling the component three-dimensional model to obtain a fabricated bridge model. According to the method, component parameter sorting, knowledge statement intelligent combination, efficient semantic retrieval, parameter optimization, three-dimensional model automatic generation and the like are organically integrated, and the modeling efficiency and quality are improved.

The invention relates to the technical field of computer vision and geographic information, and particularly discloses a cross-view geographic positioning acceleration method based on semantic description, which comprises the following steps of: performing instruction fine tuning on an input query image and a database image by adopting a visual language model to generate natural language description; coding the natural language description into a low-dimensional semantic vector by utilizing a word embedding model; constructing a semantic keyword association graph based on a graph embedding technology, and fusing a general semantic knowledge base and a geographic domain ontology to construct a surface feature special semantic tree; screening candidate subsets according to a preset ground feature priority and a saliency detection result of the query image; extracting global features and local key point descriptors of the images in the candidate subsets by adopting a lightweight model; constructing a visual similarity calculation model based on the state space model, and modeling a feature sequence dependency relationship; and fusing the semantic matching score and the visual similarity, outputting a final matching result by adopting a weighted sorting strategy, and cooperatively accelerating according to the matching result in combination with a lightweight model.

The invention belongs to the technical field of internet security, and particularly relates to a malicious task script detection method and system based on federal map learning, and the method comprises the steps: obtaining a PowerShell script, and carrying out the preprocessing of the PowerShell script, and obtaining a standardized abstract syntax tree; constructing a directed acyclic command flow graph based on the normalized abstract syntax tree, obtaining an adjacent matrix according to the command flow graph, and performing word embedding on the command flow graph to obtain a node feature matrix; performing GAT training on the node feature matrix and the adjacent matrix to obtain vector representation of the command flow graph, inputting the vector representation of the command flow graph into a preset federal learning detection model for detection, judging whether the script is a malicious script or not, outputting a judgment result, and deeply analyzing the script structure and semantics by constructing an abstract syntax tree and the command flow graph to obtain the malicious script. According to the method, wider confusion technologies and attack techniques including attacks without specific keywords can be detected, and the problem that attackers update the attack methods and cannot effectively detect the attacks is avoided.

The invention relates to the technical field of computers, in particular to an enterprise private data standardization preprocessing system based on knowledge graph driving, and the method comprises a preprocessing module which is used for carrying out cleaning and standardization preprocessing on collected enterprise private text data to generate standardization text data; the word embedding generation module is used for training a word embedding model based on the standardized text data and generating word vector representation of the text; the named entity recognition module is used for recognizing a preset category of named entities in the standardized text data; the triple extraction module is used for executing open type information extraction on the standardized text data to obtain a relation triple; the entity deduplication module is used for performing deduplication resolution on entities in the relation triple extraction result; the relation clustering module is used for carrying out relation type classification clustering on the relation triple subjected to duplicate removal processing; and the graph construction module is used for constructing a knowledge graph according to the classified and clustered relation triples. According to the method, the problems of entity ambiguity, relation redundancy and unstable atlas construction caused by scattered sources, heterogeneous formats, high noise and repetition and mixed use of alias / pronouns of enterprise private data in the prior art can be solved.

Provided in the present application is a large-language-model-based long text generation method capable of realizing context compression. The method comprises: acquiring context text to be compressed and prompt text, and performing compression-based encoding processing, so as to obtain a corresponding compression vector and a prompt embedding vector; concatenating the compression vector and the prompt embedding vector, and performing autoregression-based decoding processing on a fused feature, which is obtained by means of concatenation, so as to obtain a plurality of corresponding token identifiers; and on the basis of a preset vocabulary, mapping the token identifiers into text character strings one by one, and combining the text character strings into compressed context text. By means of the present application, long context text processed by a large language model is compressed, thereby solving the technical problem in the prior art of a semantic model consuming a large amount of model calculation resources and data storage resources when processing long context text.

The invention discloses a customs commodity classification method and system based on a text sequence and graph information, and relates to the technical field of natural language processing. Cleaning the commodity declaration text, and segmenting the text into a plurality of declaration elements; performing word segmentation on the declaration elements by using a pre-trained LERT model and generating context-aware sub-word embedding vectors; inputting the sub-word embedding vector into a bidirectional long short-term memory network BiLSTM to obtain bidirectional context representation; initializing an element importance score based on the attention weight, and optimizing and screening key elements through a key element recognition mechanism KEIM; constructing a full connection graph with adjustable edge weight and a chain graph reflecting sequence dependence; inputting the graph structure and the node features into a graph attention network GAT to generate a structure feature vector; and splicing the sequence features and the structural features, and predicting HS codes through a full connection layer and Softmax to realize customs commodity classification. By fusing the text sequence and the graphic information and introducing a key element recognition mechanism, the understanding ability of customs commodity description texts is remarkably improved.

The invention belongs to the technical field of computer information retrieval, and discloses a technology transfer-oriented term extension retrieval method and device and electronic equipment, and the method comprises the following steps: carrying out keyword extension on query input by a user; determining a plurality of target windows according to a co-occurrence relationship between the extended keyword set and technical terms in different time windows in a patent database; respectively extracting Top-N terms with most frequent co-occurrence from each window, and generating a period constraint vector by using a word embedding model; generating a group of candidate terms under the constraint of the period constraint vector corresponding to each target window according to user input query; according to the distribution of the candidate terms in each time window, calculating a time importance score and a time span degree score of the candidate terms; and according to the time importance score, the time spanning degree score and the semantic similarity score, performing retrieval after screening the candidate terms. According to the method, the problem of time sequence semantic fault existing in an existing retrieval method is solved.

The invention relates to the field of computer vision and artificial intelligence, in particular to a multi-modal fusion document image classification method based on multi-branch deep convolution and hierarchical semantic modeling. According to the method, visual and text information of a document is co-processed through a double-flow architecture; a visual semantic feature of a document image is efficiently extracted from a visual flow by adopting a lightweight convolution operation and multi-scale feature fusion mechanism; an original text is obtained from a text stream through an optical character recognition technology, text error correction and word embedding extraction are carried out through a fine tuning language model, word-level and sentence-level context modeling is realized by inputting a hierarchical semantic coding network, and document-level features are generated. And high-precision document image classification is realized by fusing the double-flow feature vectors and combining a full-connection layer. Experimental results show that the method has an excellent multi-modal feature fusion effect, relatively high classification precision and strong robustness to complex document scenes in a document image classification task, and is suitable for application scenes of various document image classification.

The application provides a text classification method based on combination of bidirectional long short-term memory model and knowledge graph retrieval, which retrieves relevant prior supporting facts from the knowledge graph according to the task by using an attention mechanism, and incorporates the prior supporting facts into a feature space together with features learned from training data to classify the text. It firstly generates a word embedding model of a sentence by using a GloVe tool, and then respectively puts the word embedding model into a knowledge graph retrieval module and a bidirectional long short-term memory network BiLSTM, and then splices the output of the retrieval model and the output of the BiLSTM model to obtain a final classification. Compared with a traditional method, the accuracy is obviously improved by using the method of the knowledge graph. Finally, the model is evaluated on a 20Newsgroups text classification data set, and the experimental results prove the effectiveness of the model.

The present application relates to a kind of based on large model fault detection method, belong to big data field.The method of the present application carries out fault detection using improved model, and the improved model includes: encoder-dyLLM, sound embedding module and word embedding module, wherein the improved model includes the following processes: text and sound data are segmented into n sections according to timestamp, in the kth inference, the kth text data paragraph is generated by word embedding module Text vector, the kth sound data is generated by sound embedding module Sound vector, the jth encoder-dyLLM inference output buffer vector memj, text vector, sound vector and buffer vector are concatenated, input into encoder-dyLLM inference, complete the kth improved model inference;The classification result output by encoder-dyllama includes: normal, IO fault and system fault.The present application realizes the fusion of multiple data by reforming decoder module, introducing global cache, introducing sound, ECC data, and improves the robustness of model for fault detection.

According to one embodiment, a document classification apparatus includes a processing circuit. The processing circuit is configured to: acquire text content for each of logical elements for semi-structured document data including text data stored for each of the logical elements; select logical elements from the logical elements and generating logical element sets each including the logical elements; analyze text contents for the respective logical element sets and constructing respective word embedded spaces; select a first word embedded space and a second word embedded space including a common word shared with the first word embedded space from the word embedded spaces, and update the first word embedded space based on similarity to the common word in the second word embedded space; and output a classification result of the document data using the first word embedded space and embedding information of a feature quantity of a classification target.

A Cybersecurity Event Handling Processor (CEHP) and method for processing security alerts includes: a File System containing a Universal Target Schema (UTS) of target language representations (UTS JSONs); a Normalizer running Feature Extraction and Word Embeddings algorithms; a Tree Converter; and a Transformer running linguistic and structural matching algorithms. The CEHP: (a) captures threat events in one or more native formats generated by cybersecurity tools; (b) runs Feature Extraction and Word Embeddings algorithms for tokenization and categorization of the captured events to create normalized events; (c) converts the normalized events into trees and then translates the trees into event representations in JSON (or XML) format (Event JSONs); and (d) runs nearest neighbor and / or linguistic and structural matching algorithms to compare the Event JSONs to the UTS JSONs to generate output JSONs (Translation JSONs) from the UTS corresponding to the captured events.

The application relates to the field of artificial intelligence and digital medicine, and proposes an intention recognition model training method and device based on artificial intelligence, an electronic device and a storage medium. The intention recognition model training method based on artificial intelligence comprises the following steps: performing word embedding on a plurality of natural sentences collected in advance to obtain a sentence vector of each natural sentence; identifying an intention vector and a slot vector of the sentence vector by using a preset semantic coding model; labeling the intention vector and the slot vector to construct a training data set; constructing an initial intention recognition model, training the initial intention recognition model by using the training data set, and obtaining an intention recognition model. The method can jointly train the intention recognition model by using the intention vector and the slot vector of the natural sentence, so that the accuracy of the intention recognition model can be improved.

A system and method utilizes masked language models in order to provide data-type detection, such as (but not limited to) prediction of columnar headings. Two masked language models are pre-trained on example columnar text. One model predicts missing data at the entity level (e.g., masked entity names that may be made up of whole words), while the other predicts missing data at the character level (e.g., masked individual characters). The table with missing column headings is fed into both models, and the output is contextual word embeddings and contextual character embeddings. These results are merged, and then fed into a neural network classifier to then predict the column names.

The application relates to an intelligent relic material parameter reasoning method and system based on a multi-modal large model, the method comprising the following steps: acquiring a high-resolution digital image and structured archive text data of a to-be-tested relic, performing standardization preprocessing on the image, and performing word segmentation, stop word removal and word embedding coding on the structured archive text data; adopting a visual encoder and a text encoder of a multi-modal pre-training large model to respectively extract a visual feature vector and a text feature vector; realizing semantic alignment of the two types of features through a cross-attention mechanism to generate a cross-modal fusion feature vector; inputting the fusion feature into a material parameter prediction head network constructed by a multi-layer perception mechanism to obtain SVBRDF material parameters; and outputting SVBRDF material parameter reasoning results through parameter analysis and physical constraint processing. The method can fuse visual and semantic information of the relic, and realize efficient and accurate SVBRDF material parameter reasoning in a non-contact manner, thereby supporting digital protection and high-fidelity virtual display of large-scale relics.

The application belongs to the technical field of natural language processing, and particularly relates to an aspect-level sentiment classification method based on a graph attention network, which comprises the following steps: obtaining word embedding representation of context text in which an aspect word is located; dynamically adjusting the weight of a context word according to the relative position of the context word and the aspect word, and obtaining context semantic features; aggregating syntactic information through an improved graph attention network to obtain syntactic features of the text; using a deep cross network to fuse the syntactic features of the text and the context semantic features to obtain final feature representation; and performing sentiment prediction on the final feature representation through a full connection layer to obtain the sentiment polarity distribution of the aspect word in the text. The application solves the problem of feature information loss that may occur in a multilayer network of the graph attention network, and considers the position information of the context word when extracting semantic features, so that the syntactic features and the context semantic features are fully fused, thereby improving the accuracy of aspect-level sentiment classification.

The application provides an OCR text correction method and device. A correct probability of a Chinese character at each position in to-be-corrected text is obtained. A character shape structure word embedding vector of the Chinese character at each position in the to-be-corrected text is obtained. An input word embedding vector of the Chinese character at each position is calculated according to the word embedding vector corresponding to the Chinese character at each position in the to-be-corrected text, the correct probability and the character shape structure word embedding vector, and an input word embedding vector set is formed. The input word embedding vector set is input into a second encoder to form a coded input word embedding multi-dimensional vector. The coded input word embedding multi-dimensional vector is input into a correction layer to obtain corrected text. The correction layer is provided with a correction neural network formed by training of a Chinese corpus. The OCR text correction method and device provided by the application no longer selects a candidate word from an existing confusion set for correction, reduces the occurrence of missing words, and improves the accuracy of correction.

This invention proposes a text-enhanced temporal prediction method and system based on a large-scale language model word embedding space, belonging to the field of large-scale language models and integrated energy systems. The method includes: acquiring historical load time-series data, segmenting it into time-series fragments and mapping them to fragment embedding vectors; reprogramming the fragment embedding vectors using the word embedding matrix of a frozen large-scale language model to obtain temporal semantic vectors; acquiring unstructured text data and inputting it into the frozen large-scale language model for encoding to obtain word-level text semantic vectors; using the temporal semantic vectors as queries and the text semantic vectors as keys and values, performing gated cross-attention fusion through a gated cross-attention fusion module to obtain deep fusion features; and inputting the deep fusion features into a prediction head module to output the final prediction result. This invention achieves deep co-spatial fusion of temporal numerical values ​​and text semantics, improving prediction accuracy and interpretability.

The application provides a training method and device of an intent recognition model, and an intent recognition method and device. The method comprises: obtaining a training sample set; a sample in the training sample set comprises a question sentence labeled with an intent label; a pre-training model is trained using the sample in the training sample set; an initial output vector corresponding to the sample is obtained; an initial back propagation gradient of the pre-training model is determined according to the initial output vector; a preset number of disturbances are added to the initial output vector based on the initial back propagation gradient to obtain a target back propagation gradient; and a model parameter of the pre-training model is updated according to the target back propagation gradient to obtain an intent recognition model. The application uses the gradient of the back propagation in the pre-training model to perform adversarial disturbance on the output vector corresponding to the model sample, i.e. the word embedding layer vector. This adversarial training method can improve the robustness of the model.

This invention relates to a big data topic analysis method based on an embedding model. First, the Sentence-BERT model is used to perform sentence embedding representation on preprocessed Chinese text data. Then, the UMAP projection dimensionality reduction algorithm is used to reduce the dimensionality of the embedded vectors. Next, the HDBSCAN clustering algorithm is used to cluster the dimensionality-reduced vectors. Based on the assignment of each Chinese text in the target Chinese dataset to a corresponding topic class, the Chinese words with the highest c-TF-IDF scores are selected to represent each topic class. Finally, the DSG model is used to perform word embedding representation on the topic words, calculating the similarity between different topic words and between different topic classes, thereby detecting the volatility of newly emerging topic classes. The entire scheme design has higher topic consistency and topic diversity, and can detect new hot topics in a timely and accurate manner, providing early warnings.

Dynamic content tags are generated as content is received by a dynamic content tagging system. A natural language processor (NLP) tokenizes the content and extracts contextual N-grams based on local or global context for the tokens in each document in the content. The contextual N-grams are used as input to a generative model that computes a weighted vector of likelihood values that each contextual N-gram corresponds to one of a set of unlabeled topics. A tag is generated for each unlabeled topic comprising the contextual N-gram having a highest likelihood to correspond to that unlabeled topic. Topic-based deep learning models having tag predictions below a threshold confidence level are retrained using the generated tags, and the retrained topic-based deep learning models dynamically tag the content.