907 results about "Word embedding" patented technology

The invention discloses a retrieval generation method and system based on multi-agent collaboration, a terminal and a medium, and relates to the field of artificial intelligence. Performing semantic analysis on the input word embedding converted by the natural language query instruction through a query analysis agent, and determining a semantic intention vector; performing reinforcement learning and meta learning on the semantic intention vector through a strategy construction agent, and determining a retrieval strategy; performing semantic enhancement on the semantic intention vector according to knowledge graph node embedding to obtain a semantic enhancement vector; determining a data channel according to the semantic enhancement vector, a retrieval strategy and a real-time system load, and calling the data channel for retrieval to obtain candidate documents; and generating a target answer according to each candidate document based on an adaptive reflection feedback mechanism in combination with an auto-encoder and a generative adversarial network. The problems that the prior art depends on a fixed retrieval strategy, has limitation when facing complex query, multi-round interaction and cross-modal data fusion, is easily interfered by noise and is not accurate enough in semantic matching are effectively solved.

The invention discloses a retrieval enhancement generation method based on token-level word embedding, and relates to the technical field of artificial intelligence and natural language processing. The method specifically comprises the following steps of: segmenting a text block, namely segmenting a document into paragraphs; enhancing the document information; the text coding is used for converting a text into a numerical vector; segmenting the problem to obtain a character array, encoding the character array, and converting the problem into a numerical vector; calculating the similarity between the problem and the text block by adopting the average maximum similarity to obtain a related text block; performing Prompt construction on the question and the related text block to obtain a prompt text; the prompt text is input into a large language model LLMs, and question answers conforming to the answer format are output; and in combination with the question answers and the related text blocks, quoting and marking the question answers to obtain answers with quotations. According to the method, a token-level text embedding method is utilized to vectorize questions and text blocks, so that the accuracy of RAG retrieval is improved, and the confidence of LLMs answers is increased.

The invention provides a track prediction method and device based on a pre-trained large language model, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring historical flight path data of a target aircraft and carrying out data preprocessing to obtain a flight path sequence; constructing a cue word, and inputting the cue word into a pre-trained large language model to obtain a word embedding representation; the cue word comprises track prediction task description and track data description; encoding the track sequence to obtain an encoded track sequence and a track position sequence, and performing word segmentation on the track position sequence; constructing a space-time diagram of the track sequence based on the encoded track sequence, and extracting space-time dependent feature representation of the track sequence; fusing the word embedding representation and the space-time dependent feature representation by using a pre-trained large language model to generate a track information feature representation; and inputting the track information feature representation into a trained prediction module to obtain predicted values of the longitude, the latitude and the height of the target aircraft at the future moment.

The invention provides a financial fraud detection method based on a large language model. The method comprises the steps of obtaining a to-be-recognized text; performing word segmentation on the to-be-recognized text through the target word segmentation tool and the financial fraud dictionary, and determining a fraud sensitive word list; calculating the weight of each sensitive word in the sensitive word list according to a target algorithm to obtain a sensitive word weight feature vector; inputting the sensitive word weight feature vector and a to-be-recognized text into a large language model, and determining a context semantic vector of the sensitive word in combination with a word embedding technology; determining the similarity between the context semantic vector of the sensitive word and a preset financial fraud type semantic vector; and determining a financial fraud type according to the similarity. Through the implementation of the method, the generalization ability of the pre-training model is utilized to capture text deep semantics, priori knowledge is injected in combination with a sensitive word weight mechanism, the model is guided to focus high-risk vocabularies, the defect of a traditional method in semantic comprehension is overcome, the financial fraud recognition rate is increased, and the omission ratio is reduced.

The invention discloses a word embedding vector extraction method and system based on a large language model, and belongs to the field of natural language processing, and the word embedding vector extraction method based on the large language model comprises the following steps: S1, embedding a global context attention module, generating a Q / K / V vector through linear transformation, and calculating a global attention weight in a cross-position manner; s2, splicing coding features of adjacent layers, and performing gating fusion to generate a global context dynamic semantic state; s3, multi-head attention joint coding is carried out on the dynamic semantic state of the current layer and the global state; s4, performing average pooling to generate a global semantic vector, and calculating a dynamic attention weight; s5, performing weighted fusion on the dynamic weight and the global vector to generate an enhanced code; s6, outputting a final word vector by a GELU nonlinear transformation full-connection layer; the method has the beneficial effects that the global context semantic capture capability is enhanced, and the expression effect of the word embedding vector is improved.

Graphic visualizations, such as charts or graphs conveying data attribute values, can be generated based on natural language queries, i.e., natural language requests. To do so, a natural language request is parsed into n-grams, and from the n-grams, word embeddings are determined using a natural language model. Data attributes for the graphic visualization are discovered in the vector space from the word embeddings. The type of graphic visualization can be determined based on a request intent, which is determined using a trained intent classifier. The graphic visualization is generated to include the data attribute values of the discovered data attributes, and in accordance with the graphic visualization type.

The invention provides a communication index prediction method based on a multi-modal large model and related equipment, relates to the technical field of data processing, and can obtain multi-dimensional time sequence data, geographic space data and corresponding text description data in a user communication process; performing time sequence feature extraction processing on the multi-dimensional time sequence data to obtain a time sequence feature vector, and performing spatial feature extraction processing on the geographic spatial data to obtain a spatial feature vector; encoding the text description data based on a large language model word embedding layer to obtain a semantic feature vector; performing cross-modal attention fusion on the three types of feature vectors based on a text prototype to generate joint feature representation; and the joint feature representation is input into the large language model to generate communication index prediction information, so that the coupling relationship among the multi-modal indexes can be effectively modeled, the prediction precision is improved, and the effectiveness of a network optimization decision is ensured.

The invention particularly relates to a document abstracting method based on domain knowledge and a multi-granularity graph network, which comprises the following steps: S1, constructing a knowledge coding module, carrying out domain knowledge graph transfer learning, extracting triple structure features in combination with a capsule network, and generating a text entity coding module; s2, constructing a knowledge fusion module, converting a document text into a semantic vector, and adaptively fusing the semantic vector with external knowledge; s3, constructing a word-level semantic graph, and aggregating neighborhood information through a semantic graph attention mechanism to obtain word embedding representation containing context; s4, constructing a sentence-level semantic graph, performing self-attention calculation on the word-level semantic vector in the S3 to obtain semantic node representation of the sentence-level semantic vector, and performing feature aggregation on sentence-level nodes to obtain global semantic information of the document; and S5, constructing an output layer, and predicting according to the document feature vector in the S4 to obtain a final extraction type abstract result. The method has the advantages that redundant information in the document is removed, and ambiguity of text semantics is eliminated.

The invention discloses a visual language alignment-based visual narrative generation method and device, electronic equipment and a storage medium, and belongs to the technical field of image understanding. The method comprises the steps of obtaining a to-be-processed image sequence and an instruction text; inputting the to-be-processed image sequence and the instruction text into a multi-modal large model to obtain a visual narrative text output by the multi-modal large model; the multi-modal large model comprises a visual encoder, a multi-modal mapper and a large language model which are connected in sequence, and further comprises a text encoder, and the output end of the text encoder is connected with the input end of the large language model; the step of inputting the to-be-processed image sequence and the instruction text into a multi-modal large model to obtain a visual narrative text output by the multi-modal large model comprises the following steps: sequentially encoding images in the to-be-processed image sequence through the visual encoder to generate visual features; sequentially projecting the visual features to a word embedding space of the large language model for semantic alignment through the multi-modal mapper to obtain a language embedding vector; through the text encoder, converting the instruction text into an instruction embedding vector; and through the large language model, performing logical reasoning according to the language embedded vector and the instruction embedded vector, and generating the visual narrative text. According to the method, visual information and language output can be effectively aligned, and the visual correlation of multi-graph narration is improved.

The invention relates to the technical field of power marketing, in particular to an AI large model-based power marketing data analysis method, which comprises the following steps of: acquiring structured data and unstructured data in a power marketing system, and generating unified coded data after space-time alignment and pre-training word embedding model processing; inputting the unified coding data into a pre-trained power field large model, and extracting static, dynamic and semantic features through multi-modal fusion, feature decoupling and semantic anchoring; constructing entity link feature pairs in combination with a power knowledge graph, and enhancing fusion feature expression through a graph attention mechanism; and finally, inputting a dynamic weight gating network, outputting an abnormal user identification tag, a demand response strategy and a customer loss early warning probability, and executing strategy optimization under specific conditions. The method can be widely applied to risk identification, strategy making and user behavior prediction tasks in power marketing.

Despite the usefulness of foundation models, they often return irrelevant or inaccurate responses to domain-specific queries, which often include technical jargon and patterns that are unique to the domain. Training data for specialized domains is not readily accessible or frequently discussed in online forums, so foundation models lack understanding in specialized domains. Moreover, while the embedding models underlying foundation models may be finetuned using annotated datasets, generating these datasets for niche and specialized domains is an extremely arduous, time-consuming, and expensive task. These issues are overcome by implementing a generative pseudo labeling (GPL) approach to creating labeled data for finetuning embedding models to recognize semantic similarities for specialized domains. In this way, the accuracy and relevance of foundation-model responses to domain-specific queries is improved.

The invention relates to a retrieval method and device based on static word embedding, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of performing word segmentation on an original text to obtain a first word segmentation result, training a static word embedding model by utilizing the first word segmentation result to obtain word vectors, and generating a synonym word library; the method comprises the following steps: establishing a full-text inverted index by utilizing an original text, and expanding query words by utilizing a synonym library in a retrieval stage; encoding the original text into a semantic vector by using a semantic generation model, and constructing a vector index based on the semantic vector; based on a to-be-queried text in a user query request, performing retrieval by using the full-text inverted index to obtain a first candidate document, and performing retrieval by using the vector index to obtain a second candidate document; performing fusion processing on the first candidate document and the second candidate document to obtain a target candidate document; and inputting the target candidate document into the text generation model to obtain a retrieval result. By adopting the method, the accuracy of text retrieval can be improved.

The invention belongs to the technical field of natural language processing, and particularly relates to a text sentiment classification method integrated with a novel adversarial strategy and enhanced feature extraction, which comprises the following steps of: carrying out preprocessing and word segmentation operation on a text; constructing a model; the embedding learning layer obtains word embedding vectors of a text through BERT and BiLSTM; the enhanced feature extraction layer performs weighted fusion on a local attention mechanism and a TextCNN model; the classification layer performs emotion polarity classification according to the extracted feature vectors to obtain a final classification result; and carrying out data enhancement by adopting a large-batch adversarial training strategy. According to the method, a local attention mechanism and TextCNN weighting are combined, and emotion features of different granularities in a text are fully mined. The local attention mechanism can identify key information in a text and endow the key information with a higher weight, and the TextCNN can capture local features and a context dependency relationship, so that compared with an existing method, the method has more advantages in the aspect of feature extraction, and text representation with more semantic representativeness is generated.

The invention provides a prediction method and device for operation parameters of a virtual power plant, a storage medium and a computer program product. Comprising the steps of performing data preprocessing on acquired influence factor data; using a time sequence decomposition algorithm to decompose the influence factor data into three-dimensional sequences including a trend component, a season component and a residual component, and segmenting each sequence into sequence blocks and mapping the sequence blocks into time sequence feature vectors; performing K-means clustering on word embedding used for pre-training the large language model, and selecting K clustering centers as semantic anchor points to be spliced with the time sequence feature vectors; finely adjusting the position embedding parameters of the large language model, the weight of the feedforward neural network in residual connection and the parameters of the normalization layer; and according to the semantic enhancement time sequence feature vector, predicting to obtain a trend component, a season component and a residual component of each dimension, and carrying out splicing and reverse normalization processing on the components to obtain a prediction result. According to the prediction method, the accuracy and real-time performance of virtual power plant load and electricity price prediction are improved.

A system is configured to: (a) analyze data records using word embeddings to produce vectors indicative of similarity scores for words or phrases in the data records relative to electronically stored reference data; (b) compute shapeley additive explanation (SHAP) values to analyze and rank contributions of the words or phrases to the similarity scores from the word embeddings; (c) use a language model trained on a vocabulary used in the data records to identify contextual insights that are not explicitly disclosed in the data to improve accuracy of the word embeddings; and (d) iteratively and dynamically adjust or update the model using human-in-the-loop based reinforcement learning from human feedback by: receiving an input representative of a modification to at least one of the contributions determined by the SHAP computation, and responsive to receiving the input, adjusting at least one of the ranks of the word embeddings in the model.

The invention relates to an intelligent energy-saving method for a large building refrigerating system based on a large language model, and belongs to the field of intelligence of large building refrigerating systems. A reinforcement learning framework is constructed, a multi-dimensional state space formed by building thermal loads, equipment operation states and thermal environment parameters is defined, and a dynamic reward mechanism is designed by taking improvement of energy efficiency, guarantee of thermal safety constraints and maintenance of thermal comfort as optimization objectives; developing a word embedding model special for the refrigeration field and a cross-modal decision conversion model, and realizing bidirectional analysis of a natural language instruction and a physical control parameter through feature coding and embedding mapping; and performing low-rank adaptive fine tuning on the pre-trained large language model based on the reinforcement learning experience set, and constructing a closed-loop verification system. Generalized migration and multi-target collaborative decision-making of equipment control strategies are achieved, the self-adaptive regulation and control capacity of a refrigeration system under dynamic loads is improved, and a universal intelligent solution for energy efficiency optimization, thermal safety and comfort is provided for a large building.

The invention relates to the technical field of natural language processing and deep learning matching recommendation, and discloses a supply and demand matching method for digital science and technology personalized services, which comprises the following steps: carrying out semantic understanding and analysis on a submitted technical long text by adopting a natural language large model, effectively extracting a core text of technical contents, and carrying out semantic analysis on the core text; and based on the text classification model, accurately delimiting industry field labels, thereby solving matching obstacles caused by cross-industry term differences. Precise extraction of clear numerical parameters and performance indexes in technical texts is realized by using a natural language large model, a refined demand text set is established, and deep semantic vectorization representation is performed through a word embedding model; through cosine similarity calculation and a screening rule based on industry labels, the matching accuracy and reliability of cross-industry technical services are greatly improved; based on an association weight mechanism of performance indexes, semantic similarity and performance index association degree are comprehensively considered, and the accuracy of matching recommendation results is optimized.

The invention discloses an active learning-based electric power knowledge graph construction method and system, and relates to the technical field of electric power market transaction data analysis, and the method comprises the steps: collecting data, cleaning the collected data, generating word prediction task data of an electric power transaction scene, and training an electric power field word embedding model; generating an entity relationship training sample, inputting a word embedding model, performing entity relationship label prediction through an entity relationship joint extraction model, extracting a high-quality sample based on active learning to perform part-of-speech tagging, and training the model based on a tagged sample; and applying the trained entity relationship joint extraction model to power transaction data to construct a knowledge graph, performing manual correction and entity unification, and storing the knowledge graph into a Neo4j database. According to the method, active learning and deep learning are combined, so that the sample labeling cost is effectively reduced, and meanwhile, the model performance is ensured.

The invention discloses an intra-domain model training method and system based on a knowledge graph, and belongs to the technical field of artificial intelligence, and the method comprises the steps: constructing a domain knowledge graph; extracting newly added high-frequency professional terms from the domain knowledge graph; generating a newly-added lexical element for each newly-added high-frequency terminology, and integrating the newly-added lexical element into a vocabulary of the pre-trained large language model to obtain an expanded vocabulary; initializing a word embedding vector of the lexical element by utilizing an entity embedding vector in the knowledge graph, adjusting a word embedding layer of the pre-training large language model based on the word embedding vector of the newly-added lexical element in the expanded word list, and carrying out retraining; taking question and answer pairs constructed based on the domain knowledge graph as an instruction fine tuning data set after being audited and perfected by experts, and performing fine tuning on the retrained large language model; and performing model performance evaluation and continuous optimization. The method can provide a more comprehensive word list, improve the understanding ability of the model for vocabularies in a specific field, improve the model training efficiency, and is suitable for large language model training in the field.

A method, computer program product, and computing system for generating a plurality of chunks for a plurality of text portions of a document, wherein the document includes the plurality of text portions and a plurality of images. Each chunk is indexed using a word embedding. Each of the plurality of images is indexed based upon, at least in part, a position of a respective image relative to a corresponding chunk. An image placeholder is generated for each of the plurality of images. A plurality of image-enhanced embeddings is generated by inserting the image placeholder for each of the plurality of images into a respective word embedding for the corresponding chunk. The plurality of image-enhanced embeddings are provided for processing a query using a generative artificial intelligence (AI) model.

The invention provides a semantic similarity analysis method for science and technology project management, and belongs to the technical field of information, the semantic similarity analysis method for science and technology project management comprises the following steps: preprocessing collected project documents, including text cleaning, word segmentation and stop word removal; extracting vocabulary features in the document, and converting the vocabulary features into numerical vector representation; calculating the semantic similarity between the documents or terms; and applying a similarity analysis result to document management and information retrieval: performing automatic preprocessing on the document, including text cleaning, word segmentation, stop word removal and standardization processing, so that a system can quickly prepare document data, prepare for subsequent analysis, automatically extract vocabulary features in the document by using an advanced word embedding model (such as Word2Vec, GloVe or BERT), and perform data processing on the document. And the information is converted into a numerical vector for representation, so that the requirement of manual intervention is greatly reduced.

The invention is applied to the technical field of image subtitle generation, and particularly discloses an image subtitle method based on an attention and state space model, which comprises the following steps: S1, constructing an image subtitle dynamic hybrid network model based on an attention and state space; according to the image subtitle method based on the attention and the state space model, an encoder and a decoder serve as a framework, a hybrid encoder is constructed, multi-modal features of an image serve as input of the encoder, an attention mechanism is adopted to capture relevance in the modal features, serialization features are extracted in combination with the state space model, and the image subtitle method based on the attention and the state space model is obtained. Fusion is carried out through a self-adaptive gating mechanism, and rich feature information is provided for the decoder; according to the method, word embedding and multi-modal feature interaction are carried out in a decoder, a dependency relationship between word embedding and multi-modal features is obtained, a dynamic fusion module is designed to realize multi-modal dynamic fusion, multi-modal heterogeneous features of an image can be fully utilized to provide richer feature information, and sentence description better conforming to human cognition is generated.

The invention provides a Text2SQL cue word index rewriting method and system for achieving retrieval enhancement and an equipment medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps that statistics is conducted on user-defined index names, calculation methods are determined and described through a natural language, and a knowledge base representing the mapping relation between the user-defined index names and the calculation methods is established; importing the mapping relation into an inverted retrieval library, constructing the user-defined index names into vectors through a word embedding model, and importing the vectors into a vector retrieval library; obtaining user input, comparing the user input with respective defined index names or performing self-defined index name matching according to a semantic query vector retrieval library; and querying the inverted retrieval library, matching a calculation method, adding the calculation method into cue words and feeding back the cue words to the large language model. According to the method, the knowledge base and the retrieval base are established through statistics and analysis of user-defined index names and calculation methods, and accurate matching of user input and automatic adding of the calculation methods are achieved.

The invention provides a full-text retrieval method and system fusing various types of documents, and relates to the technical field of information retrieval, and the method comprises the following steps: obtaining document representation through document content extraction and structure recognition, generating a cross-modal semantic vector by using word embedding and nonlinear transformation, constructing a hierarchical index and a cross-document association graph, and obtaining a full-text retrieval result; the basic correlation score is calculated after the query request is received, and the comprehensive score of the candidate content segments is calculated based on the association graph to determine the optimal retrieval result, so that unified representation and retrieval of heterogeneous documents are realized, the cross-document retrieval precision and relevance are improved, and the processing capability of a retrieval system on complex queries is enhanced.

The invention discloses an equipment operation and maintenance decision-making method based on large-scale knowledge graph multi-granularity learning reasoning, which comprises the following steps of: firstly, extracting a multi-granularity sub-graph of a subject entity in an equipment operation and maintenance knowledge graph, and integrating multilevel information such as an equipment state, a fault mode and an environmental factor to form a relation path set; secondly, performing joint coding on a question text and a relation path through an attention enhanced word embedding module based on a graph convolutional network, forming a word embedding vector, calculating a similarity score between a relation in the path and the question, pruning a sub-graph, and reducing the calculation amount; and finally, fusing a plurality of particle size sub-graphs after pruning processing, inputting the fused particle size sub-graphs into a reasoning decision module, extracting feature information from graph structure data by the reasoning decision module through graph convolution operation, dynamically adjusting a GCN learning process in combination with LSTM, finally realizing question reasoning and answer generation, and ensuring the accuracy of an equipment operation and maintenance decision.

The invention discloses an event detection and extraction method and system based on entity and trigger word fusion recognition. The method comprises the steps that trigger words in event text data are extracted; mapping the event text to a vector space based on a word embedding model, and constructing an event text classifier to perform similarity measurement on all words of which the part-of-speech is a verb and the trigger word in the event text; constructing an event detection model to obtain an event category in the event text data; identifying a key element entity of the event; constructing a dependency syntax tree to represent a syntactic dependency relationship of the sentences; obtaining the score of the dependency syntax tree; and inputting the preprocessed word vector and the part-of-speech vector representation obtained through supervised learning into a role tagging model for fusion to obtain a vector representation with part-of-speech features, and after training, carrying out conditional random field processing to obtain a final semantic role tag of the word. Accurate event detection based on entities is realized, and high-quality extraction of events is towed.

A system is configured to: (a) provide a deep learning model; (b) receive a concept, the concept including text; (c) determine a word embedding representing the concept using the deep learning model; (d) determine a centroid for the concept, the centroid representing an average position of concept characteristics in a high-dimensional word embeddings space; (e) determine Euclidean distances for each of the concept characteristics; (f) determine cosine similarities for each of the concept characteristics; (g) determine a prominence index for each of the concept characteristics based on the determined Euclidean distances and the determined cosine similarities; and (h) provide, to a client device, a ranking of the concept characteristics based on the prominence index.

The invention discloses a method and system for establishing a software code and demand tracking relation, and relates to the computer and software engineering technology, and the method comprises the steps: translating a Chinese demand and a code annotation into English, and separating a code logic from an annotation text based on a specified rule; word embedding is carried out on the preprocessed Chinese requirements, the code logic and the annotation text to obtain embedded semantic vector representation; performing hierarchical mixed matching based on the obtained semantic vector representation so as to associate the demand with the code; and fusing the association results through configurable decision logic to determine a final matching result. According to the method provided by the embodiment of the invention, based on a multi-modal semantic alignment technology of the pre-training model, high-precision and robust tracking relation identification is realized by combining deep semantic association of the modeling demand text and the code snippets.

The invention relates to the field of video description, in particular to a dense video description method based on multi-modal memory knowledge, which comprises the following steps: extracting visual features and audio features of an input video and carrying out cross-modal fusion to generate a final audio code and a final visual code; determining event visual features and event audio features of a plurality of candidate events from the input video based on the final audio code and the final visual code; for each candidate event, retrieving matched external knowledge from an external memory knowledge base based on the corresponding event visual feature and event audio feature, and generating corresponding multi-modal external memory knowledge; based on the multi-mode external memory knowledge, the event visual features and the event audio features of each candidate event, a word embedding sequence is constructed step by step through an autoregression mechanism, and description of the input video is generated. According to the method, the corresponding relation between the event and the description can be learned from more comprehensive information, and the accuracy and richness of generating the description are remarkably improved.

The invention discloses a customer portrait construction method and device, a storage medium and electronic equipment, and relates to the field of financial science and technology. The method comprises the following steps: acquiring target customer data, wherein the target customer data comprises customer static information data, customer dynamic information data and article information data; word embedding processing is carried out on the target customer data to obtain a first word vector and a second word vector, the first word vector corresponds to the customer static information data, and the second word vector corresponds to the customer dynamic information data and the article information data; performing fusion processing on the first word vector and the second word vector through a target model to obtain a customer fusion feature vector; and determining a customer fusion feature tag according to the customer fusion feature vector, and constructing a customer portrait according to the customer fusion feature tag. According to the method and the device, the technical problem that the accuracy of the constructed customer portrait is relatively low due to the fact that discrete labels are adopted to represent customer portrait information in the prior art is solved.