747 results about "Word list" patented technology

The invention discloses a hybrid enhanced index and system based on vector retrieval and a BM2 algorithm, and the method comprises the steps: uploading a document by a user, analyzing the document by the system to obtain text content, segmenting the text content, converting the segmented text content into dense vectors, storing the dense vectors in a vector library, extracting keywords of segmented texts by using a large model, processing the keywords, and inserting the keywords into a word list. Word segmentation is carried out on the text based on the constructed word list and a word segmentation device; in response to the received user query, vectorizing the user query, and calculating the similarity with each vector in the vector library to obtain a preliminary query result; performing word segmentation on user query, performing keyword retrieval by using a BM25 algorithm according to a word list and a word segmentation device to obtain query results, and filtering the similarity of the two query results by the reordering model according to a set threshold value; selecting a result with the highest similarity according to large model parameter limitation and an upper limit set by business requirements; and splicing the document content corresponding to the result and the user query into a cue word, inputting the cue word into the large model, and analyzing a semantic relationship and a logic structure in the cue word to generate an answer.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology and the like, and discloses a voice generation method which comprises the following steps: constructing a multi-language voice synthesis model, obtaining plain text data and paired voice text data, and constructing an expansion vocabulary; updating a language perception embedding layer and model parameters, and converting an input text into a mark sequence; and the encoder extracts context semantic features, extracts pronunciation rule features, and the decoder fuses the features to generate an acoustic feature sequence, and converts the acoustic feature sequence into target voice data. According to the invention, the multi-language speech synthesis model is combined with the language perception embedding layer, so that the speech generation capability of a low-resource language is improved; the text conversion accuracy is improved by expanding the vocabulary, the target language learning ability is enhanced by unsupervised training, the low data environment adaptability is optimized by supervised training, and the speech naturalness and fluency are improved by feature fusion.

Techniques for personalized batch and streaming speech-to-text transcription of audio reduce the error rate of automatic speech recognition (ASR) systems in transcribing rare and out-of-vocabulary words. The techniques achieve personalization of connectionist temporal classification (CT) models by using adaptive boosting to perform biasing at the level of sub-words. In addition to boosting, the techniques encompass a phone alignment network to bias sub-word predictions towards rare long-tail words and out-of-vocabulary words. A technical benefit of the techniques is that the accuracy of speech-to-text transcription of rare and out-of-vocabulary words in a custom vocabulary by automatic speech recognition (ASR) system can be improved without having to train the ASR system on the custom vocabulary. Instead, the techniques allow the same ASR system trained on a base vocabulary to realize the accuracy improvements for different custom vocabularies spanning different domains.

The invention relates to a model fine tuning method, a text processing method, a medium, equipment and a program product. The method comprises the following steps: acquiring a vocabulary and an embedded layer parameter of a large language model from a server; respectively carrying out confusion processing on the word list and the embedded layer parameter to obtain a target word list and a target embedded layer parameter; sending the target embedded layer parameter to a server, so that the server updates the embedded layer parameter of the large language model into the target embedded layer parameter to obtain a new large language model; performing word segmentation and index conversion operation on the text sample by utilizing the target word list to obtain a first lexical element index corresponding to the text sample; and sending the first lemma index to a server, so that the server performs fine adjustment on the new large language model based on the first lemma index. Thus, the server side can be prevented from obtaining the corresponding relation between the embedded vector and the lexical element, and therefore the model effect can be better guaranteed through fine adjustment operation on the premise that data of a server side user is effectively protected.

The invention provides a financial fraud detection method based on a large language model. The method comprises the steps of obtaining a to-be-recognized text; performing word segmentation on the to-be-recognized text through the target word segmentation tool and the financial fraud dictionary, and determining a fraud sensitive word list; calculating the weight of each sensitive word in the sensitive word list according to a target algorithm to obtain a sensitive word weight feature vector; inputting the sensitive word weight feature vector and a to-be-recognized text into a large language model, and determining a context semantic vector of the sensitive word in combination with a word embedding technology; determining the similarity between the context semantic vector of the sensitive word and a preset financial fraud type semantic vector; and determining a financial fraud type according to the similarity. Through the implementation of the method, the generalization ability of the pre-training model is utilized to capture text deep semantics, priori knowledge is injected in combination with a sensitive word weight mechanism, the model is guided to focus high-risk vocabularies, the defect of a traditional method in semantic comprehension is overcome, the financial fraud recognition rate is increased, and the omission ratio is reduced.

The invention relates to the technical field of program similarity, in particular to a source code similarity analysis method and system based on semantic enhancement AST, and the method comprises the steps: respectively generating corresponding abstract syntax trees AST for two functions using different programming languages; enabling nodes with the same type but different names in different programming languages AST to have the same representation by utilizing a node vocabulary, and shielding the node representation difference of the different programming languages AST; a control flow edge and a data flow edge are expanded for the AST after node standardization processing, a semantic enhancement AST is generated, and the semantic enhancement AST integrates grammar, structure and semantic information of a function; a function similarity analysis network is designed, semantic enhancement AST is converted into graph vectors in combination with a graph convolutional network and an attention mechanism, the two graph vectors are spliced and then input into a full-connection neural network, and the similarity of different programming language functions is judged. According to the method, similarity judgment of different programming language functions can be realized, and the detection efficiency is improved while high accuracy of the detection effect is achieved.

The invention discloses a modal alignment and multi-scale extraction remote sensing image description generation method, system and device based on remote sensing image-text comparison pre-training features and a medium. The method comprises the steps of obtaining a remote sensing subtitle generation data set NWPU-Captions; preprocessing texts and images of a training set, a verification set and a test set of the remote sensing subtitle generation data set NWPU-Captions to obtain word lists, text features and image features; building a remote sensing subtitle generation network based on a Transform model; training the remote sensing subtitle generation network based on the Transform model by using the training set, inputting the verification set into the models obtained in each round for verification, and selecting the model with the best effect; inputting the image features in the test set into the model with the best effect to obtain subtitles corresponding to the image; the system, the equipment and the medium are used for implementing the method. Compared with a pre-trained image encoder, a multi-scale fusion extraction module and a text encoding module, features with more semantics and more scales can be extracted, and generated descriptions can contain semantic vocabularies with higher levels and more details.

Methods and systems for generating training data for training a contrastive language-audio machine-learning model. A plurality of audio segments are retrieved from a speech emotion recognition (SER) database along with metadata associated with the audio segments. The metadata of each audio segment includes an emotion class. Words or terms associated with emotions are retrieved from a lexicon. A large language model (LLM) is executed on (i) the classes of emotion associated with the audio segments and (ii) the words or terms from the lexicon. This generates a plurality of text captions associated with emotion, which are stored in a caption pool. For each audio segment retrieved from the SER database, that audio segment is paired with one or more of the text captions from the caption pool that were generated based on the emotion class associated with that audio segment. This yields audio-text pairs for training a contrastive learning model.

The invention discloses a method for embedding a text watermark during Logs generation based on a large language model. The method comprises the following steps of: analyzing a generated text tl-1 of a large model through dependency syntax analysis, constructing a dependency syntax tree of the generated text tl-1, sorting according to scores of a dependency syntax relationship, and selecting and adding biased Logs; a candidate word list is generated, probability distribution corresponding to each dependency type is calculated, the selection probability of the candidate word list is adjusted, vocabularies are sampled and generated at key positions from the candidate word list according to a set rule, and a watermark embedded text is obtained; the candidate word list comprises a first list and a second list, and the selection probability of the first list is higher than that of the second list; and extracting a to-be-detected fragment in the watermark-embedded text, detecting the watermark of the text, and outputting a detection result. According to the method, the problem that the anti-attack capability of adding watermarks to the generated text is insufficient is solved, and meanwhile, the problems of text quality and large long text calculation amount of a current watermarking method in the Logits generation period based on a large model are solved.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing tasks. One of the methods includes obtaining a sequence of input tokens, where each token is selected from a vocabulary of tokens that includes text tokens and audio tokens, and wherein the sequence of input tokens includes tokens that describe a task to be performed and data for performing the task; generating a sequence of embeddings by embedding each token in the sequence of input tokens in an embedding space; and processing the sequence of embeddings using a language model neural network to generate a sequence of output tokens for the task, where each token is selected from the vocabulary.

The invention discloses a three-dimensional model intelligent generation method and system based on a natural language, and the method comprises the steps: receiving and describing the natural language inputted by a user in real time, based on the three-dimensional model data structure, the de-noising symbol set and the stop word list, key information data are extracted according to natural language description and mapped into a three-dimensional model parameter set; according to the scene type described by the natural language and the three-dimensional model parameter set, calling a three-dimensional model generation network corresponding to the scene type described by the natural language to generate an initial three-dimensional model; and receiving and analyzing a model optimization instruction input by a user in real time, updating the geometric structure, detail features and physical attributes of the initial three-dimensional model according to an analysis result, generating a dynamic interaction model, and outputting a three-dimensional model file of the dynamic interaction model. By applying the method and the system provided by the invention, the efficiency, the accuracy, the flexibility, the usability and the user experience of three-dimensional model modeling are improved.

The invention provides a voice recognition method and system for constructing a small language based on whispertoken, and relates to the technical field of natural language processing and voice recognition, and the method comprises the steps: extracting all tokens related to a target small language in a whispertoken, and forming an initial candidate set; matching and analyzing the tokens in the initial candidate set and the collected training text corpus of the target small language, and counting the occurrence frequency of the tokens in the corpus; and screening high-frequency tokens according to a frequency statistical result, and supplementing low-frequency tokens to construct a dynamic vocabulary. According to the method, the vocabulary quality is improved, the model training efficiency is optimized, the speech recognition accuracy is enhanced, the model generalization ability is improved, and the model construction process is simplified, so that an efficient, accurate and easy-to-implement solution is provided for the field of minority language speech recognition.

The invention provides a construction method and device for a coal mine vertical field large model, and relates to the technical field of deep learning, and the method comprises the steps: obtaining field corpus data and general corpus data, and constructing a field word list and a general word list based on the field corpus data and the general corpus data; performing weighted fusion on the segmented words in the general word list and the field word list based on respective weights to obtain a fused word list; based on the fusion word list, utilizing domain corpus data and general corpus data to train the embedded model; loading a pre-trained original large language model, and replacing the embedded layer of the original large language model with the trained embedded model to obtain an updated large language model; and performing incremental pre-training on the updated large language model by using the domain corpus data and the general corpus data to obtain a coal mine vertical domain large model. The coal mine vertical field large model constructed by the method realizes accurate expression of professional knowledge of the coal mine industry.

The present application provides a translation model training method, translation method, device, electronic equipment, and medium, relating to the field of translation model technology. The translation model training method includes: expanding vocabulary of the translation model with commonly used vocabulary and specialized vocabulary in advance, optimizing feature vectors in the vocabulary based on a large language model, cleaning an open-source dataset using a data cleaning method to filter and retain corpora meeting a quality standard, and training the translation model using filtered corpora. The translation method includes: obtaining source language text to be translated, inputting the source language text into a translation model to obtain target language text after translation. The embodiments of this application not only enhance the vocabulary of the translation model but also improve the accuracy of the translation model in specialized domains.

The invention provides a language model training method and device, a text processing method and device, equipment and a medium, and relates to the technical field of natural language processing, and the method comprises the steps: predicting a first probability matrix corresponding to each data unit in a sample text based on a teacher model; the first probability matrix comprises a probability value of each data unit belonging to each lexical element in the first word list; according to the numerical value of each probability value in the first probability matrix, compressing the first probability matrix to obtain a second probability matrix corresponding to each data unit; performing alignment operation on the second word list according to lexical elements corresponding to probability values in the second probability matrix to obtain a third word list; and according to the third word list and the second probability matrix, carrying out distillation training on the student model to obtain a target language model, thereby reducing the storage cost, improving the distillation training efficiency, and enabling the target language model trained according to the method to better adapt to different model architectures and text processing scenes while keeping high performance.

Embodiments described herein provide methods and systems for open vocabulary object detection of images. given a pre-trained vision-language model and an image-caption pair, an activation map may be computed in the image that corresponds to an object of interest mentioned in the caption. The activation map is then converted into a pseudo bounding-box label for the corresponding object category. The open vocabulary detector is then directly supervised by these pseudo box-labels, which enables training object detectors with no human-provided bounding-box annotations.

The invention relates to the technical field of information extraction, in particular to an information extraction method and system for electronic archives. The method comprises the following steps: obtaining a to-be-processed electronic file and carrying out OCR identification to generate initial text data; error detection is carried out on the initial text data, and OCR error recognition candidate items in the initial text data are recognized; for each OCR misrecognition candidate item, generating a first data name according to context semantics and a layout structure of the candidate item; extracting low-level features of the first data name, and performing named entity recognition on each OCR misrecognition candidate item by utilizing a preset field word list in combination with the first data name; through a four-in-one process of ''misrecognition detection + named entity recognition + semantic error correction + templated extraction'', the core technology bottlenecks of inaccurate recognition, poor error correction capability, low information extraction intelligence and the like in the prior art are solved, and the accuracy, stability and intelligent level of electronic archive information extraction are remarkably improved.

Techniques are disclosed herein for generating dynamic vocabularies for conditioning a language model. A dynamic vocabulary is constructed from an input prompt, database schema information for a database to be queried, and programming language information for a programming language to be used for querying the database to condition the language model to predict an output statement in the programming language. The dynamic vocabulary can be included in prompt information that is provided to the language model. The number of tokens in the dynamic vocabulary can be different than a number of tokens included in a vocabulary of the language model. By utilizing a dynamic vocabulary, the language model can be conditioned to predict tokens for the output statement that are contextually consistent with the tokens included the dynamic vocabulary.

The invention discloses a multi-modal visual language model illusion relieving method based on illusion induction optimization, and relates to the field of artificial intelligence and computer vision. According to the method, when a generative large language model generates new word marks, firstly, the probability of each word mark in a vocabulary is generated according to an input mark sequence, the contrast ratio of the current input mark sequence is calculated through a contrast model, the contrast ratio is mapped into a preset vector space, and the corrected logarithmic probability is obtained. And selecting a plurality of illusion tokens with the maximum contrast from the illusion tokens, and adjusting the probability of the illusion tokens according to a preset optimization strategy. If the illusion token belongs to the selected illusion token, amplifying the probability of the illusion token according to preset optimization intensity; otherwise, the current probability is the corrected probability. And selecting a new word mark according to the corrected probability, and circulating the process to generate a final text. According to the method, by introducing the illusion induction optimization strategy, the illusion phenomenon generated when the large visual language model generates the text is remarkably reduced.

According to the pre-training language model-based drug-target interaction prediction method designed by the invention, natural language processing and graph neural network technologies are fused, context semantic features are automatically extracted from drug molecule SMILES character strings and protein sequences, and by constructing a graph structure taking drug-target pairs as nodes, the drug-target interaction is predicted. The weight of an edge is defined according to the similarity between embedded vectors, and a simplified graph convolutional network is adopted to carry out graph structure modeling to realize complex relation learning, so that the accuracy, generalization and interpretability of prediction are improved, the limitation of a traditional method on the problems of sparse feature expression, mutual information loss and'words outside a vocabulary 'is overcome, and the prediction accuracy, generalization and interpretability are improved. And finally, the accuracy of predicting the drug-target interaction relationship is improved.

The invention discloses a hole making process parameter recommendation method based on a knowledge graph, which uses an optimized seven-step method, and provides an ontology evaluation system and an ontology updating method, so that a hole making process ontology can be updated and iterated according to different processing requirements; according to the method, the fine-tuning UIE model is adopted to perform knowledge extraction on the hole-making process text data, good extraction accuracy can be obtained only by performing fine-tuning on the UIE model by using small sample data, the marking cost of knowledge extraction of the hole-making process text data is reduced, and the extraction efficiency is improved; according to the method, an entity alignment algorithm and a processing case filtering algorithm based on text similarity and word list mapping are designed, so that the repetition rate of knowledge in the knowledge graph is reduced, and the retrieval efficiency is improved.

The invention relates to the technical field of model watermarking, and discloses a multi-modal model watermarking method based on synonym replacement, which comprises the following steps of: inputting a given image and a prompt into a multi-modal model to generate a text sequence for describing the image or a text sequence for answering a question proposed based on the image; watermark information can be detected in the text sequence; the method comprises the following steps of: mapping a secret image into a hash value as a private key by using a perceptual hash algorithm; dividing the vocabulary by using a private key and changing an output sample to embed watermark information; and checking whether the watermark exists or not by counting the number of words in a green list in a text sequence output by the multi-modal model. According to the method, the watermark can be embedded into the multi-modal model only by changing the text output sample, retraining is not needed, and detection can be carried out without accessing the model, so that the method is suitable for all mainstream multi-modal model structures, and strong generalization is provided.

The invention provides a low-resource language-oriented large language model training method and device, equipment and a medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring low-resource language data and preprocessing the low-resource language data to obtain low-resource language unsupervised training data; generating a low-resource language vocabulary and performing vocabulary expansion on the existing model; performing unsupervised pre-training on the model after word list expansion according to the low-resource language unsupervised training data in combination with a first low-rank adaptive module; performing instruction fine tuning on the pre-trained model according to the low-resource language instruction fine tuning data set in combination with a second low-rank adaptive module; and respectively inputting a high-resource language supervised corpus into the model subjected to instruction fine tuning and the existing mature model to respectively obtain negative example data and positive example data so as to generate a reinforcement learning low-resource language data set based on model feedback, and carrying out reinforcement learning training on the model subjected to instruction fine tuning in combination with a third low-rank adaptive module.

The invention provides a construction method of an intelligent personalized interaction system. The construction method comprises the following steps: constructing a database; preparing a training set and a test set with problem texts, constructing a vocabulary of high-frequency words for the training set and the test set, and performing text coding; constructing a workflow scheduler for counting high-frequency problem types; the user portrait building module is used for obtaining a user portrait according to historical dialogues, user questions and high-frequency question types of the database; the intelligent personalized interaction system comprises a user question input interface, a question classification module and a database which are connected in sequence, and the database is connected with a workflow scheduler and a user portrait construction module. And outputting the user portrait and the high-frequency question type to a pre-trained large language model to generate a recommendation question. The invention further provides a corresponding system. According to the method, deep understanding, accurate analysis and personalized interaction of user questions are realized, so that the user experience is remarkably improved, and diversified requirements of users in different scenes are met.

The invention provides an electronic medical record connotation quality control method and system based on a large language model, and the method comprises the steps: carrying out the modal classification of a whole-process medical record of a patient, and uniformly converting the whole-process medical record into a corresponding text modal; performing hierarchical division on the whole-process medical record of the text mode to form a medical record content tree; according to different nodes of the medical record content tree, retrieving a quality control rule set corresponding to the nodes of the medical record content tree, and retrieving a corresponding diagnosis and treatment knowledge set; for each quality control rule in the quality control rule set, if a plurality of medical record content tree nodes are needed, establishing a corresponding context set, constructing a quality control cue word list, inputting cue words into the connotation quality control big language model, judging whether a quality control problem exists in the cue words or not by the connotation quality control big language model, and proposing a modification suggestion; and checking and revising the output result to obtain a final quality control result. According to the method, key point missing, description conflicts and deep diagnosis and treatment logic contradictions in medical records can be recognized and corrected, and the accuracy and stability of quality control results are improved.

The embodiment of the invention is suitable for the technical field of artificial intelligence, and provides a data privacy protection method, system and device for large language model application, the method is applied to client equipment, and the client equipment is deployed with an input layer and an output layer of a large language model. The method comprises the steps that text input data of the large language model is coded according to a vocabulary, a mark list represented by integers is obtained, and the vocabulary comes from a server; processing the mark list through the input layer to obtain intermediate input data; the intermediate input data is sent to the server; receiving intermediate output data returned by the server; and processing the intermediate output data based on the output layer and the vocabulary to obtain text output data. Through the method, the privacy protection of the input data can be realized while the output data is automatically obtained by using the large language model.

The invention discloses an emotion analysis method and device for promoting multi-mode information fusion, and the method comprises the steps: building an acoustic vocabulary and a visual vocabulary based on a K-Means algorithm, querying the features of a non-text mode, obtaining a word index sequence of the non-text mode, and embedding the word index sequence into the initial features of the text mode to build a non-language displacement vector; adding the non-language displacement vector into the initial feature of the text mode for displacement operation to obtain a multi-mode vector; and performing context sensing and sentiment analysis on the multi-modal vector to obtain and output a sentiment analysis result. According to the method, the feature sequence of the non-text mode is converted into the word index sequence which is the same as the text mode through the K-Means algorithm, the distribution gap between the language text mode and the non-language text mode during fusion is reduced, and subsequent feature fusion is facilitated; during feature fusion, the expression of a language text mode is enhanced through a non-language displacement vector, so that the accuracy of sentiment analysis is improved.

The invention discloses a large language model high-quality text data set construction method and system, belongs to the technical field of deep learning, and aims to solve the technical problems of how to construct a high-quality text data set and how to reduce noise data. Comprising the following steps: collecting related literatures from industry data to obtain a text corpus; performing data preprocessing on the collected text corpus to obtain a pre-labeled text corpus; constructing a label system based on the industry terminologies and the indexes, and labeling the relationship between the entities in the pre-labeled text corpus based on the label system; the method comprises the following steps: updating an original vocabulary of a pre-trained large language model based on an industry dictionary, carrying out unsupervised training on the pre-trained large language model, taking a masked text corpus and corresponding label information as input attributes, carrying out upper and lower semantic analysis on the input attributes through the large model subjected to unsupervised training, predicting words masked by the text corpus, and obtaining a word masked by the text corpus. And outputting the enhanced text data, and performing data testing on the text data.

The invention relates to the technical field of artificial intelligence, in particular to an automatic deep thinking model selection training method for a large language model. Comprising the following steps: expanding a word segmentation device vocabulary, and adding a special mark lt; carrying out think gt; and lt; / (ginkgt); the identification module is used for identifying a content structure of the deep thinking mode; a dialogue template is designed, input and output formats of a common mode and a deep thinking mode are distinguished, and the deep thinking mode comprises lt; carrying out think gt; marking a guided reasoning process; a common mode training data set and a deep thinking mode training data set with the same scale are constructed, and joint training is performed on the model, so that the model has response capabilities of the two modes at the same time. The method comprises the following steps: adding a special mark (lt; carrying out think gt; and lt; / (ginkgt); and a dialogue template is customized, semantic distinguishing between a common mode and a deep thinking mode is achieved, and the model can accurately switch response strategies according to requirements.

The invention provides a text watermark adding and detecting method and device, equipment and a storage medium, and can be applied to various fields such as text creation. The method comprises the steps of obtaining cue words of a to-be-generated text and a word list of a language model; for the ith token to be generated of the text to be generated, T generated tokens are obtained from the current generated token sequence, the T generated tokens are processed through a language model, and probability distribution corresponding to the ith token to be generated is obtained; selecting K generated tokens from the current generated token sequence, determining semantic feature information of the K generated tokens, and determining a random seed corresponding to the ith to-be-generated token based on the semantic feature information of the K generated tokens; and adjusting the probability in the probability distribution based on the random seed, determining a target token of the i-th token to be generated based on the adjusted probability distribution, and obtaining a text added with a watermark based on the target token. According to the invention, the robustness of text watermark addition is improved.