505 results about "Word list" patented technology

The invention discloses a hybrid enhanced index and system based on vector retrieval and a BM2 algorithm, and the method comprises the steps: uploading a document by a user, analyzing the document by the system to obtain text content, segmenting the text content, converting the segmented text content into dense vectors, storing the dense vectors in a vector library, extracting keywords of segmented texts by using a large model, processing the keywords, and inserting the keywords into a word list. Word segmentation is carried out on the text based on the constructed word list and a word segmentation device; in response to the received user query, vectorizing the user query, and calculating the similarity with each vector in the vector library to obtain a preliminary query result; performing word segmentation on user query, performing keyword retrieval by using a BM25 algorithm according to a word list and a word segmentation device to obtain query results, and filtering the similarity of the two query results by the reordering model according to a set threshold value; selecting a result with the highest similarity according to large model parameter limitation and an upper limit set by business requirements; and splicing the document content corresponding to the result and the user query into a cue word, inputting the cue word into the large model, and analyzing a semantic relationship and a logic structure in the cue word to generate an answer.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology and the like, and discloses a voice generation method which comprises the following steps: constructing a multi-language voice synthesis model, obtaining plain text data and paired voice text data, and constructing an expansion vocabulary; updating a language perception embedding layer and model parameters, and converting an input text into a mark sequence; and the encoder extracts context semantic features, extracts pronunciation rule features, and the decoder fuses the features to generate an acoustic feature sequence, and converts the acoustic feature sequence into target voice data. According to the invention, the multi-language speech synthesis model is combined with the language perception embedding layer, so that the speech generation capability of a low-resource language is improved; the text conversion accuracy is improved by expanding the vocabulary, the target language learning ability is enhanced by unsupervised training, the low data environment adaptability is optimized by supervised training, and the speech naturalness and fluency are improved by feature fusion.

The invention provides a financial fraud detection method based on a large language model. The method comprises the steps of obtaining a to-be-recognized text; performing word segmentation on the to-be-recognized text through the target word segmentation tool and the financial fraud dictionary, and determining a fraud sensitive word list; calculating the weight of each sensitive word in the sensitive word list according to a target algorithm to obtain a sensitive word weight feature vector; inputting the sensitive word weight feature vector and a to-be-recognized text into a large language model, and determining a context semantic vector of the sensitive word in combination with a word embedding technology; determining the similarity between the context semantic vector of the sensitive word and a preset financial fraud type semantic vector; and determining a financial fraud type according to the similarity. Through the implementation of the method, the generalization ability of the pre-training model is utilized to capture text deep semantics, priori knowledge is injected in combination with a sensitive word weight mechanism, the model is guided to focus high-risk vocabularies, the defect of a traditional method in semantic comprehension is overcome, the financial fraud recognition rate is increased, and the omission ratio is reduced.

Methods and systems for generating training data for training a contrastive language-audio machine-learning model. A plurality of audio segments are retrieved from a speech emotion recognition (SER) database along with metadata associated with the audio segments. The metadata of each audio segment includes an emotion class. Words or terms associated with emotions are retrieved from a lexicon. A large language model (LLM) is executed on (i) the classes of emotion associated with the audio segments and (ii) the words or terms from the lexicon. This generates a plurality of text captions associated with emotion, which are stored in a caption pool. For each audio segment retrieved from the SER database, that audio segment is paired with one or more of the text captions from the caption pool that were generated based on the emotion class associated with that audio segment. This yields audio-text pairs for training a contrastive learning model.

The invention discloses a three-dimensional model intelligent generation method and system based on a natural language, and the method comprises the steps: receiving and describing the natural language inputted by a user in real time, based on the three-dimensional model data structure, the de-noising symbol set and the stop word list, key information data are extracted according to natural language description and mapped into a three-dimensional model parameter set; according to the scene type described by the natural language and the three-dimensional model parameter set, calling a three-dimensional model generation network corresponding to the scene type described by the natural language to generate an initial three-dimensional model; and receiving and analyzing a model optimization instruction input by a user in real time, updating the geometric structure, detail features and physical attributes of the initial three-dimensional model according to an analysis result, generating a dynamic interaction model, and outputting a three-dimensional model file of the dynamic interaction model. By applying the method and the system provided by the invention, the efficiency, the accuracy, the flexibility, the usability and the user experience of three-dimensional model modeling are improved.

The invention provides a voice recognition method and system for constructing a small language based on whispertoken, and relates to the technical field of natural language processing and voice recognition, and the method comprises the steps: extracting all tokens related to a target small language in a whispertoken, and forming an initial candidate set; matching and analyzing the tokens in the initial candidate set and the collected training text corpus of the target small language, and counting the occurrence frequency of the tokens in the corpus; and screening high-frequency tokens according to a frequency statistical result, and supplementing low-frequency tokens to construct a dynamic vocabulary. According to the method, the vocabulary quality is improved, the model training efficiency is optimized, the speech recognition accuracy is enhanced, the model generalization ability is improved, and the model construction process is simplified, so that an efficient, accurate and easy-to-implement solution is provided for the field of minority language speech recognition.

The invention provides a construction method and device for a coal mine vertical field large model, and relates to the technical field of deep learning, and the method comprises the steps: obtaining field corpus data and general corpus data, and constructing a field word list and a general word list based on the field corpus data and the general corpus data; performing weighted fusion on the segmented words in the general word list and the field word list based on respective weights to obtain a fused word list; based on the fusion word list, utilizing domain corpus data and general corpus data to train the embedded model; loading a pre-trained original large language model, and replacing the embedded layer of the original large language model with the trained embedded model to obtain an updated large language model; and performing incremental pre-training on the updated large language model by using the domain corpus data and the general corpus data to obtain a coal mine vertical domain large model. The coal mine vertical field large model constructed by the method realizes accurate expression of professional knowledge of the coal mine industry.

The present application provides a translation model training method, translation method, device, electronic equipment, and medium, relating to the field of translation model technology. The translation model training method includes: expanding vocabulary of the translation model with commonly used vocabulary and specialized vocabulary in advance, optimizing feature vectors in the vocabulary based on a large language model, cleaning an open-source dataset using a data cleaning method to filter and retain corpora meeting a quality standard, and training the translation model using filtered corpora. The translation method includes: obtaining source language text to be translated, inputting the source language text into a translation model to obtain target language text after translation. The embodiments of this application not only enhance the vocabulary of the translation model but also improve the accuracy of the translation model in specialized domains.

The invention provides a language model training method and device, a text processing method and device, equipment and a medium, and relates to the technical field of natural language processing, and the method comprises the steps: predicting a first probability matrix corresponding to each data unit in a sample text based on a teacher model; the first probability matrix comprises a probability value of each data unit belonging to each lexical element in the first word list; according to the numerical value of each probability value in the first probability matrix, compressing the first probability matrix to obtain a second probability matrix corresponding to each data unit; performing alignment operation on the second word list according to lexical elements corresponding to probability values in the second probability matrix to obtain a third word list; and according to the third word list and the second probability matrix, carrying out distillation training on the student model to obtain a target language model, thereby reducing the storage cost, improving the distillation training efficiency, and enabling the target language model trained according to the method to better adapt to different model architectures and text processing scenes while keeping high performance.

The invention relates to the technical field of information extraction, in particular to an information extraction method and system for electronic archives. The method comprises the following steps: obtaining a to-be-processed electronic file and carrying out OCR identification to generate initial text data; error detection is carried out on the initial text data, and OCR error recognition candidate items in the initial text data are recognized; for each OCR misrecognition candidate item, generating a first data name according to context semantics and a layout structure of the candidate item; extracting low-level features of the first data name, and performing named entity recognition on each OCR misrecognition candidate item by utilizing a preset field word list in combination with the first data name; through a four-in-one process of ''misrecognition detection + named entity recognition + semantic error correction + templated extraction'', the core technology bottlenecks of inaccurate recognition, poor error correction capability, low information extraction intelligence and the like in the prior art are solved, and the accuracy, stability and intelligent level of electronic archive information extraction are remarkably improved.

Techniques are disclosed herein for generating dynamic vocabularies for conditioning a language model. A dynamic vocabulary is constructed from an input prompt, database schema information for a database to be queried, and programming language information for a programming language to be used for querying the database to condition the language model to predict an output statement in the programming language. The dynamic vocabulary can be included in prompt information that is provided to the language model. The number of tokens in the dynamic vocabulary can be different than a number of tokens included in a vocabulary of the language model. By utilizing a dynamic vocabulary, the language model can be conditioned to predict tokens for the output statement that are contextually consistent with the tokens included the dynamic vocabulary.

The invention discloses a multi-modal visual language model illusion relieving method based on illusion induction optimization, and relates to the field of artificial intelligence and computer vision. According to the method, when a generative large language model generates new word marks, firstly, the probability of each word mark in a vocabulary is generated according to an input mark sequence, the contrast ratio of the current input mark sequence is calculated through a contrast model, the contrast ratio is mapped into a preset vector space, and the corrected logarithmic probability is obtained. And selecting a plurality of illusion tokens with the maximum contrast from the illusion tokens, and adjusting the probability of the illusion tokens according to a preset optimization strategy. If the illusion token belongs to the selected illusion token, amplifying the probability of the illusion token according to preset optimization intensity; otherwise, the current probability is the corrected probability. And selecting a new word mark according to the corrected probability, and circulating the process to generate a final text. According to the method, by introducing the illusion induction optimization strategy, the illusion phenomenon generated when the large visual language model generates the text is remarkably reduced.

According to the pre-training language model-based drug-target interaction prediction method designed by the invention, natural language processing and graph neural network technologies are fused, context semantic features are automatically extracted from drug molecule SMILES character strings and protein sequences, and by constructing a graph structure taking drug-target pairs as nodes, the drug-target interaction is predicted. The weight of an edge is defined according to the similarity between embedded vectors, and a simplified graph convolutional network is adopted to carry out graph structure modeling to realize complex relation learning, so that the accuracy, generalization and interpretability of prediction are improved, the limitation of a traditional method on the problems of sparse feature expression, mutual information loss and'words outside a vocabulary 'is overcome, and the prediction accuracy, generalization and interpretability are improved. And finally, the accuracy of predicting the drug-target interaction relationship is improved.

The invention discloses a hole making process parameter recommendation method based on a knowledge graph, which uses an optimized seven-step method, and provides an ontology evaluation system and an ontology updating method, so that a hole making process ontology can be updated and iterated according to different processing requirements; according to the method, the fine-tuning UIE model is adopted to perform knowledge extraction on the hole-making process text data, good extraction accuracy can be obtained only by performing fine-tuning on the UIE model by using small sample data, the marking cost of knowledge extraction of the hole-making process text data is reduced, and the extraction efficiency is improved; according to the method, an entity alignment algorithm and a processing case filtering algorithm based on text similarity and word list mapping are designed, so that the repetition rate of knowledge in the knowledge graph is reduced, and the retrieval efficiency is improved.

The invention relates to the technical field of model watermarking, and discloses a multi-modal model watermarking method based on synonym replacement, which comprises the following steps of: inputting a given image and a prompt into a multi-modal model to generate a text sequence for describing the image or a text sequence for answering a question proposed based on the image; watermark information can be detected in the text sequence; the method comprises the following steps of: mapping a secret image into a hash value as a private key by using a perceptual hash algorithm; dividing the vocabulary by using a private key and changing an output sample to embed watermark information; and checking whether the watermark exists or not by counting the number of words in a green list in a text sequence output by the multi-modal model. According to the method, the watermark can be embedded into the multi-modal model only by changing the text output sample, retraining is not needed, and detection can be carried out without accessing the model, so that the method is suitable for all mainstream multi-modal model structures, and strong generalization is provided.

The invention provides a low-resource language-oriented large language model training method and device, equipment and a medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring low-resource language data and preprocessing the low-resource language data to obtain low-resource language unsupervised training data; generating a low-resource language vocabulary and performing vocabulary expansion on the existing model; performing unsupervised pre-training on the model after word list expansion according to the low-resource language unsupervised training data in combination with a first low-rank adaptive module; performing instruction fine tuning on the pre-trained model according to the low-resource language instruction fine tuning data set in combination with a second low-rank adaptive module; and respectively inputting a high-resource language supervised corpus into the model subjected to instruction fine tuning and the existing mature model to respectively obtain negative example data and positive example data so as to generate a reinforcement learning low-resource language data set based on model feedback, and carrying out reinforcement learning training on the model subjected to instruction fine tuning in combination with a third low-rank adaptive module.

The invention provides a construction method of an intelligent personalized interaction system. The construction method comprises the following steps: constructing a database; preparing a training set and a test set with problem texts, constructing a vocabulary of high-frequency words for the training set and the test set, and performing text coding; constructing a workflow scheduler for counting high-frequency problem types; the user portrait building module is used for obtaining a user portrait according to historical dialogues, user questions and high-frequency question types of the database; the intelligent personalized interaction system comprises a user question input interface, a question classification module and a database which are connected in sequence, and the database is connected with a workflow scheduler and a user portrait construction module. And outputting the user portrait and the high-frequency question type to a pre-trained large language model to generate a recommendation question. The invention further provides a corresponding system. According to the method, deep understanding, accurate analysis and personalized interaction of user questions are realized, so that the user experience is remarkably improved, and diversified requirements of users in different scenes are met.

The invention provides an electronic medical record connotation quality control method and system based on a large language model, and the method comprises the steps: carrying out the modal classification of a whole-process medical record of a patient, and uniformly converting the whole-process medical record into a corresponding text modal; performing hierarchical division on the whole-process medical record of the text mode to form a medical record content tree; according to different nodes of the medical record content tree, retrieving a quality control rule set corresponding to the nodes of the medical record content tree, and retrieving a corresponding diagnosis and treatment knowledge set; for each quality control rule in the quality control rule set, if a plurality of medical record content tree nodes are needed, establishing a corresponding context set, constructing a quality control cue word list, inputting cue words into the connotation quality control big language model, judging whether a quality control problem exists in the cue words or not by the connotation quality control big language model, and proposing a modification suggestion; and checking and revising the output result to obtain a final quality control result. According to the method, key point missing, description conflicts and deep diagnosis and treatment logic contradictions in medical records can be recognized and corrected, and the accuracy and stability of quality control results are improved.

The embodiment of the invention is suitable for the technical field of artificial intelligence, and provides a data privacy protection method, system and device for large language model application, the method is applied to client equipment, and the client equipment is deployed with an input layer and an output layer of a large language model. The method comprises the steps that text input data of the large language model is coded according to a vocabulary, a mark list represented by integers is obtained, and the vocabulary comes from a server; processing the mark list through the input layer to obtain intermediate input data; the intermediate input data is sent to the server; receiving intermediate output data returned by the server; and processing the intermediate output data based on the output layer and the vocabulary to obtain text output data. Through the method, the privacy protection of the input data can be realized while the output data is automatically obtained by using the large language model.

The invention discloses a large language model high-quality text data set construction method and system, belongs to the technical field of deep learning, and aims to solve the technical problems of how to construct a high-quality text data set and how to reduce noise data. Comprising the following steps: collecting related literatures from industry data to obtain a text corpus; performing data preprocessing on the collected text corpus to obtain a pre-labeled text corpus; constructing a label system based on the industry terminologies and the indexes, and labeling the relationship between the entities in the pre-labeled text corpus based on the label system; the method comprises the following steps: updating an original vocabulary of a pre-trained large language model based on an industry dictionary, carrying out unsupervised training on the pre-trained large language model, taking a masked text corpus and corresponding label information as input attributes, carrying out upper and lower semantic analysis on the input attributes through the large model subjected to unsupervised training, predicting words masked by the text corpus, and obtaining a word masked by the text corpus. And outputting the enhanced text data, and performing data testing on the text data.

The invention relates to the technical field of artificial intelligence, in particular to an automatic deep thinking model selection training method for a large language model. Comprising the following steps: expanding a word segmentation device vocabulary, and adding a special mark lt; carrying out think gt; and lt; / (ginkgt); the identification module is used for identifying a content structure of the deep thinking mode; a dialogue template is designed, input and output formats of a common mode and a deep thinking mode are distinguished, and the deep thinking mode comprises lt; carrying out think gt; marking a guided reasoning process; a common mode training data set and a deep thinking mode training data set with the same scale are constructed, and joint training is performed on the model, so that the model has response capabilities of the two modes at the same time. The method comprises the following steps: adding a special mark (lt; carrying out think gt; and lt; / (ginkgt); and a dialogue template is customized, semantic distinguishing between a common mode and a deep thinking mode is achieved, and the model can accurately switch response strategies according to requirements.

The invention provides a text watermark adding and detecting method and device, equipment and a storage medium, and can be applied to various fields such as text creation. The method comprises the steps of obtaining cue words of a to-be-generated text and a word list of a language model; for the ith token to be generated of the text to be generated, T generated tokens are obtained from the current generated token sequence, the T generated tokens are processed through a language model, and probability distribution corresponding to the ith token to be generated is obtained; selecting K generated tokens from the current generated token sequence, determining semantic feature information of the K generated tokens, and determining a random seed corresponding to the ith to-be-generated token based on the semantic feature information of the K generated tokens; and adjusting the probability in the probability distribution based on the random seed, determining a target token of the i-th token to be generated based on the adjusted probability distribution, and obtaining a text added with a watermark based on the target token. According to the invention, the robustness of text watermark addition is improved.

The invention discloses an unmanned vehicle inspection small target detection method based on an efficient attention mechanism. The method comprises four stages of image feature extraction, vocabulary embedding extraction, efficient attention coding and cross attention decoding. And image feature extraction: performing feature extraction on the input image by using the backbone network to generate a multi-scale feature map. And vocabulary embedding extraction: generating vocabulary embedding in the defined category vocabulary through a CLIP text encoder. And efficient attention coding: performing deep feature interaction, space attention guidance and multi-scale feature aggregation processing on the multi-scale feature map of the picture to obtain an image feature map fusing the visual context and the multi-scale information. And cross attention decoding: embedding the aggregated feature map and vocabulary, and outputting a final detection result through cross attention fusion, IoU perception query and regional text comparison processing. Compared with the prior art, the method has the advantages of good prediction effect, good practicability and the like.

The invention relates to an encrypted network traffic classification method based on a pre-trained large language model, and belongs to the technical field of encrypted network traffic classification. The method mainly comprises three stages: a pre-training stage: converting original encrypted network traffic data into a double-byte hexadecimal format through preprocessing, generating and optimizing a basic vocabulary by using a byte pair coding algorithm, then constructing a large language model, and obtaining a pre-training model through distributed training; in the retraining stage, the data is subjected to head byte shuffling processing, and the pre-training model is quickly retrained to improve the generalization ability. In the fine tuning stage, to-be-classified data is preprocessed to generate hexadecimal double-byte data with labels, and classification task training is performed by using the retrained model to obtain an encrypted network traffic classification fine tuning model and classification accuracy. Through the combination of pre-training and retraining and the fine tuning of the pre-training model by means of data classification, the efficient processing and accurate classification of the complex encrypted network traffic are realized.

The invention provides a two-stage training method for a lightweight Chinese-English bilingual large language model. The method comprises the following steps: step 1, collecting and preprocessing Chinese-English training data; the preprocessing process comprises special mark clearing, sensitive data clearing, data duplicate removal, advertisement removal and formatting processing; 2, performing first-stage high-resource language training, namely performing model training by using English; 3, second-stage bilingual mixed training is carried out, and the Chinese data and the English data are mixed for second-stage training by adopting language migration training; wherein the step 3 comprises the following steps: step 31, performing word list expansion; and step 32, language migration training. The bilingual processing capability can be improved.

The invention discloses a text information extraction method and system based on a large language model, and belongs to the technical field of natural language processing. Inputting the obtained text information data into the trained text information extraction model, and performing text information processing; wherein the training of the text information extraction model comprises the following steps: constructing a generative and discriminant text information extraction model based on a large language model; constructing a dialogue strategy gradient discrete prompt optimization framework to enhance the prompt stability; and constructing a noise channel matrix, and performing data enhancement and stability enhancement. According to the method, the stability of the model and the accuracy of the extraction result are effectively improved by introducing the judgment component, the balance between generation and copying is optimized by designing the dynamic generation probability calculation module and the extension word list mechanism, the stability and accuracy of text information extraction are effectively improved, and the user experience is improved. And efficient and reliable technical support is provided for scenes such as public opinion analysis and knowledge graph construction.

The invention relates to a sensitive data sharing method and system based on data consanguinity, and belongs to the technical field of data management and sharing. The system comprises four core components including a data consanguinity module, a sensitive word list recognition module, a data desensitization module and a data sharing module, wherein the data consanguinity module dynamically collects and analyzes a full life cycle flow path of data from a source end to an application end, and constructs a consanguinity map of structured storage; the sensitive word list identification module identifies sensitive data according to the sensitive word list and the blood relationship map, and executes risk grading, dynamic monitoring and compliance auditing; the data desensitization module configures a desensitization rule to deform or convert sensitive data; and the data sharing module performs intelligent approval, dynamic authorization and full-link audit on the data sharing request based on the sensitive word list and the desensitization rule. According to the method, the data security sharing efficiency is remarkably improved, and the security goals of data availability and invisibility and no risk in sharing are achieved.

The invention provides a GNN-based cross-architecture binary program similarity detection method, apparatus and device, and relates to the technical field of information security processing. The method comprises the following steps: acquiring two binary programs to be detected, and disassembling the binary programs into a low-level virtual machine intermediate representation (LLVM) IR; constructing a program diagram based on LLVM IR; then inputting the program diagram into a FastText model to extract an LLVM IR instruction, and taking a corpus created based on the LLVM IR instruction as a vocabulary of the FastText model to perform multiple rounds of training so as to express an instruction mark as a word vector in a continuous vector space, and generating an instruction vector; according to the program diagram and the instruction vector, processing by using a global attention enhanced diagram neural network GNN to generate a diagram embedding vector with a fixed dimension; and calculating the similarity between the graph embedding vectors corresponding to the two binary programs to evaluate the similarity. According to the method, unified program representation of cross-framework binary programs can be realized, high-level semantic features are captured, and the processing efficiency of large-scale program library analysis is effectively improved.

The invention relates to the technical field of intelligent questioning and answering workers, in particular to a power grid equipment intelligent questioning and answering optimization method and system based on a large language model. The method comprises the following steps: acquiring problem data of power grid equipment, and creating a dynamically updated power grid equipment vocabulary to store the problem data; a part-of-speech tagging model of a conditional random field is used for tagging part-of-speech of question data, power grid equipment knowledge retrieval and large language model building are performed based on the question data of tagging and semantic roles, parameter updating is performed on the built large language model, and answer optimization is performed according to a processing result of the large language model. According to the method, a position coding formula combined with semantic information is introduced, so that a large language model can better understand the relationship between a word order and semantics in a text in the field of power grid equipment, semantic logic in the text can be more deeply mastered, the ability of understanding the operation process and the equipment relationship is improved, and answers more conforming to professional logic are output.

The invention provides a credible time sequence prediction method based on a big language model fusion knowledge graph, and the method comprises the steps: carrying out the reversible normalization processing of original time sequence data, and decomposing the normalized time sequence data into trend, season and residual components; constructing a time sequence knowledge graph with rich semantics by utilizing a vocabulary of the pre-trained large language model; screening the time sequence knowledge graph to obtain a prefix prompt sequence, and splicing the prefix prompt sequence with the time sequence embedding to obtain an enhanced time sequence embedding; and embedding and inputting the enhanced time sequence into a pre-trained large language model for processing, and performing inverse standardization processing to obtain a predicted value. According to the method, diversity of time sequence data components can be effectively captured, high-quality context prompts are provided by means of the time sequence knowledge graph, and the prediction performance of the model is remarkably improved; and meanwhile, cognitive uncertainty and accidental uncertainty of a prediction result can be quantified, so that credible time sequence prediction is realized.