Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

149 results about "Sentence pair" patented technology

Pair in a sentence A pair of. Pair of aces. was one of a pair. A pair of hawks. Very rare pairing. It was pairing time. paired up and synced. Paired off and cocky. Two pairs of. They fly in pairs.

Method and system for detecting and repairing output illusion of large language model

The invention relates to the technical field of knowledge questions and answers, in particular to a method and system for detecting and repairing large language model output illusion, which comprises the following steps of: dividing answer content into a plurality of sentences, calculating the confidence coefficient of each sentence, judging whether the confidence coefficient of each sentence exceeds a set threshold value or not, performing fact extraction on the sentences with the confidence coefficient lower than the set threshold value, obtaining triple data of each sentence; generating a subgraph A of each sentence based on the triple data of each sentence; extracting a sub-graph B from the local knowledge graph based on the subject in the triple data of each sentence; calculating the similarity between the sub-graph A and the sub-graph B, and if the similarity is smaller than a set threshold value, repairing the sub-graph A according to the sub-graph B to obtain a repaired sub-graph A; according to the triad data corresponding to the repaired subgraph A, obtaining a repaired sentence; and after all the sentences with the confidence lower than a set threshold value are repaired, final answer content is obtained. And the accuracy and reliability of model output are improved.
Owner:SHANDONG COMP SCI CENTNAT SUPERCOMP CENT IN JINAN +3

Data management method based on sentence semantic perception and related equipment

The invention provides a sentence semantic perception-based data management method and related equipment, and relates to the technical field of cache management, and the method comprises the following steps: segmenting a long text to obtain a plurality of sentences and a sorting sequence of the sentences; calculating importance scores of the tokens corresponding to the sentences based on the sentences and the sorting sequence of the sentences; key tokens in the sentences are screened out according to the importance scores of the tokens corresponding to the sentences and the dynamic budget corresponding to the sentences, the key tokens are stored in a reserved set of the corresponding sentences, and then the local weighted attention score of each token is calculated; performing weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of a preset attention head to obtain a sentence-level semantic vector; and constructing semantic index cache and key value pair (KV) content cache based on the sentence-level semantic vectors of the sentences and the corresponding reserved sets. Therefore, in combination with a KV cache management method of context-dependent dynamic loading, a more efficient and more intelligent reasoning process is realized.
Owner:SHANDONG INSPUR SCI RES INST CO LTD

LLM-based document structuring automatic processing method and system

The invention discloses an LLM-based document structuring automatic processing method and system, and relates to the technical field of document processing, and the method comprises the following steps: S1, document input and preprocessing: removing special characters in a disordered text, unifying a coding format, and adjusting the semantic density; s2, redundancy removal and keyword extraction: removing repeated information in the disordered text, and extracting a first keyword; s3, semantic extraction and structuring: parsing the disordered text sentence by sentence according to the first keyword, and extracting main meaning and related semantic information corresponding to each sentence to obtain an extraction result which comprises a second keyword and sentence meaning of the same sentence; s4, constructing and updating a semantic stack: constructing and updating the semantic stack according to an extraction result, and storing a semantic unit; one semantic unit corresponds to the same extraction result; and S5, structured output: outputting the data in the semantic stack in a preset format. The method is suitable for structured processing of complex disordered documents in multiple fields and multiple types.
Owner:GUANGZHOU YUTONG INFORMATION TECH CO LTD

Teaching course recommendation method and system based on English learning data

The invention discloses a teaching course recommendation method and system based on English learning data, particularly relates to the field of semantic processing, is used for solving the problem of poor pertinence of a traditional English learning course, and comprises the following steps: aiming at systematic difference of native language and English expression on a syntactic structure, constructing a language structure difference map and extracting a structure offset label; the method comprises the following steps: performing dependency syntactic analysis on semantic consistent sentence pairs of multi-language aligned corpora to generate a structural difference feature set; on the basis, a semantic-grammar dimension mapping graph is constructed in a classified mode, and a general structure expression vector is established for graph nodes. A teaching course is divided into knowledge point units in combination with a context label, and a mapping relation between a structure label and the course is established. The system performs structure analysis and semantic matching on sentences input by the learner, identifies structure migration type expression errors, and recommends accurate teaching content according to context and structure labels.
Owner:HUNAN SPORTS VOCATIONAL COLLEGE (HUNAN SPORTS SCHOOL)

Large language model training and error correction method for text error correction and related device

The invention discloses a large language model training and error correction method for text error correction and a related device, and relates to the field of text error correction, and the method comprises the steps: obtaining an error correct sentence pair set, the error correct sentence pair set comprises a plurality of subsets, each subset comprises an error text and at least one corresponding correct text, the error texts in different subsets are different; respectively labeling a positive error correction preference label and a negative error correction preference label for the error text and the correct text according to the error text and the corresponding correct text contained in each subset to obtain a first training corpus; and based on the first training corpus, performing preference tuning training on the initial large language model by adopting preset joint loss to obtain a large language model for text error correction. According to the method, the training corpus is generated based on the wrong text and each corresponding correct text, waste of the correct texts is avoided, model overfitting can be relieved, and the model generalization performance and the error correction accuracy are improved.
Owner:ASIAINFO TECH CHINA INC

Self-adaptive emotion-driven timbre clone character-to-speech method and device

The invention relates to an adaptive emotion-driven timbre clone character-to-speech method and device. Segmenting the target text according to sentences, extracting semantic features, and splitting each sentence into a target phoneme sequence; the method comprises the following steps: splitting a reference text into reference phoneme sequences, splicing the reference phoneme sequences with each target phoneme sequence to obtain a combined phoneme sequence, processing a reference audio to obtain a reference voice feature and a frequency spectrum, processing based on a semantic feature corresponding to a sentence, the combined phoneme sequence and the reference voice feature to obtain a corresponding voice feature, and extracting a corresponding emotional feature from each sentence semantic feature, and processing based on the speech feature, the emotional feature and the target phoneme sequence corresponding to each sentence and the frequency spectrum to obtain a speech signal corresponding to the target text. And extracting emotional features from the semantic features of the target text, enabling the emotional colors of the voice signals to fit the text, and automatically adjusting the emotional colors of the voice signals based on the mapping relationship between the preset output features and the emotional features.
Owner:GUANGZHOU ZIWEIYUN TECH CO LTD

Method and system for aspect-level sentiment classification by merging graphs

System and method for aspect-level sentiment classification. The system includes a computing device, the computing device has a processer and a storage device storing computer executable code. The computer executable code is configured to: receive an aspect term-sentence pair; embed the aspect term-sentence pair; parse the sentence using multiple parsers to obtain dependency trees, and perform edge union to obtain a merged graph; combine the embedding and the merged graph to obtain a relation graph; perform a relation graph neural network on the relation graph; extract hidden representation of the aspect term from updated relation neural network; and classify the aspect term based on the extracted representation to obtain a predicted classification label of the aspect term. During training, the computer executable code is further configured to calculate a loss function based on the predicted label and the ground truth label, and adjust parameters of models.
Owner:CHINABANK PAYMENT (BEIJING) TECH CO LTD

Specific domain model training method based on expert knowledge base

The invention relates to the technical field of text data processing, in particular to a specific domain model training method based on an expert knowledge base, and the method comprises the steps: carrying out word segmentation processing on each sentence in each text in an obtained initial general knowledge base and an initial expert knowledge base; determining a target similarity between every two texts; all texts are clustered, and the initial expert knowledge base and the initial general knowledge base are expanded and deleted respectively; determining a target professional contribution degree corresponding to each target segmented word and a target learning rate corresponding to each sentence in the initial expert knowledge base; and updating the scaling factor, corresponding to each sentence in the initial expert knowledge base, in the LoRA model to a corresponding target learning rate, and performing fine tuning on the language model through the LoRA model to obtain a trained language model. According to the method and the device, the text data in the knowledge base is processed, so that the rationality of training the language model in the specific field is improved.
Owner:BEIJING ZHONGWEI SHENGDING TECH CO LTD

Two-stage text abstract generation method based on PEGASUS model and dynamic error correction

The invention discloses a two-stage text abstract generation method based on a PEGASUS model and dynamic error correction, and the method comprises the steps: obtaining to-be-processed text data, carrying out the preprocessing, obtaining all features, and carrying out the fusion of all features, and obtaining a sentence weight; selecting sentences according to the sentence weights, and selecting key sentences of the abstract; performing hierarchical clustering on the key sentences; identifying potential unlisted words, marking the unlisted words as wrong words, replacing the words marked as wrong words with alternative words, and performing error correction on the alternative words through replacement loss; calculating the comparison loss of the replacement text through a PEGASUS model; and calculating the comprehensive loss through the replacement loss and the comparison loss, adjusting each weight coefficient, the dynamic threshold value and the similarity threshold value until the comprehensive loss reaches the minimum or the number of iterations reaches the maximum, and outputting the abstract. The unregistered words are replaced by different alternative words through a'dynamic error correction mechanism ', so that the optimal selection of the abstract is realized, and the accuracy of generating the abstract is improved.
Owner:YANGZHOU UNIV

Long text matching method and system based on key information and difference characteristics

The invention relates to the technical field of natural language processing, in particular to a long text matching method and system based on key information and difference characteristics, and solves the problems that in the prior art, key information is dispersed due to noise interference in long text processing, and semantics of words or phrases in long text processing are more fuzzy and diversified. According to the method, a long text is preprocessed to obtain a test set sentence pair, a training set sentence pair and a verification set sentence pair, then a training model is obtained through sentence-level information entropy screening, word-level dynamic filtering, semantic difference enhancement and adaptive feature fusion, and finally after verification is conducted through the verification set sentence pair, the test set sentence pair predicts and outputs a result. The system comprises a long text matching system data preprocessing unit, a long text matching system function module training unit and a long text matching system function module result output unit. Key information is extracted for learning in long text matching, a large amount of computing power does not need to be consumed, and the matching effect is improved.
Owner:SHANXI UNIV

Bidirectional information retrieval enhancement generation method for large language model

The invention discloses a bidirectional information retrieval enhancement generation method for a large language model, and belongs to the technical field of artificial intelligence. In order to overcome the defects that noise is introduced and key evidences are omitted due to the fact that traditional RAG only executes'query-document 'one-way retrieval, a two-way semantic perception retrieval enhancement generation model and a two-stage training framework are constructed, wherein in the first stage, the positive / negative example distance is increased in an embedded space in a contrast learning self-supervision mode; in the second stage, fine-grained correlation discrimination is carried out on query-document bidirectional sentences through supervised dichotomy, and probabilistic correlation scores are output; in the reasoning stage, the bidirectional probabilities are fused according to Bayesian to obtain final relevancy, document reordering is carried out, and plug and play can be achieved without fine adjustment of LLM in the whole process. According to the method, the accuracy and consistency of single-hop and multi-hop questions and answers and fact checking tasks are remarkably improved, and the method has the advantages of light weight and low deployment cost.
Owner:中华人民共和国大连海关

Keyword guidance and large language model near-end strategy optimization combined key sentence extraction method

The invention provides a key sentence extraction method combining keyword guidance and large language model near-end strategy optimization. The key sentence extraction method comprises the following steps: constructing a keyword-key sentence pair; the relevancy is evaluated by using a joint matching model, and a reward value is generated; introducing a KL divergence to measure the difference between the training model and the reference model, and estimating the value score of the current state in combination with a state value network; and optimizing the guidance model through a near-end strategy to realize extraction of key sentences. According to the method provided by the embodiment of the invention, the keyword-key sentence pair is constructed, the relevancy of the keyword-key sentence pair is evaluated by using the joint matching model, the reward value is generated, the KL divergence is introduced to measure the difference between the training model and the reference model, the value score of the current state is estimated by combining the state value network, and the guidance model is optimized through the near-end strategy. According to the method, the key sentences are extracted, the problems of high dependence on annotation data and high training cost of a large language model are solved to a great extent, and the key sentence extraction effect is remarkably improved.
Owner:BEIJING INFORMATION SCI & TECH UNIV

Embedded context extraction using natural language models for dynamic remediation

Method and apparatus for dynamic remediation. A set of records associated with a project is accessed. The set of records is processed using one or more natural language processing techniques to generate textual data comprising a plurality of pairs of sentences corresponding to one or more topics associated with the project. An issue for at least one topic associated with the project is identified based on the textual data, comprising identifying a pair of sentences that comprises a first sentence and a second sentence, calculating a sentence similarity score by comparing the first and second sentences using a similarity metric, and determining that the sentence similarity score satisfies one or more criteria. In response to determining that the one or more criteria are satisfied, a project meeting for the issue is scheduled based at least in part on a criticality of the issue.
Owner:INTERNATIONAL BUSINESS MACHINE CORPORATION

Sentence vector generation method and device, matching method and device, and storage medium

The application relates to a sentence vector generation method and device, a matching method and device and a storage medium. The sentence vector generation method comprises the following steps: preprocessing a received original sentence to obtain a filtered stop word sentence and a filtered stop word and entity label marked sentence; processing the filtered stop word sentence to obtain a first sentence vector, wherein the first sentence vector comprises attribute information of characters included in the filtered stop word sentence; processing the filtered stop word and entity label marked sentence and the original sentence to obtain a second sentence vector, wherein the second sentence vector comprises information of the entity label; and a target vector of the original sentence comprises the first sentence vector and the second sentence vector. The target vector of the sentence obtained by the method can contain more effective information, which is beneficial to improving the accuracy of the result output by an intelligent question and answer system.
Owner:ULTRAPOWER SOFTWARE

Translation method and related device

The translation method includes: obtaining a to-be-translated sentence including a polysemous word expressed in a first language; determining a target semantic meaning of the polysemous word according to a disambiguation rule, where the disambiguation rule indicates to determine the target semantic meaning of the polysemous word by using a translation model, the translation model may perform data processing on a word vector sequence corresponding to the to-be-translated sentence to obtain the semantic meaning of the polysemous word in the to-be-translated sentence, the translation model is obtained by training a parallel training sentence pair, and both training sentences in the parallel training sentence pair are the first language; and determining a translation result of the to-be-translated text based on the target semantic meaning, where the translation result includes content whose target semantic meaning is expressed in a second language, and the first language is different from the second language.
Owner:HUAWEI TECH CO LTD

Low-resource language machine translation method based on active learning

The invention provides a low-resource language machine translation method based on active learning, which belongs to the technical field of natural language processing, and comprises the following steps: extracting a token sequence corresponding to each sentence from an initial data set; inputting the token sequence corresponding to each sentence into a preset model, and outputting a feature vector corresponding to each sentence; constructing a feature vector set according to the feature vectors, and selecting a core set from the feature vector set; training a preset translation model by using the core set to obtain an initial translation model, and enabling the initial translation model to output the uncertainty ranking of each core sentence in the core set; acquiring initial data corresponding to each core sentence from the initial data set by utilizing a selection function according to the uncertainty ranking; labeling and serializing all the initial data to obtain a target training data set, and training the initial translation model by using the target training data set to obtain a target translation model. The accuracy of machine translation in a low-resource environment can be improved.
Owner:XINJIANG TECH INST OF PHYSICS & CHEM CHINESE ACAD OF SCI

Text information identification method and device, equipment and storage medium

The invention provides a text information recognition method and device, equipment and a storage medium. In some embodiments of the present disclosure, a first paragraph text and a second paragraph text of an audit report text are obtained; performing sentence segmentation processing on the first paragraph text and the second paragraph text to obtain a first sentence of the first paragraph text and a second sentence of the second paragraph text; combining the first clause and the second clause pairwise to obtain a sentence pair; encoding each sentence pair to obtain a sentence vector corresponding to each sentence pair; performing semantic emotion recognition on the sentence vector corresponding to each sentence pair to obtain a consistency result of each sentence pair; determining consistency information of the first paragraph text and the second paragraph text according to a consistency result of each sentence pair; based on semantic emotion recognition in the field of natural language processing, the audit report text is subjected to consistency auditing automatically, the labor cost is reduced, the auditing efficiency is improved, and the auditing accuracy is improved.
Owner:PICC INFORMATION TECH CO LTD

Method, medium and device for improving automatic evaluation of machine translation quality using retrieval

ActiveCN114896992BMetadata text retrievalNatural language translationSentence pairEvaluation of machine translation
The present invention discloses a method, medium, and device for improving automatic machine translation quality assessment using retrieval. The method comprises: a retrieval phase: for machine translation quality assessment sentence pairs, relevant parallel sentence pairs are retrieved from a database for the words to be assessed in the machine translation quality assessment sentence pairs; and a machine translation quality assessment phase: the retrieved parallel sentence pairs are encoded and incorporated into a machine translation quality assessment model. The present invention can directly and effectively utilize relevant parallel sentence pairs, while also alleviating the problem of sparse training data for machine translation quality assessment. It also better explains the reasons why the model makes relevant decisions, eliminates the need for model retraining, and avoids the drawback of end-to-end models forgetting training data during training, thereby improving the performance of machine translation quality assessment models.
Owner:NANJING UNIV

Grammar error correction method based on large model syntax preference optimization

The invention discloses a grammar error correction method based on large model syntactic preference optimization. According to the grammar error correction method, syntactic relevance between retrieval example sentence pairs and syntactic relevance between the retrieval example sentence pairs are cooperatively utilized. On one hand, a Monte Carlo tree search algorithm is used for exploring the influence of syntactic structure differences among retrieval examples on error correction performance, search path selection is adjusted based on dynamic weights, and syntactic perception corpora containing hidden syntactic association are constructed. On the other hand, by constructing a syntactic preference alignment mechanism, syntactic knowledge contained in syntactic perception corpora is effectively utilized, dynamic adjustment of grammar error correction model parameters is achieved, and then the performance of the grammar error correction model in a complex grammar error correction task is improved. According to the method, a comprehensive contrast experiment is carried out on two Chinese error correction data sets and two English error correction data sets. Experimental results show that the method can effectively integrate the structured syntactic features in the retrieval examples, and effectively relieve the over-correction phenomenon of the grammar error correction model.
Owner:KUNMING UNIV OF SCI & TECH

Text sentence processing method, device, computer equipment and storage medium

ActiveCN111950269BSemantic analysisSentence processingSentence pair
The present application relates to a text sentence processing method, apparatus, computer device, and storage medium, comprising: obtaining sample text sentences containing entity pairs and relationship labels for the entity pairs; extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels, and performing positive and negative sampling processing to obtain a training set; inputting the training set into a relationship extraction model to be trained to generate a loss value including a contrast loss value; the contrast loss value is used to characterize the difference between the similarity of the sentences in the positive sentence pairs and the similarity of the sentences in the negative sentence pairs; adjusting the parameters of the relationship extraction model based on the loss value, and returning to the step of extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels to perform iterative training until a training stop condition is met to obtain a relationship extraction model; the relationship extraction model is used to identify entity relationships between entity pairs in the text sentences. The present method can effectively improve the accuracy of entity relationship extraction.
Owner:TSINGHUA UNIVERSITY +1

Information processing method, apparatus, electronic device, and storage medium

To provide an information processing method, an apparatus, an electronic device, and a storage medium for improving translation quality for a first language sentence, and obtaining a second bilingual sentence pair that is more accurate in translation.SOLUTION: An information processing method using machine learning includes: obtaining a first bilingual sentence pair including a source language sentence and a target language sentence; and obtaining a distilled second bilingual sentence pair by distilling a first language sentence in the first bilingual sentence pair based on a large language model. The first language sentence is the source language sentence or the target language sentence.SELECTED DRAWING: Figure 1
Owner:BEIJING BAIDU NETCOM SCI & TECH CO LTD

Parallel sentence pair construction method, device, electronic device and storage medium

The present invention provides a method, device, electronic device and storage medium for constructing parallel sentence pairs, wherein the method comprises: obtaining a first sentence and a second sentence, wherein the first sentence and the second sentence correspond to different languages; determining a first semantic feature of the first sentence and a second semantic feature of the second sentence based on a cross-language language model, wherein the cross-language language model is obtained by training based on a semantic relationship between each participle in a first sample sentence and each participle in a second sample sentence, wherein the first sample sentence and the second sample sentence correspond to different languages; constructing parallel sentence pairs based on a similarity between the first semantic feature and the second semantic feature, wherein in an embodiment of the present invention, model training is performed using the semantic relationship between the participles contained in sample sentences of different languages, so that the performance of the trained model can be improved, and when facing a scarce resource language, the accuracy of sentence embedding is improved, the construction process of parallel sentence pairs is refined, and the quality of parallel sentence pair construction is improved.
Owner:UNIV OF SCI & TECH OF CHINA +1

Large model Mian machine translation method based on language knowledge retrieval

The invention relates to a large model Mian machine translation method based on language knowledge retrieval, and belongs to the field of natural language processing. Comprising the following steps: constructing tree root nodes of a tree-shaped retrieval structure for parallel corpora from Chinese to Muranju language; dividing the parallel corpora from Chinese to Myanforn into long sentence pairs and short sentence pairs according to sentence lengths; constructing a preliminary tree-shaped retrieval structure through the short sentence pair; inserting the long sentence pair into a tree-shaped retrieval structure; in the retrieval stage, firstly, a to-be-translated text is coded through an LASER model and then matched with related words and sentence pairs in a tree retrieval structure, candidate sentence pairs are selected through a BM25 algorithm, and the cosine similarity between the candidate sentence pairs and the to-be-translated text is further calculated; before the large language model is used for final translation, the text reordering model is used for reordering the context prompt sentence pairs obtained through retrieval, and the context prompt sentence pairs and the to-be-translated text are input into the large language model to obtain a final translation result. The accuracy and smoothness of the translation result can be improved.
Owner:KUNMING UNIV OF SCI & TECH

Weak semantic low-resource character machine translation method based on semantic enhancement

Taking a translation task from Naxi Dongba to Chinese as an example, the invention provides a weak semantic low-resource character machine translation method based on semantic enhancement, which comprises the following steps: S1, designing a Naxi Dongba encoding system, and establishing a Naxi Dongba electronic dictionary; s2, a sufficient number of Naxi Dongba text-Chinese parallel sentence pairs are collected and marked, and a Naxi Dongba text-Chinese parallel corpus is constructed; s3, dividing the data set into a fine adjustment data set and a test data set, and further dividing the fine adjustment data set into a training set and a verification set; s4, constructing a semantic enhancement model based on fine tuning and custom word list embedding; s5, providing an iterative reverse translation method combined with word replacement, and constructing an extended data set; s6, constructing a weak semantic low-resource text machine translation model based on semantic enhancement, adopting an increment updating mechanism, taking the high-quality pseudo-parallel corpus generated in the step S5 as increment, inputting the increment into the semantic enhancement model in the step S3, and adjusting and optimizing the weight of the model through parameters; and S7, inputting the Naxi Dongba coded sentences to be translated into the updated model for translation, and outputting a result. According to the method, translation research from the Naxi Dongba text to Chinese is carried out based on traditional expert experience, automatic translation of the Naxi Dongba text can be achieved, meanwhile, the method has the capacity of continuous learning and adapting to new data, the machine translation effect of weak-semantic low-resource characters is improved, and technical support is provided for research in related fields.
Owner:SOUTHWEST UNIV

A method and system for forming dynamically sized training batches for a neural machine translation system

The present invention discloses a method and system for forming a dynamic size training batch of a neural machine translation system, which is based on the field of neural machine translation. The method comprises: parsing parallel corpora, segmenting and vectorizing the corpora to obtain vectorized sentence pairs; dividing the vectorized sentence pairs into blocks, and sorting the sentence pairs in the vectorized sentence pair blocks in ascending order according to the number of terms; grouping and filling the sorted sentence pairs in the sentence pair blocks according to the batch size, wherein the batch size is the sum of the lengths of all sentence pairs in a batch, so as to form a training batch. The present invention dynamically adjusts the training batch size according to the number of sentence terms in the training of the neural machine translation system, and after sorting, the short sentence pairs in the sentence pair block are placed in front, and the sentence pairs with similar lengths are adjacent. Since the sentence pairs with similar number of terms are adjacent in position in the block after sorting, the sentence pairs in each group after grouping are relatively close in the number of terms, and the sentence pairs that need to be filled and the length of the filling are less. Under the same video memory, a training batch with a larger average length can be formed, thereby improving the training efficiency and effect.
Owner:HUAZHONG UNIV OF SCI & TECH

Training method for Chinese sentence simplification model, Chinese sentence simplification method and device

This application proposes a training method, a Chinese sentence simplification method, and a device for the Chinese sentence simplification model. The training method for the Chinese sentence simplification model includes: obtaining a dataset of complex sentence-simple sentence pairs containing supervisory signals and a Chinese monolingual pre-training model; selecting a simple sentence from the current complex sentence-simple sentence pair as a positive sample in each training batch, and randomly selecting a preset number of simple sentences from other sentence pairs in the same training batch as negative samples; projecting the complex sentence, positive sample, and negative sample into a vector representation space, and obtaining the hidden layer vectors of the last layer of the encoder respectively; calculating the contrastive learning loss, and calculating the cross-entropy loss of the desired simple sentence through the decoder; and jointly training the Chinese monolingual pre-training model by minimizing the contrastive learning loss and cross-entropy loss of the simple sentence output by the Chinese monolingual pre-training model. The simplified model obtained by this method can improve the controllability and fidelity of the generated simplified sentences.
Owner:BEIJING KNOWLEDGE ATLAS TECHNOLOGY CO LTD

A bidirectional information retrieval augmented generation method for large language models

The application discloses a bidirectional information retrieval enhancement generation method for a large language model, and belongs to the technical field of artificial intelligence. In order to overcome the defects of traditional RAG that only performs one-way retrieval from 'query to document' and introduces noise and omits key evidence, a bidirectional semantic perception retrieval enhancement generation model and a two-stage training framework are constructed. In the first stage, the positive / negative example distance is pulled apart in the embedding space in a contrast learning self-supervised manner. In the second stage, a supervised two-classification is used to finely distinguish the relevance of the query-document bidirectional sentence pair, and an output probability correlation is obtained. In the reasoning stage, the bidirectional probability is fused according to Bayes to obtain the final correlation degree, and the document is reordered. The whole process can realize plug and play without fine-tuning the LLM. The application significantly improves the accuracy and consistency of single-hop, multi-hop question answering and fact checking tasks, and has the advantages of light weight and low deployment cost.
Owner:中华人民共和国大连海关

Sentence semantic matching method and device for sentence subject interaction

The invention provides a sentence semantic matching method and device for sentence subject interaction, and relates to the technical field of natural language processing. The method comprises the following steps of: training a sentence semantic matching model, namely extracting a subject of a to-be-matched sentence pair by a feature aggregation network to obtain a distillation subject feature, extracting a keyword of the to-be-matched sentence pair by a subject extraction layer to obtain a subject information sequence, extracting a subject of the subject information sequence by the feature aggregation network to obtain an optimal subject feature, the classification layer carries out classification according to the distillation objective characteristic and the optimal objective characteristic to obtain a matching label; and respectively calculating loss values of the distillation pursuit feature and the optimal pursuit feature, and optimizing a sentence semantic matching model according to the loss values. According to the method, the semantic relation between sentences is comprehensively captured by deeply fusing the extracted contextual subject features of the sentences and the optimal subject features, the similarity of sentence pairs in overall semantics is more accurately judged, matching tags are generated, and an optimized sentence semantic matching model is trained.
Owner:GUILIN UNIV OF ELECTRONIC TECH

Sentence similarity scoring using neural network distillation

The disclosure herein describes a system and method for attentive sentence similarity scoring. A distilled sentence embedding (DSE) language model is trained by decoupling a transformer language model using knowledge distillation. The trained DSE language model calculates sentence embeddings for a plurality of candidate sentences for sentence similarity comparisons. An embedding component associated with the trained DSE language model generates a plurality of candidate sentence representations representing each candidate sentence in the plurality of candidate sentences which are stored for use in analyzing input sentences associated with queries or searches. A representation is created for the selected sentence. This selected sentence representation is used with the plurality of candidate sentence representations to create a similarity score for each candidate sentence-selected sentence pair. A retrieval component identifies a set of similar sentences from the plurality of candidate sentences responsive to the input query based on the set of similarity scores.
Owner:MICROSOFT TECHNOLOGY LICENSING LLC

Code conversion method and device based on sentence pair equivalent replacement, equipment and medium

The application belongs to the field of databases and provides a code conversion method, device, equipment and medium based on statement pair equivalent replacement, which comprises the following steps: obtaining a to-be-converted statement including a first operator and a first logical statement in PLSQL code to be converted, the first operator being used for performing a logical NOT operation, and the logical operation result of the first logical statement being Null; obtaining a target logical operation result of the to-be-converted statement; removing the first operator and converting the first logical statement into a second logical statement with a logical operation result of the target logical operation result; and replacing the second logical statement with the to-be-converted statement to convert the obtained target PLSQL code into target Java code. According to the technical scheme of the embodiment, when the logical operation result of the first logical statement in the to-be-converted statement is Null, the to-be-converted statement can be replaced by the second logical statement in a pair-equivalent manner, so that the operation logic remains unchanged after being converted into a Java statement and the normal operation of the system is ensured.
Owner:CHINA PING AN LIFE INSURANCE CO LTD