165 results about "Participle" patented technology

The invention relates to the technical field of language translation, in particular to a translation precision optimization method and system based on artificial intelligence, and the method comprises the steps: obtaining a source language text, carrying out the phrasing operation on the source language text to obtain a source language sentence sequence of the source text, carrying out the word segmentation operation on each source language sentence in the source language sentence sequence, and carrying out the word segmentation operation on each source language sentence; obtaining a word sequence of the source language sentence; and performing sentiment classification on each source language sentence in the source language sentence sequence based on a pre-trained first sentiment classification model. The sentiment classification is performed on the source language sentences and the target language sentences on the basis of the first sentiment classification model and the second sentiment classification model, and the similarity between the sentences is calculated, so that the sentences after translation are kept consistent in sentiment, the problem of sentiment deviation generated in a machine translation process is favorably avoided, and the translation efficiency is improved. Especially for some texts with strong emotion colors, it is ensured that translated emotion information can be correctly conveyed.

The invention relates to the technical field of natural language processing, in particular to an AI reading management and control method and system based on artificial intelligence, and the method comprises the following steps: carrying out text word segmentation processing on an input original text, segmenting the text into independent sentences, recognizing basic word units in each sentence, analyzing semantic adjacency relationships and syntactic structure features among vocabularies, and carrying out word segmentation processing on the word units; screening and extracting potential phrases representing paragraph meanings, and establishing a candidate semantic unit set; according to the method, through word segmentation, syntactic structure recognition and semantic adjacency analysis of the original text, potential phrases capable of representing paragraph significance are extracted, the candidate semantic unit set is constructed, and modeling of the semantic structure in the text is achieved. And associating the set with the reading fixation duration and the playback action of the user sentence by sentence to obtain a reading behavior response of each semantic fragment, and executing semantic weighting and reading behavior cross analysis according to the reading behavior response. Through the linkage mode, the semantic focus actually focused by the user at present can be recognized.

The invention is suitable for the technical field of semantic understanding, and provides a dialogue type AI multi-intention understanding and decision-making method for a tourism scene, and the method comprises the steps: obtaining a tourism scene dialogue text which is input by a user and is expressed by a dialect; dialect expression is converted into standard Chinese statements through a dialect translation engine, and translation semantic deviation is corrected through a semantic correction model; performing multi-intention modeling on the converted standard Chinese statements, adopting a pre-training language model to combine with dynamic word segmentation processing, aligning special words of dialects to a standard Chinese word list through a dialect word mapping table, and adjusting semantic representation to adapt to dialect features; generating word list probability distribution, and generating a dialect compatible reply containing a multi-intention decision result in combination with an autoregression model; a dialogue state tracking model is trained according to geographic information, user historical interaction data and a cross-dialect corpus, intention switching and composite requests in multiple rounds of dialogues are analyzed, and dialect compatibility and intention recognition accuracy are effectively improved.

The invention discloses an information management method based on intelligent classification and efficient retrieval, relates to the technical field of information management, and solves the technical problem of inaccurate retrieval results caused by lack of deep understanding of information semantics and content. Secondary classification is carried out based on semantics, a standard classification result is determined through an integration and screening mechanism, the information classification accuracy is improved, the core content of information is reflected, word segmentation is carried out through a statistical model to determine retrieval heads, the retrieval heads without retrieval content are removed, the retrieval efficiency is improved, the content to be analyzed is screened by calculating the number proportion, and the retrieval efficiency is improved. Retrieval content is screened according to a scientific quantitative standard, information highly related to user requirements can be quickly positioned, the association strength of a retrieval head and retrieval information is accurately analyzed by means of assigning a grammatical position of a pre-selected retrieval head, calculating cosine similarity and the like, and a strong-association retrieval head and a weak-association retrieval head are distinguished.

The embodiment of the invention provides a long text processing method and device, equipment and a medium. The method comprises the following steps: performing word segmentation on a long text to be processed to obtain a token sequence; the token sequence comprises a plurality of tokens; performing prediction processing on the token sequence by adopting a pre-acquired language processing model to obtain a processed token sequence; the processed token sequence comprises the token sequence and at least one prediction token behind the token sequence; the attention layer of the language processing model is obtained based on fusion of a state space model SSM and a self-attention mechanism, the SSM is used for obtaining global attention of the input token, and the self-attention mechanism is used for obtaining local attention of the input token. According to the method, when the long text is modeled, the effects of reducing the calculation complexity of the model and ensuring the semantic integrity and coherence of model text modeling are achieved.

The invention provides a knowledge base and knowledge graph combined text analysis method and system.The method comprises the steps that for each unregistered vocabulary, similar fields related to the unregistered vocabulary are searched for in at least two word segmentation fragments of a word segmentation sequence, the similar fields in the word segmentation sequence are replaced with the unregistered vocabulary, and the unregistered vocabulary is obtained; obtaining an optimized word segmentation sequence; the method comprises the steps of obtaining an optimized word segmentation sequence, converting the optimized word segmentation sequence into an optimized word segmentation vector, calculating a first similarity based on the optimized word segmentation vector, and generating a summary text and a viewpoint list according to the similarity. According to the method, not only can unregistered vocabularies be automatically recognized, but also the most possible meaning of the unregistered vocabularies can be determined according to the context environment of the unregistered vocabularies; and explanations of different types of texts need to be dynamically adjusted, so that the consistency and accuracy of the meanings of the texts are ensured.

The invention provides a machine translation system and method based on artificial intelligence, and the method comprises the steps: carrying out the word segmentation of each sentence in a source language text through the semantic complexity of the sentences, and obtaining a word set of each sentence; determining the semantic contribution degree of each sentence on the sentence level in the machine translation process according to the co-occurrence distance and the syntactic dependency relationship among the words in each sentence; further determining the semantic association degree between each sentence and all other sentences on a chapter structure, and determining the chapter-level translation attention of each sentence in the machine translation process through all the semantic association degrees and the semantic complexity of each sentence; and performing text-level machine translation on the source language text based on the semantic contribution degree of each sentence on the sentence level and the translation attention of each sentence on the text level to obtain a target language text. By means of the scheme, semantic association translation between chapter-level sentences can be achieved, and therefore the semantic accuracy of machine translation is improved.

The invention relates to the technical field of text data processing, in particular to a specific domain model training method based on an expert knowledge base, and the method comprises the steps: carrying out word segmentation processing on each sentence in each text in an obtained initial general knowledge base and an initial expert knowledge base; determining a target similarity between every two texts; all texts are clustered, and the initial expert knowledge base and the initial general knowledge base are expanded and deleted respectively; determining a target professional contribution degree corresponding to each target segmented word and a target learning rate corresponding to each sentence in the initial expert knowledge base; and updating the scaling factor, corresponding to each sentence in the initial expert knowledge base, in the LoRA model to a corresponding target learning rate, and performing fine tuning on the language model through the LoRA model to obtain a trained language model. According to the method and the device, the text data in the knowledge base is processed, so that the rationality of training the language model in the specific field is improved.

The invention relates to an access control strategy automatic generation system and method based on multi-task learning, and the method comprises the steps: carrying out the word segmentation, cleaning and embedded vector conversion of an original access control text through a data preprocessing module, and constructing a normative input format; the feature sharing layer module is used for extracting deep semantic features of a text through multi-layer bidirectional coding and an attention mechanism and providing unified representation for downstream tasks; the access control statement identification module is used for judging whether each sentence in the text is an access control statement or not and realizing automatic identification of strategy related contents; and the attribute extraction and annotation module is used for annotating words in the access control statements and extracting subject, object and operation access control attributes. A word coding layer and a sentence coding layer are shared, local and global attention mechanisms are combined, key information of a text is extracted, the semantic understanding ability is enhanced, and cooperative training of statement recognition and attribute extraction is achieved; a conditional random field CRF structure is used for sequence labeling, and the structural rationality of attribute labels is ensured.

The invention discloses a translation quality evaluation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-evaluated task; for each to-be-evaluated translated sentence, evaluating the target to-be-evaluated translated sentence based on a target reference translated sentence set associated with the target source text statement to obtain a sentence level evaluation attribute corresponding to the target to-be-evaluated translated sentence; evaluating each to-be-evaluated segmented word in the target to-be-evaluated translated sentence based on a reference translated word in the target reference translated sentence set to obtain a word level evaluation attribute corresponding to each to-be-evaluated segmented word; and according to the sentence level evaluation attribute of each to-be-evaluated translated sentence and the word level evaluation attribute of each to-be-evaluated segmented word, generating a translated text evaluation report corresponding to the to-be-evaluated task. The effect of more objectively and accurately evaluating the overall translation quality of the translated text and the translation quality of the proprietary entity is achieved.

The invention discloses an intelligent equipment fault relation extraction method based on multi-semantic knowledge interaction and dynamic pruning. Performing word segmentation and sentence segmentation on the original equipment fault text to obtain a statement set; obtaining a character-level word embedding representation set of the text by using a pre-training language model; extracting fault entity representations, obtaining fault type representations based on the constructed fault type knowledge base, constructing semantic association between the fault type representations, and generating a context feature representation sequence; calculating a correlation score between the enhanced semantic representation and an original text statement based on a dot product attention mechanism, dynamically selecting a core statement set, focusing the core statement, updating the semantic representation, and inputting the semantic representation into a relationship classification module to predict a relationship type between the entity pairs; and finally, outputting a named entity boundary and a relation label based on conditional random field decoding to obtain a final relation extraction result. According to the method, the ability of understanding the context is enhanced, and the fault relation extraction task under the complex context can be better processed.

The invention relates to the technical field of risk portrait generation based on a knowledge graph, and discloses a risk portrait generation method based on the knowledge graph. According to the scheme, the document is segmented through the sentence end symbols, semantic coherence is ensured, and the accuracy and stability of candidate word extraction are improved. Continuous Chinese character sequence segmentation is adopted, dependence of a word segmentation tool is avoided, the occurrence frequency and position of candidate words are recorded, and a detailed statistical basis is provided for risk factor screening. A minimum occurrence number constant is set, noise interference and information coverage are balanced, the screening flexibility is improved, and candidate word screening is more representative. And counting the co-occurrence relationship of candidate words in the sentence, quantifying the co-occurrence frequency through an indicator function, enhancing word association expression, and providing key data for initial graph construction. Finally, the scheme constructs a knowledge graph based on the candidate words and the co-occurrence relation thereof, ensures clear expression of risk information, and improves the accuracy of the system in risk identification and portrait generation.

The invention discloses a malicious content detection method and device, electronic equipment and a storage medium, which are used for solving the problem of low detection accuracy of the existing malicious content detection method, and the method comprises the following steps: obtaining a to-be-detected text; performing word segmentation processing on the to-be-detected text to obtain a word sequence corresponding to the to-be-detected text; the word sequence is input into a malicious content detection model, sensitive words in the word sequence and context information of the sensitive words are output, the malicious content detection model is obtained through training based on feature vectors, part-of-speech features and dependency syntax features of words in a training sample text, the part-of-speech features represent the types of the words, and the dependency syntax features represent the types of the words; the dependency syntactic features represent a syntactic association relationship between the word and other words in the sentence to which the word belongs. According to the malicious content detection model, on the basis of the features of the words of the training sample text, the part-of-speech features and dependency syntax features of the words are combined for training, key features of the text are highlighted, the context perception ability of the model to the malicious content is enhanced, and the detection accuracy of the malicious content is improved.

The present invention relates to the technical field of language translation, and specifically to a method and system for optimizing translation accuracy based on artificial intelligence, including: obtaining a source language text, performing a sentence segmentation operation on the source language text to obtain a source language sentence sequence of the source text, and performing a word segmentation operation on each source language sentence in the source language sentence sequence to obtain a word sequence of the source language sentence; performing sentiment classification on each source language sentence in the source language sentence sequence based on a pre-trained first sentiment classification model. The present invention performs sentiment classification on the source language sentence and the target language sentence respectively based on the first sentiment classification model and the second sentiment classification model, and calculates the similarity between them to ensure that the translated sentence is consistent in sentiment, which helps to avoid the problem of sentiment deviation in the process of machine translation, especially for some texts with strong sentiment colors, and ensures that the sentiment information of the translation can be conveyed correctly.

The present invention belongs to the technical field of large-model natural language processing, and discloses a text generation method and device based on automated hallucination detection and large models. The method comprises: obtaining an input initial text; extracting a number of vocabulary units of the initial text using a word segmenter; constructing a knowledge database using automated hallucination detection, wherein the knowledge database is composed of a number of knowledge fragments, each of which is associated with a number of candidate words; determining the similarity between the several vocabulary units and each of the knowledge fragments; determining a number of target candidate words based on the similarity; and generating a target text based on the several target candidate words. In the above manner, low hallucination data is obtained from the knowledge base data through an automated hallucination detection and recognition algorithm, and the data in the knowledge base is integrated to adjust the probability distribution of the candidate words in the decoding process to enhance the reliability of the generated text, thereby improving the accuracy of the generated text.

The invention provides a credit granting approval process analysis method and device. Comprising the steps of performing word segmentation on text content in a credit granting approval document set, and combining the obtained segmented words to obtain combined phrases; calculating weighted word frequency-inverse document word frequency of the combined phrases; target phrases with weighted word frequency-inverse document word frequency larger than a word frequency threshold value are screened out from the combined phrases, and an amplified word list is generated according to the target phrases and the original word list; training a target word segmentation device based on the amplified word list; finely adjusting the basic vector model according to the training corpus to obtain a term vector model; analyzing the auditing opinion information based on a large language model to obtain an initial risk analysis rule containing a preset dimension; based on a similarity algorithm, a target word segmentation device and a term vector model, grouping and integrating the initial risk analysis rules to obtain risk analysis rules; and processing the information of each examination and approval stage by using a big language model and adopting a risk analysis rule to obtain examination and approval suggestion information.

The invention provides a multi-word expression-oriented machine translation differential test method aiming at the problem of inaccurate multi-word expression semantic translation in a mainstream machine translation system. The method comprises the following steps that a word segmentation tool based on deep learning is adopted to divide words into vocabulary units, syntactic labels are distributed in combination with a pre-training sequence marking model, and a dependency analysis tool spaCy is utilized to mark the syntactic relation between the words; converting the tagged corpus into a standard CoNLL format, extracting a multi-word expression of a sentence through an automatic tool, and establishing a test data set of a sentence-level and phrase-level corresponding relationship; inputting the test set into a multi-translation system to generate a translation, and using an alignment tool AWESOME to accurately locate a corresponding relationship between a source language and a target language MWEs; the translation similarity is calculated based on BERTScore, mistranslation, translation omission and non-translation are recognized through an intra-group and inter-group dual check mechanism in combination with a dynamic threshold value, and evaluation of the translation accuracy of machine translation on multi-word expression is completed. According to the method provided by the invention, multi-word expression translation errors can be accurately recognized, and the accuracy of phrase-level semantic translation of a machine translation system is finely evaluated through a differential test method.

The invention relates to the technical field of contract intelligent auditing, in particular to an international trade contract intelligent auditing system based on natural language processing, which comprises a data acquisition preprocessing module, a key vocabulary extraction module, a key vocabulary analysis screening module and a text similarity association module, the data acquisition preprocessing module carries out word segmentation and segmentation on a contract text and a regulation standard text, then the key word extraction module extracts a key word set in each text segment through a word frequency statistical method, and in order to prevent the situation that the key word set extracted by the key word extraction module is inaccurate, the key word set is extracted by the key word extraction module. The key vocabulary analyzing and screening module analyzes the paraphrases of the key vocabularies in the corresponding words, removes the key vocabularies which are not nouns in the key vocabulary set, provides a paraphrasing basis for the vocabulary same paraphrasing mapping unit, unifies same paraphrasing word vectors, and transmits the same paraphrasing word vectors to the key vocabulary same paraphrasing mapping unit; and the contract and the regulation text are deeply associated in terms of vocabulary semantics.

The present application provides a method and device for processing polyphones, and a model training method and device, wherein the polyphone processing method includes receiving a Chinese sentence containing polyphones, and determining a first position feature of the polyphones in the Chinese sentence; segmenting the Chinese sentence, and determining the part-of-speech feature of the polyphones based on the words obtained by the word segmentation and the part-of-speech of the words; obtaining a word containing the polyphone, and determining a second position feature of the polyphone in the word containing the polyphone; inputting the first position feature, the part-of-speech feature, and the second position feature into a polyphone disambiguation model to obtain the pronunciation corresponding to the polyphone.

The present invention provides a method, device, electronic device and storage medium for constructing parallel sentence pairs, wherein the method comprises: obtaining a first sentence and a second sentence, wherein the first sentence and the second sentence correspond to different languages; determining a first semantic feature of the first sentence and a second semantic feature of the second sentence based on a cross-language language model, wherein the cross-language language model is obtained by training based on a semantic relationship between each participle in a first sample sentence and each participle in a second sample sentence, wherein the first sample sentence and the second sample sentence correspond to different languages; constructing parallel sentence pairs based on a similarity between the first semantic feature and the second semantic feature, wherein in an embodiment of the present invention, model training is performed using the semantic relationship between the participles contained in sample sentences of different languages, so that the performance of the trained model can be improved, and when facing a scarce resource language, the accuracy of sentence embedding is improved, the construction process of parallel sentence pairs is refined, and the quality of parallel sentence pair construction is improved.

The embodiment of the invention discloses a wrongly written character processing method, device and equipment, and the method comprises the steps: receiving text data of a target domain input by a user, the text data comprising special words of the target domain; performing word segmentation processing on the text data through a preset word segmentation strategy aiming at a target field, and performing identification processing aiming at special words of the target field on segmented words corresponding to the obtained text data to obtain target special words in the segmented words corresponding to the text data; carrying out wrongly written character recognition processing on the target proprietary word through a proprietary word error recognition model for the target field to obtain a first recognition result, and carrying out wrongly written character recognition processing on segmented words, except the target proprietary word, in the segmented words corresponding to the text data through a general word error recognition model to obtain a second recognition result; and based on the first recognition result and the second recognition result, determining wrongly written character information contained in the text data through fusion processing of the two different recognition results.

The present application discloses a pronunciation prediction method and related devices. First, the text to be synthesized is segmented to obtain a segmentation sequence. For the first category of words in the segmentation sequence, the pronunciation information of the first category of words is determined based on a preset corpus resource library. For the second category of words other than the first category of words in the segmentation sequence, their pronunciation categories are determined based on the part-of-speech information of each word in the segmentation sequence, and their pronunciation information is determined based on the pronunciation information determination method corresponding to their pronunciation category. In the present application, combined with the corpus resource library and the preset pronunciation information determination method corresponding to each pronunciation category, it is possible to cover the pronunciation information determination in various situations. Therefore, it is possible to accurately determine the pronunciation information of each word in the text to be synthesized, thereby improving the effect of speech synthesis.

The application belongs to the field of natural language processing, and particularly relates to a method and system for paraphrase recognition based on semantic primitive knowledge and abstract semantic representation, which comprises the following steps: performing word segmentation on a sentence, and performing word-level vector representation and semantic primitive knowledge representation; performing mean value processing on the semantic primitive knowledge representation result, and extracting interactive attention feature information of the mean value processing result by using global semantic information to obtain global semantic primitive representation; performing abstract semantic analysis on a to-be-recognized paraphrase sentence from a sentence structure to obtain a single-root directed acyclic graph, and performing global semantic primitive representation and word-level vector representation; extracting global and local feature information in the order of the directed acyclic graph, and performing distance feature measurement on the information; inputting the distance feature measurement result into a neural network to obtain a recognition result; the application introduces external semantic primitive knowledge to perform semantic representation, the accuracy of the semantic primitive knowledge representation is assisted by global semantic information, and the abstract semantics of a Chinese paraphrase sentence is analyzed to obtain semantic relations, so that the accuracy of paraphrase recognition is improved.

The invention discloses a work order integration method and device based on a large model, equipment and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: determining a target field corresponding to each initial work order, and dividing each initial work order into a plurality of work order clusters according to each target field; performing initial word segmentation processing on each initial work order based on a preset noun knowledge base to obtain a work order after word segmentation, performing vocabulary analysis on each work order after word segmentation, and constructing a target word segmentation table according to a corresponding analysis result; and performing word segmentation processing again on the work orders after word segmentation according to the target word segmentation list to obtain target work orders, refining the target work orders based on the number of the target work orders in the work order clusters and the work order refining large model to obtain corresponding refining results, and integrating the refining results to obtain a refined work order. And obtaining a question list corresponding to each question main body. And through two times of word segmentation processing, the reliability of a work order content extraction result is improved.

This invention provides a method for calculating semantic similarity based on adjacent word features, relating to the field of natural language processing. First, in the example word extraction module, example words are extracted from the example sentence and the sentence to be matched using the longest common substring algorithm. Second, in the example sentence processing module, word segmentation is performed on the example sentence excluding the example words, and part-of-speech tagging is applied to the example words. Then, in the feature extraction module, features are extracted from the words surrounding the example words in the example sentence. The correlation between the surrounding words and the example words and option words is calculated using a corpus. The calculated results are weighted and combined with their respective features to form the features of the example words and option words. Finally, in the option processing module, the part-of-speech of the option words is compared with that of the example words, and a similarity score is calculated based on the features of the option words to select the optimal option word. This invention determines the features of the current word by extracting features from surrounding words and calculating correlation; it uses a corpus to calculate the mutual information between two words to achieve the correlation between the two words.

The present application discloses an input method word frequency adjustment method and device, which are used to solve the technical problem of poor word frequency adjustment effect of input method phrases. An input method word frequency adjustment method includes the following steps: obtaining corpus data; segmenting the corpus data through a word segmentation model to generate a number of word segmentation units; annotating the word segmentation units through a phonetic recognition model to generate word segmentation unit syllables; saving word segmentation units with the same syllables to the same syllable vocabulary; counting the occurrence probability of the first word segmentation unit in the same syllable vocabulary; comparing the occurrence probability of the first word segmentation unit with a preset threshold to obtain a comparison result; adjusting the word frequency of the first word segmentation unit according to the comparison result; arranging the word segmentation unit order of the syllable vocabulary where the first word segmentation unit is located in a preset order according to the adjusted word frequency of the first word segmentation unit, and updating the syllable vocabulary. By dynamically adjusting the word frequency of phrases in the same syllable vocabulary, the accuracy of input is improved.

A system and method guide an Artificial Intelligence engine to determine the semantic and grammatical correctness of a user-expanded sentence in real-time. The sentence validation process involves receiving input from the user, the input includes sentence fragment that the user wishes to expand and user-expanded sentence that the user constructs on the fragment provided. The inputs are broken down into tokens. The word-level tokenization algorithm is used, which identifies tokens by splitting the text into spaces, punctuation marks, and other delimiters. Further, a token comparison algorithm is used to assess the relationship between the sentence fragment and the user-expanded sentence to analyze order and placement. Once the token comparison is complete, a prompt is generated using prompt generator to evaluate grammatical and semantic evaluation of the user-expanded sentence. Real-time feedback is provided to the user based on grammatical and semantic evaluation.

The invention relates to the technical field of data processing, in particular to an intelligent foreign form filling and error correction method and system based on natural language processing. The method comprises the following steps: acquiring text images of a historical corpus and vocabularies; screening segmented words to be corrected; determining the ambiguity of the module based on the frequency proportion of the input vocabulary and the vocabulary type; determining a first possibility based on the co-occurrence probability of the adjacent vocabularies of the to-be-corrected segmented word and the similar segmented word and the similarity of the to-be-corrected segmented word and the similar segmented word; determining a second possibility through an error correction result corresponding to the similar word segmentation history and the image similarity of the word segmentation to be corrected and the similar word segmentation; and determining the total possibility and the necessity of word segmentation error correction through the first possibility, the second possibility and the ambiguity, and completing word segmentation error correction. The error correction accuracy is improved.

The invention relates to an improved key phrase extraction method based on key word screening, the improved key phrase extraction method is suitable for key phrase extraction of a single English patent text, the method is based on an existing KeyBERT technology, a key word screening step is introduced, and the accuracy of key phrase extraction is improved. Performing word segmentation on the patent text, removing stop words and punctuation marks, and extracting key words of the patent text; secondly, generating a key phrase candidate list by utilizing a CountVectorizer function in the KeyBERT, and generating a key phrase candidate list according to the key phrase candidate list; then, screening the candidate key phrases based on strong correlation key words in the field to which the patent text belongs; finally, final key phrases are determined through cosine similarity calculation, and the key word screening step is introduced, so that the number of irrelevant candidate phrases can be effectively reduced, the key phrase extraction precision and efficiency are improved, and the method is particularly suitable for patent text analysis with high technicality and high specialty.