42 results about "Text normalization" patented technology

The invention relates to the technical field of artificial intelligence in the education industry, in particular to an intelligent composition quality evaluation method and system based on a large language model, and the method comprises the steps: carrying out the text normalization and semantic unit segmentation of a composition, extracting a semantic vector through a first large language model in combination with a context enhancement strategy, and positioning a semantic fracture risk position; recognizing composition core elements through a second large language model, and mapping the composition core elements back to the semantic unit sequence; constructing a demonstration logic diagram, extracting a core demonstration path and abstracting the core demonstration path into a logic role topological graph; in combination with a pre-constructed writing specification knowledge graph, comparing structural compliance, connection strength and an expected support relationship, identifying and demonstrating logic defects, and generating a global deduction item list; semantic clustering is carried out on illegal items to form an error label set, comprehensive weight is calculated in combination with historical data of students, and core weak items are positioned; according to the application, the logic analysis depth of intelligent evaluation of the argument is remarkably improved, and the pertinence and practicability of teaching feedback are improved.

The invention discloses a self-adaptive text extraction method and system based on artificial intelligence, and the method comprises the steps: carrying out the analysis of the document structure entropy of an example document set, quantifying the noise density, geometric distortion degree and background complexity of the example document set, and carrying out the self-adaptive selection of a preprocessing assembly line intensity grade according to the above; dynamically configuring image preprocessing parameters and AI recognition model parameters, and generating a recognition engine instance to output a preliminary recognition text; after regularized coarse screening extraction is carried out based on key field description, a multi-candidate generation strategy is started for low-confidence-coefficient candidate text fragments, a multi-person cooperative verification process is triggered for lower-confidence-coefficient fragments, finally all the fragments are processed through a text standardization module, and structured text extraction information is output. According to the method, accurate adaptation of processing intensity is achieved through document quality quantitative evaluation, the extraction accuracy and system robustness of complex heterogeneous documents are effectively improved through a multi-level confidence coefficient verification mechanism, and the identification error risk caused by image quality fluctuation or rule solidification is reduced.

The invention relates to a power transformation equipment defect text standardization method and system in combination with deep semantic matching and a knowledge graph, and the method comprises the steps: inputting an actual power transformation equipment defect text into a text entity recognition model, and extracting an entity in the defect text as a to-be-standardized entity, querying a device model and a defect level corresponding to the defect text, and obtaining a defect device type according to the device model; all the standard descriptions corresponding to the entity types in the defect standard are used as candidate standard entities, the candidate standard entities and the to-be-standardized entities form entity pairs in sequence, the formed entity pairs are input into the FTS-BERT, and the standard descriptions of the entities are obtained through matching; constructing a power transformation equipment defect knowledge graph; in combination with FTS-BERT and knowledge graph search, standardizing a power transformation equipment defect text into a unified form; and adding the new knowledge into the original knowledge graph through incremental updating. The method and the system can automatically and effectively complete a power transformation equipment text standardization task, and solve the problem that defect text data is not standard due to manual recording.

The invention discloses a streaming video subtitle generation method and system based on a localized large model, and a storage medium, and belongs to the technical field of artificial intelligence and natural language processing. The method comprises the following steps: constructing an asynchronous streaming audio extraction queue, and realizing parallel processing of millisecond-level response and background fragmentation of a first audio segment of a video; a decoupled voice activity detection model is adopted to carry out refined slicing on the audio stream, and noise filtering and muting are carried out; text decoding is carried out through a speech recognition model subjected to fine tuning of synthetic data, and term enhancement in professional fields is supported; carrying out multi-modal fusion and time sequence calibration on an identification result to realize sound and picture synchronization and text standardization; and finally, rendering subtitles through a real-time callback mechanism and a cross-platform interface. The method solves the problems of high delay of long video subtitle generation, time axis drift and low terminology recognition rate, is especially suitable for Linux / Windows cross-platform localization deployment, and has high privacy security and low-cost field adaptive capability.

The invention provides a high-priority test case screening method and device, a storage medium and computer equipment, and relates to the technical field of computers.The method comprises the steps that a demand document and a test case set are received; preprocessing the demand document, and constructing a cue word based on a preprocessing result; inputting the cue word and the test case set into a large language model to obtain an initial high-priority test case; after text normalization processing is conducted on the initial high-priority test case, a target high-priority test case is obtained through text matching processing, and the target high-priority test case is marked. According to the method, high-priority test cases can be automatically and accurately screened out in a rapid iteration software development environment, the coverage rate and efficiency of regression testing and smoking testing are improved, meanwhile, the correctness of core functions is guaranteed, and the reliability and applicability of case screening results are remarkably enhanced.

The invention provides an FST + NN-based double-engine German inverse text standardization method, which comprises the following steps of: S1, sending a German spoken input text into an input and preprocessing layer to obtain an input representation which can be stably processed by a rule / FST and lightweight neural module; s2, the candidate fragments provided by the input and preprocessing layer are standardized through a rule and FST modular analysis layer, inverted-order composite word formation, word form change and cross-category combination of German are covered, and output consistency is ensured through a unified disambiguation strategy; s3, candidate fragments which cannot be recognized and converted by the rule and FST modular analysis layer are recognized and transferred through a lightweight neural network auxiliary layer; and S4, converting'types + standard values' given by the rule and FST modular analysis layer and the lightweight neural network auxiliary layer into texts which can be directly used through an output and post-processing layer. According to the method, a finite-state machine is taken as a core, modular modeling is performed on different types of expressions in German, and long digits are decomposed into a plurality of subunits to be identified and combined step by step through a hierarchical analysis mechanism, so that the processing difficulty is effectively reduced, and the method is provided for overcoming the defects of a rule method in flexibility and coverage. A lightweight neural network is introduced as an auxiliary module and is used for identifying unregistered words and special composite structures which are difficult to cover by rules.

The invention discloses a semantic focusing test question duplicate checking method based on a large language model, and the method comprises the steps: completing the construction of a question corpus and metadata labeling based on corpus construction and text standardization; mapping the topics in the corpus into dense vectors by adopting a semantic vectorization representation strategy, and constructing an offline semantic vector library; a semantic vector recall-SimHash denoising-Reranker model rearrangement screening mechanism is provided, and the problem that synonym rewriting cannot be recognized in traditional literal comparison is solved; a multi-level screening-large model deep judgment-online threshold value self-adaptive cooperation framework is provided, and semantic-level accurate duplicate checking is realized through real-time feedback continuous iteration; by starting a review mechanism, a vector recall threshold value and a large model deep judgment threshold value are dynamically adjusted according to data in a manual review library, so that the accuracy and efficiency of duplicate checking are maximized. According to the method, the problems of synonymous rewriting missing net, short text representation failure and static threshold false alarm / missing alarm are solved.

The invention discloses a natural language interaction intelligent customer service system, and relates to the technical field of intelligent interaction. The system comprises a standardization module used for collecting original data from a database of an intelligent customer service system and performing text standardization processing on the original data to obtain standard data, the original data including text data, voice data and image data; the word vector module is used for converting the standard data into word vectors through a word embedding method; the method is technically characterized in that the relations of sentences can be classified more accurately according to sentence meanings, probability distribution is output by using a Softmax function, a final classification result is given, the probability of each category is provided, the prediction result of the model is more interpretable and reliable, and the prediction efficiency of the model is improved. The combination of the cross entropy loss function and Softmax provides a very clear and efficient optimization direction for the model, the logarithmic probability of a correct category can be directly maximized, the gradient is clear, and the convergence speed is high.

Disclosed are some implementations of systems, apparatus, methods and computer program products for categorizing a sentence as a question. Rather than using a single model, several different models are leveraged to determine whether a sentence is a question. For example, the models can include an inverse text normalization (ITN) model, a sentence embeddings model, and a Term frequency inverse document frequency (TFIDF) model. The output of an ITN model is processed using a finite state transducer (FST) while the output of the sentence embeddings model and TFIDF model are processed using logistics regression (LR) models. A support vector machine (SVM) is then applied to the output of the FST and LR models to determine whether the sentence is a question.

The application discloses a natural language interaction intelligent customer service system and relates to the technical field of intelligent interaction; the system comprises a specification module, which is used for collecting original data from a database of an intelligent customer service system and performing text normalization processing on the original data to obtain standard data, wherein the original data comprises text data, voice data and image data; and a word vector module, which is used for converting the standard data into word vectors through a word embedding method; the technical key points are as follows: the relationship of a sentence can be classified more accurately according to the meaning of the sentence, a probability distribution is output by using a Softmax function, the final classification result is given, and the probability of each category is provided, so that the prediction result of the model is more interpretable and reliable, a cross-entropy loss function and the Softmax are combined to provide a very clear and efficient optimization direction for the model, the logarithmic probability of a correct category can be directly maximized, the gradient is clear, and the convergence speed is fast.

The invention provides an information extraction method of a clinical test scheme. The method is applied to the technical field of data processing, and comprises the following steps: receiving a clinical test protocol text, and carrying out text normalization processing; generating context feature representation corresponding to each word in the clinical test protocol text; according to each context feature representation, predicting an entity category of the corresponding word and calculating an entity category prediction loss function; mapping each context feature representation to an L2 normalized embedded vector space to obtain a corresponding embedded vector; calculating a corresponding entity category comparison loss function according to the embedded vector; a corresponding entity variant comparison loss function is calculated, so that the embedded vectors corresponding to the words of the same entity category are mutually gathered in the space; and updating the prediction model parameters according to the entity category prediction loss function, the entity category comparison loss function and the entity variant comparison loss function. In this way, the accuracy of clinical test scheme information extraction can be improved.

In various examples, generating unified text using speech recognition models for AI systems and applications is described herein. Systems and methods are disclosed that use a machine learning model that is trained to generate unified text associated with user speech, where the unified text includes punction marks, capitalizations of words, inverse text normalization formatting, end of sentence (EOS) detections, and / or end of utterance (EOU) detections. For instance, the machine learning model may receive audio data representing speech as input. The machine learning model may then process the audio data and, based at least on the processing, generate output data associated with the speech. In some examples, the output data may represent tokens, such as tokens associated with automatic speech recognition processing, punctuation and capitalization processing, EOS and / or EOU processing, and / or inverse text normalization processing. In such examples, the tokens may then be processed to generate the unified text.

Techniques for performing speaker error correction are described. In some examples, speaker error correction is a post-processing task to be performed on aligned predicted words and predicted one or more speaker identities to jointly perform speaker identities error correction and at least inverse text normalization, wherein the post-processing at least includes: predicting word and speaker contextual features from the aligned predicted words and predicted one or more speaker identities using an encoder, and predicting, from the word and speaker contextual features, inverse text normalization of the aligned predicted words and corrected speaker identities using a decoder.

Embodiments of the present disclosure relate to the technical field of speech processing, and specifically relate to a speech synthesis method and device, computer equipment and a storage medium. The main steps of the foregoing method include: obtaining text data to be synthesized, performing language recognition on the text data, and determining at least one target language category to which the text data belongs. Based on the at least one target language category, the text data is subjected to text normalization processing to obtain normalized text. The unified phoneme sequence corresponding to the normalized text is input into a pre-trained shared acoustic model to obtain target acoustic features; and based on the target acoustic features, synthesized speech is obtained. By using a unified phoneme sequence defined based on cross-language pronunciation features as an intermediate identifier, a single shared acoustic model is used for processing, which significantly reduces the overall parameter quantity, storage space requirement and computing resource consumption of the system, thereby directly reducing the deployment and operation costs.

Systems and methods for providing a resolution recommendation service. The method includes receiving at an interface a ticket; executing at least one of a plurality of processing procedures on the ticket, wherein the processing procedures include a text cleaning procedure, a text normalization procedure, or a text tokenization procedure; and processing the ticket using a pre-trained large language model to extract a context from the ticket, the context including one or more of an element, a relationship, or an intent. The method further includes analyzing one or more knowledge base articles using the model and the extracted context to generate a resolution recommendation and presenting to a user via a user interface the generated resolution recommendation.

The invention discloses a model training method and device, a text processing method and device and related equipment. The method comprises the steps that multiple first texts are acquired; inputting each first text into a large model for text normalization processing to obtain a first output text corresponding to each first text; the first output text corresponding to each first text is input into a rule processing module for text normalization processing, a second output text corresponding to each first text is obtained, and the rule processing module performs text normalization processing in a regular matching mode; screening the plurality of first texts according to the first output text and the second output text corresponding to each first text to obtain a plurality of second texts; constructing a plurality of training samples according to the plurality of second texts; and training the large model according to the plurality of training samples until a preset iteration stop condition is met, thereby obtaining a target model. The text modification accuracy is improved, and support is provided for follow-up TTS speech synthesis.

The invention belongs to the field of software development application data processing, and particularly relates to a software development application data processing method based on an AI large model, which comprises the following steps: adopting a unified acquisition and text standardization processing method for first-line feedback and work order data, and cleaning and marking description information from each terminal; semantic analysis is carried out through an AI large model, and elements such as equipment, processes, functions and problem types are extracted; a hierarchical modeling and relation extraction method is adopted for equipment, process and function information, and an equipment-process-function relation network is constructed; performing path traversal and weighted summary on the text association nodes, and calculating influence scores of each software function; performing task template filling and historical case retrieval on the high-score function; performing effect evaluation and parameter updating on a running result after the development task is online; according to the method, multi-source application data can be uniformly processed, the software function influence is quantified, the development task is automatically generated, and research and development decision refinement is realized.

This invention discloses an intelligent conversion method and system for natural language requirements to Selenium scripts based on a large language model, relating to the field of software engineering technology. The method includes: S1, requirement text normalization; S2, test element structuring; S3, enhanced context construction; S4, script template and location strategy generation; S5, initial script generation and execution; and S6, script adaptive optimization. This invention can accurately map unstructured natural language requirements to structured test elements and generate executable scripts. Throughout the process, it preserves sentence-by-sentence traceability and semantic alignment, allowing each automated test case to trace back to the original requirement. This significantly improves requirement-test consistency and reduces the risk of misjudgment and omission of test cases due to misunderstanding biases. By enhancing context representation, location fingerprints, and execution feedback loops, this solution enables automatically generated scripts to have runtime adaptive correction and self-healing capabilities.

This invention discloses a multi-agent content generation method and system driven by cross-cultural dynamic profiling, belonging to the fields of artificial intelligence and cross-cultural information processing technology. The method includes: S1, collecting multilingual text data through social media interfaces and performing structural verification, language identification, text normalization, deduplication, and quality filtering to construct a cross-cultural corpus knowledge base; S2, dividing the corpus into groups based on clustering algorithms, extracting group features, and constructing structured user profiles; S3, scheduling multiple agents to collaboratively perform content generation, trend analysis, and cross-cultural interaction tasks according to user profile tags and rule-based routing; S4, constructing a virtual audience cluster and pre-evaluating the generation strategy using a simulation mechanism that decouples the generation model and the evaluation model; S5, optimizing and iterating the generation strategy based on the evaluation results to form a closed-loop update mechanism. This invention effectively improves the stability and computational accuracy of the cross-language data processing link.

This application discloses a consultation and companion robot and its method for the elderly, relating to the field of companion robot technology. It acquires user signals through an input interaction module and transcribes them into raw text; a text normalization module performs spoken language mapping and semantic reconstruction of the text based on a domain dictionary and user attributes; a semantic retrieval module performs a hybrid search in a vector library to obtain a candidate set; a context filtering module filters the target policy context based on metadata tags; an inference generation module concatenates the query and context and inputs it into a large language model for source-based inference; and a speech synthesis module outputs emotional audio. This application effectively solves the problems of semantic comprehension bias and information accuracy in policy consultation for the elderly, achieving precise, reliable, and emotionally caring intelligent interaction.

This invention provides a parameter-efficient method for bidirectional fine-tuning of a large recommendation model that integrates collaborative information. The method includes: acquiring and normalizing the input text; designing a MoE architecture with multiple query experts to handle different types of users and integrating user-specific collaborative information into the normalized text; training parameters in the normalized text; and selecting the highest-scoring item as the next item to recommend to the user. This invention can adapt a large recommendation model into an effective recommendation system in a parameter-efficient manner, significantly outperforming state-of-the-art methods.

This paper describes, through various examples, the generation of unified text using speech recognition models for AI systems and applications. Systems and methods are disclosed that utilize a machine learning model trained to generate unified text mapped to user speech, where the unified text includes punctuation, capitalization, inverse text normalization formatting, end-of-sentence (EOS) captures, and / or end-of-utterance (EOU) captures. For example, the machine learning model can receive audio data representing speech as input. The machine learning model can then process the audio data and, based at least on this processing, generate output data mapped to the speech.In some examples, the output data can be tokens, such as tokens associated with automatic speech recognition processing, punctuation and capitalization processing, EOS and / or EOU processing, and / or inverse text normalization processing. In such examples, the tokens can then be processed to generate the unified text.

The application discloses a kind of based on the semantic focus of test question duplicate checking method of large language model, the method is based on corpus construction and text normalization, complete the construction of question corpus and metadata annotation;Using semantic vectorization representation strategy, the question in corpus is mapped to dense vector, and offline semantic vector library is constructed;Propose semantic vector recall→SimHash denoising→Reranker model rearrangement screening mechanism, solve the problem that traditional literal comparison cannot identify synonymous rewriting;Propose multi-level screening-large model deep judgment-online threshold self-adaptive cooperation architecture, through real-time feedback continuous iteration, realize semantic level accurate duplicate checking;By starting mechanism, according to the data in artificial review library Dynamic adjustment vector recall threshold and large model deep judgment threshold, to maximize the accuracy and efficiency of duplicate checking.The application solves the problem of synonymous rewriting, short text representation failure, static threshold false alarm / miss.

The application belongs to the field of software development application data processing, and particularly relates to a software development application data processing method based on an AI large model, which comprises the following steps: adopting a unified collection and text standardization processing method for first-line feedback and work order data, cleaning and labeling description information from each terminal; performing semantic analysis through an AI large model to extract elements such as equipment, process, function and problem type; adopting a hierarchical modeling and relationship extraction method for equipment, process and function information to construct an equipment-process-function relationship network; performing path traversal and weighted aggregation on text associated nodes to calculate the influence score of each software function; performing task template filling and historical case retrieval on high-score functions; and performing effect evaluation and parameter updating on the running results of the online development task; the application can uniformly process multi-source application data, quantify the influence of software functions and automatically generate development tasks, and realizes fine research and development decision-making.

The application discloses a multi-layer text representation and semantic reconstruction method and system for Chinese scene text, and relates to the fields of computer vision and natural language processing. In view of the problems that the prior art cannot distinguish between creative expression and real error, the processing flow is rigid and lacks interpretability, the application first detects and identifies image text to generate original text; then performs spelling correction and grammar correction on the original text to obtain standardized text; then extracts and analyzes the differences between the original text and the standardized text to generate explanation information containing error types and sources; finally, according to the application scene requirements, the original text, the standardized text or the explanation information is flexibly output. The application improves the text processing accuracy, retains the creative expression, and enhances the interpretability and scene adaptability of the system, and is suitable for intelligent translation, advertising creativity and visual impairment assistance scenes.

Solutions for custom display post processing (DPP) in speech recognition (SR) use a customized multi-stage DPP pipeline that transforms a stream of SR tokens from lexical form to display form. A first transformation stage of the DPP pipeline receives the stream of tokens, in turn, by an upstream filter, a base model stage, and a downstream filter, and transforms a first aspect of the stream of tokens (e.g., disfluency, inverse text normalization (ITN), capitalization, etc.) from lexical form into display form. The upstream filter and / or the downstream filter alter the stream of tokens to change the default behavior of the DPP pipeline into custom behavior. Additional transformation stages of the DPP pipeline perform further transforms, allowing for outputting final text in a display format that is customized for a specific user. This permits each user to efficiently leverage a common baseline DPP pipeline to produce a custom output.

The invention discloses a Unicode confusion command attack detection system and method based on a dual-granularity feature coupling model, and belongs to the technical field of network security. In order to solve the problem of accurate recognition of confused malicious commands, a data preprocessing module, a standardization module, a character-level confusion feature extraction module, a word-level semantic feature extraction module, a dual-granularity feature coupling module and a malicious command judgment module are connected in sequence; the data preprocessing module is used for cleaning command line data and executing command line text standardization operation; the standardization module is used for replacing a specific mode with a universal placeholder and performing word segmentation operation; the character-level confusion feature extraction module is used for extracting character-level features of command keywords; the word-level semantic feature extraction module is used for extracting word-level features; the dual-granularity feature coupling module is used for forming a mixed feature vector; and the malicious command judgment module is used for inputting the mixed feature vector into a pre-training random forest classifier to output a binary classification result.

The application relates to the technical field of computers, in particular to an AI-based after-sales service quality inspection analysis method and system, which comprises the following steps: obtaining after-sales interaction data, including customer voice, voice transcription text, complaint work order text, online evaluation text, satisfaction score and business field; performing deduplication, missing arrangement, text normalization, turn division, voice text alignment and field normalization on the after-sales interaction data to obtain standardized quality inspection samples; performing text semantic understanding and acoustic state recognition based on the standardized quality inspection samples to obtain customer emotional states by fusion; identifying after-sales service items and quantifying emotional results, combining work order attributes and satisfaction scores to form a quality inspection sample set; generating complaint attribution labels, after-sales problem abstracts, business responsibility clues and processing urgency levels by using a semantic quality inspection model, training a satisfaction risk prediction model and calculating the global importance of each after-sales service item; and generating quality inspection analysis results accordingly.

The invention discloses an aggressive speech detection method based on the combination of multi-view semantic reasoning and a self-check mechanism for training memory wakeup. The method comprises the following steps of 1, standardizing an aggressive speech text; 2, constructing a multi-view semantic reasoning normal form; step 3, a self-checking mechanism based on confusion and accuracy; 4, sample construction based on a training memory wake-up mechanism; and 5, performing model loss training and aggressive speech detection. According to the method, the deep semantics of the sentences can be accurately understood, the existence of aggressive speech in the sentences can be accurately recognized, and the limitation of a traditional aggressive speech detection method on the processing capacity and context understanding of complex aggressive speech is improved.

The invention relates to generating unified text using speech recognition models for conversational AI systems and applications. In examples, generating unified text for AI systems and applications using a speech recognition model is described herein. Systems and methods are disclosed for generating unified text related to user speech using a trained machine learning model, wherein the unified text includes punctuation marks, word capitalization, inverse text normalization formats, sentence end (EOS) detection, and / or utterance end (EOU) detection. For example, a machine learning model may receive, as input, audio data representing speech. The machine learning model may then process the audio data and generate output data related to the speech based at least on the processing procedure. In some examples, the output data may represent markings, such as markings related to automatic speech recognition processing, punctuation and capitalization processing, EOS and / or EOU processing, and / or inverse text normalization processing. In these examples, the tokens may be processed to generate unified text.