219 results about "Speech transcription" patented technology

Provided herein are computerized methods, media, and systems for real-time interpretation, comprising transcribing to extracted speech from a speaker into text; interpreting the transcribed text for a user using a context-aware machine translation system to produce an interpreted output, wherein the interpretation considers the speaker's intent and the user's context to translate the extracted speech from a source language into at least one target language of the interpreted output, and exporting the interpreted output.

The invention discloses a long video multi-modal understanding and question-answering method and system based on large model and retrieval enhancement generation. The method comprises the following steps: 1) a multi-modal feature extraction module; 2) a multi-modal synchronization and alignment mechanism; 3) constructing a structured memory pool; 4) querying a drive generation mechanism; 5) incremental updating and memory compression strategy; and 6) unifying the multi-modal representation space. The invention provides a long video multi-mode understanding method fusing a large language model and retrieval enhancement generation, and aims to break through the limitation of a traditional method in the aspects of single-mode processing and semantic fragmentation. According to the method, video image features are extracted through a visual model (such as YOLO and ViT), voice transcription and environment voice description are obtained in combination with an audio model (such as Whisper and Qwen-Audio), and unified coding of vision, voice and audio in a long video is achieved. Then, a structured memory pool is constructed through semantic consistency segmentation and timestamp alignment technologies to store time slice data of different modalities.

The invention discloses a conference recording method based on the Internet of Things, and relates to the technical field of conference recording. A microphone array and a direction sensor are adopted to position a spokesman, a wearable microphone is combined to obtain individual voice features, voice is matched with the identity of the spokesman, and in the spokesman recognition process, the voice is recognized through a voice recognition module; the speech streams of different spokesmen are distinguished in combination with a speech round detection technology, a speech time axis is generated, for speech transcription, background noise is removed through a noise reduction technology, meanwhile, semantic compensation is introduced, and texts are corrected in combination with projection and whiteboard writing data collected by IoT equipment, so that the recognition accuracy is improved; the method comprises the following steps: additionally carrying out topic segmentation, converting a text into a high-dimensional vector by adopting a semantic vectorization method, determining a topic boundary by utilizing similarity calculation and a clustering algorithm, carrying out dynamic adjustment in combination with IoT position data and individual voice features, accurately classifying the same discontinuous topic discussion, and preventing cross speaking of different topics from being mistakenly classified into a whole.

The invention provides a psychological intervention method, system and device based on multi-modal emotion recognition and a medium. The method comprises the steps that video data and voice data are preprocessed; performing emotion recognition on a frame image corresponding to each timestamp of the preprocessed video data to obtain an image emotion recognition result corresponding to each timestamp; inputting the audio corresponding to each timestamp of the preprocessed audio data, the first task cue word and the second task cue word into a voice transcription emotion recognition integrated model to obtain a voice emotion recognition result corresponding to each timestamp; obtaining a final emotion recognition result corresponding to each timestamp according to the image emotion recognition result and the voice emotion recognition result; and according to the final emotion recognition result corresponding to each timestamp, generating an emotional state recognition report, and according to the emotional state recognition report, performing psychological intervention on the user. The emotion recognition accuracy is improved, and the psychological intervention accuracy is effectively improved.

A system and method of real-time feedback confirmation to solicit a virtual assistant response from an evolving semantic state of at least a portion of an utterance. A user accesses a virtual assistant on an electronic device having the system and / or method configured to capture a command, a question, and / or a fulfillment request from audio such as, the speech emitted from the speaking user. The speech may be intercepted by a speech engine configured to transcribe the speech into text that is matched with the fragment pattern's regular expression to generate a fragment and / or the speech may be processed with a machine learning model to identify fragments. The fragments are identified by a domain handler configured to update a data structure of the current semantic state of the utterance in real-time on an interface of an electronic device.

The invention provides a two-process error correction method and device for real-time speech transcription, and relates to the technical field of speech processing, and the method comprises the steps: extracting the Mel spectrum features of each segment, inputting each Mel spectrum feature into a lightweight end-to-end model, and obtaining a preliminary transcription text; splicing the segments according to a preset number to obtain a plurality of long segments, and inputting each long segment into a speech recognition model to obtain a high-precision transcription text; performing text comparison on the preliminary transcription text and the high-precision transcription text according to the confidence degree set of the preliminary transcription text to obtain all error vocabularies in the preliminary transcription text; and performing corresponding error correction processing on each error vocabulary in the preliminary transcription text according to the type of the error vocabulary and the high-precision transcription text to obtain a final transcription text. According to the method, through a two-process transcription error correction mechanism of the preliminary transcription text and the high-precision transcription text, transcription error accumulation is reduced on the premise that the real-time performance is not affected, and the transcription accuracy in a complex scene is improved.

The invention relates to the technical field of live broadcast behavior monitoring, in particular to a live broadcast behavior tracking system based on deep learning, which obtains high-quality multi-source information and improves the accuracy of feature analysis by synchronously extracting image and audio data from a live broadcast video stream and combining frame extraction, image enhancement and voice recognition. According to the method, image features are extracted through a pre-trained convolutional neural network, audio features are extracted through a deep learning model, voice transliteration texts are fused, the weight of each modal feature is dynamically adjusted based on an attention mechanism, precise recognition of complex scenes and hidden violation behaviors is achieved, and image camouflage and latent language expression risks are effectively coped with. And the illegal type and confidence are output in real time, once suspected illegal behaviors are detected, alarm, interruption or shielding operation is triggered immediately, and related evidences are uploaded to an auditing database. And efficient, accurate and full-process management and control of the live broadcast violation behaviors are realized.

The invention relates to a speech recognition and transcription method and system based on multi-modal fusion and sentiment analysis, and relates to the field of speech recognizing.The speech recognition and transcription method comprises the steps that a target speech signal and auxiliary modal information of synchronous visual information and text context information are obtained firstly, and the speech signal is segmented and recognized to obtain speech feature vectors; the method comprises the following steps: extracting text context information to obtain a text auxiliary feature vector, carrying out multi-modal fusion on the text auxiliary feature vector and the text auxiliary feature vector to generate fusion feature representation so as to carry out voice transcription to obtain an initial transcription text, and carrying out sentiment analysis according to visual information and the initial transcription text to generate a sentiment feature tag; and finally, optimizing and correcting the initial transliteration text based on the label to obtain a target transliteration text, thereby solving the technical problems that the speech recognition transliteration is difficult to adapt to dialect diversity and the recognition accuracy and robustness are insufficient due to neglect of emotion information, and improving the recognition accuracy and robustness through fusion of multi-modal information and emotion analysis. The voice content can be recognized more accurately, the transcription text can be optimized, and the accuracy and quality of voice recognition transcription are improved.

The invention relates to a conference summary summarization method based on an AI large model, and belongs to the technical field of voice processing, and the technical scheme specifically comprises the following steps: obtaining audio information, recognizing all spokesmen involved in the audio information by using a preset spokesman separation algorithm, and splitting the audio information into a plurality of audio segments according to the spokesmen; analyzing and processing each audio segment by using a preset voice transcription algorithm to generate a transcription text; and based on a preset large language model, extracting and summarizing the transcriptional text, and generating and outputting summarized contents for describing the audio information, so that a user can know the summarized contents. The method has the effect of optimizing the information definition and the text readability of the transcribed text.

The embodiment of the invention provides a conference management system and method fusing a large language model and multi-agent collaboration. The system comprises a multi-modal central language large model and a multi-agent platform, and the multi-modal central language large model comprises a multi-modal interaction module, a full-duplex interaction interface, a task reasoning module, a task planning module and an agent scheduling module; the multi-agent platform comprises a data searching and issuing agent, a agenda management agent, a voice transcription agent and a knowledge base updating agent. According to the embodiment of the invention, the conference process is divided into a plurality of independent intelligent agents, and then the intelligent agents are scheduled and dynamically arranged in a unified manner by the central big model. Strategies can be flexibly adjusted according to real-time situations, and intelligence and high adaptability of the whole process are guaranteed. The conference process and the emotional state are monitored. And the central big model sends an intervention instruction. The active conference management can be realized, a targeted intervention means is provided, and the conference efficiency and the communication effect can be improved.

According to the movie and television play table book extraction method and device, the storage medium and the computer equipment provided by the invention, after an audio and video file of a movie and television play is split into a video file and an audio file, feature recognition is performed on the video file to obtain a subtitle text, speaker face information and a video understanding text; performing voice understanding on the audio file to obtain a voice transcription text and a voice understanding text; wherein the voice transcription text can be corrected into the standard transcription text with high accuracy through the subtitle text. Therefore, based on the face information of the speaker, the line segment of each speaker in the standard transcriptional text and the audio and video file is aligned, so that speaker information with accurate segmentation and semantic coherence can be obtained; and then, through combination with a character side-writing text generated by side-writing analysis on the speaker based on the video, the voice understanding text and the speaker information, table book information is constructed, and related feature description of the character can be covered on the basis of containing the line content, so that the content and depth of the table book are enriched.

The invention relates to the field of classroom teaching analysis, in particular to a classroom teaching analysis method based on a large language model and multi-modal information fusion. According to the scheme, the method comprises the steps that classroom audio is processed through an ASR model, N-best candidate lists are output, each candidate list comprises confidence score and time alignment information, low-confidence lists in N-best candidates are screened by setting a confidence threshold, and only high-confidence candidates are reserved; the N-best candidate list is corrected, and the corrected SRT is obtained; dividing the corrected SRT document into an effective voice transfer time period, a noisy time period and a quiet time period; according to the method, frame extraction is carried out on a classroom video, teacher behaviors and student behaviors are marked, a traditional visual model and a visual large model are trained through marked data, then detection is carried out, a teacher and student behavior time period is obtained, and multi-modal data analysis is carried out as assistance of SRT analysis. The method is suitable for classroom teaching analysis.

The invention discloses an intelligent nursing record generation method and device, and the method comprises the steps: obtaining nursing voice data, and carrying out the noise reduction and sound enhancement processing to obtain a preprocessed voice stream; a speech recognition technology fusing ECAPA-TDNN and an x vector is adopted to recognize medical terminologies, and a text transcription result is generated; recognizing the voice of the target nurse from the preprocessed voice stream and the transcription result in combination with a speaker-independent model and a speaker condition model to obtain a voice transcription text of the voice; performing voice analysis and structured processing on the text by using a multi-modal self-supervised learning model and a hierarchical CNN-BiLSTM framework to obtain a structured text with clinical semantic features; key clinical information is extracted in combination with the medical ontology knowledge base, and a nursing record element set is obtained; and based on the nursing record element set, automatically generating a nursing record meeting the specification. According to the invention, the automatic generation from the nursing voice to the standardized nursing record is realized, and the efficiency and accuracy of the medical nursing record are improved.

Described techniques may be utilized to receive a transcription stream including transcribed text that has been transcribed from speech, and to receive a summary request for a summary to be provided on a display of a device. Extracted text may be identified from the transcribed text and in response to the summary request. The extracted text may be processed using a summarization machine learning (ML) model to obtain a summary of the extracted text, and the summary may be displayed on the display of the device. When an image is captured, an augmented summary may be generated that includes the image together with a visual indication of one or more of an emotion, an entity, or an intent associated with the image, the summary, or the extracted text.

The invention particularly relates to a training reasoning method based on voice-text-image multi-mode comparative learning, and relates to the technical field of artificial intelligence, and the method comprises the steps: constructing a voice-text-GUI screenshot triple, and processing element masks; generating positive and negative alignment pairs of the instruction clauses and the GUI elements; the multiple encoders process corresponding modes, and the Rewriter converts a voice transcription text into a structured sequence; through global / local double fusion and double contrast learning, coordinate loss optimization alignment is carried out. According to the invention, through global and local double-layer fusion and a comparative learning mechanism, fine-grained alignment of voice, text and GUI elements is realized; element masks are generated by means of SAM, post-processing optimization is carried out, accurate element-level visual features are extracted in combination with RoIAlign, and a foundation is laid for alignment; all modal dimensions are unified through linear projection, coarse-grained semantic association is realized through a global fusion layer, and a local fusion layer fuses voice rhythm information to enhance association through cross-modal attention and fine correspondence of a gating mechanism capture sub-step and elements.

The invention relates to the field of voiceprint recognition and voice transfer, and discloses a real-time voiceprint recognition method and system for multi-user overlapped voices. The method comprises the following steps: acquiring a multi-source audio stream, and carrying out standardization and framing processing to generate a streaming audio frame sequence; based on the sequence, through circular buffering, noise reduction, endpoint detection and overlap detection model processing, obtaining an overlap interval label; task assembly, speaker separation, track numbering, voiceprint feature extraction and identity judgment are carried out, and a track identity binding structure is generated; and finally, voice transcription, fragment splicing and conflict cutting are executed, and the voiceprint template library is updated. According to the method, real-time separation and identity recognition of overlapped voices of multiple persons are realized, and the accuracy and robustness of voiceprint recognition in a complex scene are effectively improved.

The invention provides an adaptive hierarchical representation alignment training method and device for a voice large model, and relates to the technical field of natural language processing. The method comprises the following steps: based on an internal voice adapter, according to a candidate voice large model, performing model training by using a source voice and a text cue word to obtain a first voice large model; based on a cross-modal semantic retrieval task, according to the source voice and the source voice transcription text, performing semantic retrieval capability screening on the first voice large model to obtain an optimal neural network hierarchy; based on the optimal neural network hierarchy, according to the source voice, the text cue word and the source voice transcription text, performing prediction through a first voice large model, and calculating model prediction loss; performing parameter optimization on the first voice large model according to the model prediction loss to obtain a second voice large model; according to the adaptive hierarchical representation alignment training method, the mapping relation between the source voice and the transcriptional text is fully utilized, and the adaptive hierarchical representation alignment training method is efficient and accurate.

The invention belongs to the technical field of airport operation information processing, and discloses an airport emergency information release response processing platform and method. The method comprises the following steps: determining a processing strategy; generating an event report by using a preset template, and automatically reporting the event report according to a preset communication protocol; in the disposal process, high-definition video pictures and voice transcription characters are captured in real time and automatically transmitted to a background system, key video clips and pictures are recognized and intercepted through an advanced intelligent analysis algorithm, each link of the whole event disposal is recorded, key data and insight are extracted from recorded information, and the processing efficiency is improved. And optimizing the disposal process. And the processing progress of each event processing center and each event processing terminal is monitored in real time. According to the invention, various dynamic information on site can be quickly captured and understood, detailed disposal records are formed, the disposal process is continuously optimized, and quick and accurate response can be ensured when various emergencies occur.

The invention discloses an intelligent conference voice transcription and summary generation method and system for voiceprint recognition and hot word optimization in the power industry. The method comprises the following steps: acquiring voice samples in advance to construct an encrypted voiceprint database; mining terminologies from multi-source data in the power industry and dynamically allocating weights to establish a hot word library; performing unified decoding and adaptive preprocessing on the input audio; after speech features are extracted, speaker separation and recognition are achieved through deep embedding clustering; integrating the hot word bank, and generating a transliteration text with a speaker tag by adopting a transform-based field adaptive model; and carrying out multi-granularity semantic understanding and hierarchical abstract generation by utilizing the pre-training model in the power field, and automatically outputting a structured conference summary. According to the method, the problems that the accuracy of professional term recognition in the power industry is low, separation of multiple speakers is difficult, and summary generation depends on manpower are effectively solved, and the efficiency and the intelligent level of conference recording are remarkably improved.

The invention provides a spoken language understanding method and device, electronic equipment, a storage medium and a product. The spoken language understanding method comprises the steps of obtaining a to-be-understood voice signal; inputting the voice signal into a trained spoken language understanding model to obtain semantic information of the voice signal; a spoken language understanding model comprising: an automatic speech recognition encoder for converting an acoustic feature of the speech signal into an acoustic representation; the automatic speech recognition decoder is used for performing text transcription on the acoustic representation to obtain a speech transcription text representation; the fusion module is used for fusing the acoustic representation and the voice transcription text representation to obtain a joint representation; the spoken language understanding decoder is used for performing semantic prediction on the joint representation to obtain a semantic representation; and the output layer is used for obtaining the semantic information of the voice signal based on the semantic representation, so that the semantic output can be directly predicted from the voice input through the end-to-end spoken language understanding model, the error propagation problem is reduced, and the accuracy of the semantic information of the voice signal output by the model is improved.

The invention provides a low-delay real-time voice transcription method and system, and the method comprises the following steps: A, carrying out the coding compression of original audio data collected by a client, transmitting the obtained audio coding data to a server through a real-time transmission protocol, introducing an error correction and packet loss retransmission mechanism, and improving the transmission reliability; b, detecting voice in the audio data by a voice activity detection algorithm at the server side, dividing the audio data judged as the voice into a plurality of independent parts, and adding a time sequence identifier for each part; and C, performing real-time transcription on the segmented audio data according to a time sequence to generate corresponding text content.

The invention discloses a conference summary generation method based on voice recognition, relates to the technical field of conference summary generation, and solves the technical problem that the recognition precision of a voice recognition technology is reduced due to the influence of noise in the prior art. The method comprises the following steps: acquiring initial data of enterprise employees and a plurality of types of noise data; respectively training based on the initial data and the noise data to obtain a voice transcription model and a character recognition model; voice data in the real-time conference are transferred based on the voice transfer model; training based on the historical conference content text to obtain a correction model; correcting the real-time conference content text based on a correction model to obtain a target conference content text; recognizing a spokesman in the voice data based on a character recognition model to obtain identity information of the spokesman; and matching the target conference content text with the identity information of the spokesman to obtain a target text, and inputting the target text into the long text cognition large model to obtain a conference summary, so that the technical problem is solved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as financial science and technology and medical health, and discloses a label generation method and device based on emotion recognition, equipment and a medium, and the method comprises the steps: carrying out the preprocessing of a target audio, and obtaining the processed audio data; executing voice transcription based on the processed audio data and extracting acoustic features; performing emotion analysis on the acoustic features and the transliterated text; executing intention and appeal recognition based on the transliteration text; performing enhanced matching on the emotion analysis result and the intention and appeal recognition result in a knowledge base; and generating a label set according to a knowledge base enhancement result. According to the method, the acoustic and semantic information of the audio data is jointly processed, and the knowledge base is combined for enhancing matching to realize multi-dimensional tag generation, so that the recognition accuracy and the industry adaptability are improved, and the structured application of the audio data is supported.

The invention relates to an interview evaluation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target interview video, wherein the target interview video at least comprises video images of interview candidates and voices of the interview candidates and interviewers; generating a voice transcription interview text of the target interview video through a voice transcription service of the first large model, wherein the voice transcription interview text comprises an interview candidate text and an interviewer text; respectively generating corresponding semantic vectors based on the interview candidate text and the interviewer text; generating a plurality of expression vectors corresponding to different specified expressions based on the interview video; and combining the semantic vectors of the two interviews and the plurality of expression vectors into an input vector of an interview evaluation model, inputting the input vector to the interview evaluation model, and outputting an interview score through the interview evaluation model. According to the embodiment of the invention, the comprehensiveness, accuracy and scientificity of interview evaluation can be effectively improved, and the cost is low.

The invention discloses a historical record backtracking method based on LLM, and relates to the field of data retrieval and natural language processing, and the method comprises the steps: collecting various operation data of a user in a local computer, extracting text information in pictures and voices in combination with an OCR technology and an ASR technology, and storing the text information and related information in a database in a unified format; the method comprises the following steps of: processing collected multi-modal data information, firstly performing de-duplication and cleaning on the collected data, then converting a collected screenshot into a video, and finally performing vector embedding on a video image frame, an OCR (Optical Character Recognition) result and a voice transcription text and storing the video image frame, the OCR result and the voice transcription text into a vector database; optimizing the pre-trained LLM in a specific direction by adopting a fine tuning technology; according to the method, conversation with a historical timeline is realized, user input is captured and stored in a query queue, personalized response is generated through LLM interaction in combination with a memory module and an RAG technology, and answers and related image or audio links are displayed.

The invention relates to the technical field of intelligent voice processing and service quality monitoring, and discloses an intelligent quality inspection system and method based on voice transcription and emotion recognition. The method comprises the following steps: firstly, carrying out standardized preprocessing on multi-channel call data; performing voice transcription, speaker separation and verbal skill analysis, and constructing a session context; performing multi-modal emotion recognition and service state labeling by fusing the text and the acoustic features; and performing comprehensive quality inspection analysis in combination with rule matching and model reasoning, and forming closed-loop optimization of a quality inspection strategy. According to the method, the problems of high subjectivity and low coverage rate of a traditional quality inspection method are effectively solved, automatic, multi-dimensional and high-precision quality inspection and risk insight of mass calls are realized, and the efficiency and objectivity of customer service quality inspection are remarkably improved.

A computer-implemented method executed using a computing device comprises digitally generating and storing a machine learning statistical topic model in computer memory, the topic model being programmed to model call transcript data representing words spoken on a call as a function of one or more topics of a set of topics, the set of topics being modeled to comprise a set of pre-seeded topics and a set of non-pre-seeded topics, and the one or more topics being modeled as a function of a probability distribution of topics; programmatically pre-seeding the topic model with a set of keyword groups, each keyword group associating a respective set of keywords with a topic of the set of pre-seeded topics; programmatically training the topic model using unlabeled training data; conjoining a classifier to the topic model to create a classifier model, the classifier defining a joint probability distribution over topic vectors and one or more observed labels; programmatically training the classifier model using labeled training data; receiving target call transcript data comprising an electronic digital representation of a verbal transcription of a target call; programmatically determining, using the classifier model, at least one of one or more topics of the target call or one or more classifications of the target call; digitally storing the target call transcript data with additional data indicating the determined one or more topics of the target call and / or the determined one or more classifications of the target call; accessing, in computer storage, a first digitally stored electronic document comprising a first text; receiving computer input specifying a search query comprising one or more search terms; processing the search query using the classifier model to output a query topic vector representing a thematic content of the search query; processing the first text using the classifier model to output and store in the computer memory a first plurality of topic vectors each representing a topic in the text; using the query topic vector and the first plurality of topic vectors, calculating a plurality of similarity values, each of the similarity values representing a similarity of the query topic vector to a particular topic vector among the first plurality of topic vectors; outputting a visual display that specifies one or more topic vectors among the first plurality of topic vectors having one or more corresponding similarity values that are greater than a specified threshold similarity value.

The invention relates to an automatic depression risk assessment method based on a natural language processing technology. The method comprises the following steps: performing selective conversation content interviews with a subject by using a voice conversation function of artificial intelligence; dialogue recording and text data between the patient and artificial intelligence are obtained; performing automatic transcription and speaker separation on the original speech transcription text data, and constructing a structured question and answer dialogue sequence; generating a structured question prompt related to the depressive symptom by using a large language model, and splicing the structured question prompt with the original dialogue content; designing an artificial neural network pre-training language model coding word vector, fusing position coding, a self-attention mechanism, a problem-guided attention mechanism and a bidirectional long short-term memory network, and extracting deep semantic features related to depression risks; finally, continuous depression risk scores are output, correlation verification is carried out on the continuous depression risk scores and standard PHQ-8 scores, and the accuracy, robustness and interpretability of depression risk assessment are effectively improved.

The invention discloses a multilingual full-speech processing method and device based on speech recognition and a medium, and relates to the technical field of speech recognition, and the method comprises the steps: calculating a language explicit trajectory based on a multilingual speech feature set, carrying out the association with a speech segment through language preference information in a historical session, and constructing a language implicit trajectory, integrating and generating a language weight track; dividing the language weight trajectory into voice trajectory nodes, recording multi-language voice trajectory composite features, connecting the multi-language voice trajectory composite features into a voice trajectory chain, calculating inter-node continuity indexes, and generating a voice trajectory node structure; constructing a track node continuity credibility field according to the voice track node structure, and adjusting multilingual voice recognition decision parameters to generate a multilingual transcription candidate set; and carrying out time sequence splicing and language mark arrangement on the language transcription candidate set to generate a multi-language full-voice transcription result set. According to the method, self-adaptive decoding of structure perception is realized, language switching is optimized, and the transcription precision is improved.

A method, computer program product, and computing system for flowsheet population by voice. Dictation or conversation is transcribed from voice to text. The keys and values relevant to the documentation are extracted from the transcript and used to populate the flowsheet by selecting a subset of rows and examples relevant to the transcription of the information from a plurality of rows in the flowsheet and based on the transcript, each row corresponding to a key and a value for the flowsheet and extracting an instance of information from the transcript of information associated with a key from the subset of rows by processing a prompt including at least a portion of the transcript of information and the subset of rows with a generative artificial intelligence (AI) model using retrieval augmented generation (RAG).