706 results about "Audio segment" patented technology

A method for early diagnosis of Parkinson's disease based on multimodal deep learning is provided. Audio-visual data of a to-be-diagnosed subject while performing a speech task is acquired. The audio-visual data are preprocessed to extract a plurality of audio segments and a plurality of video segments. A face image sequence is extracted from each of the plurality of video segments. A Mel-spectrogram of each of the plurality of audio segments is calculated. The face image sequence and the Mel-spectrogram are input into a multimodal deep learning model to output a classification result for Parkinson's disease early diagnosis of the to-be-diagnosed subject. A system for early diagnosis of Parkinson's disease based on multimodal deep learning is also provided.

The invention relates to the technical field of audio transmission systems, and provides a noise interference compensation method applied to an audio transmission system, and the method comprises the steps: firstly obtaining a noise-containing audio signal set of a target audio transmission channel, the noise-containing audio signal set comprises a plurality of noise-containing audio segments and pure audio segments corresponding to the noise-containing audio segments; then noise interference feature extraction is carried out on the noise frequency signal set to obtain a time domain noise feature and a frequency domain noise feature of each noise frequency segment, and then dynamic feature fusion is carried out on the time domain noise feature and the frequency domain noise feature based on a preset noise compensation model to generate a noise interference feature set; the method comprises the following steps: acquiring a noise interference feature set, training a noise compensation model according to a mapping relation between the noise interference feature set and a corresponding pure audio clip, generating a noise compensation parameter set, finally loading the noise compensation parameter set to a real-time processing module, and performing noise interference compensation operation on a noise-containing audio signal transmitted in real time. Therefore, the influence of noise on audio signals is effectively reduced, and the audio transmission quality is improved.

Methods and systems for generating training data for training a contrastive language-audio machine-learning model. A plurality of audio segments are retrieved from a speech emotion recognition (SER) database along with metadata associated with the audio segments. The metadata of each audio segment includes an emotion class. Words or terms associated with emotions are retrieved from a lexicon. A large language model (LLM) is executed on (i) the classes of emotion associated with the audio segments and (ii) the words or terms from the lexicon. This generates a plurality of text captions associated with emotion, which are stored in a caption pool. For each audio segment retrieved from the SER database, that audio segment is paired with one or more of the text captions from the caption pool that were generated based on the emotion class associated with that audio segment. This yields audio-text pairs for training a contrastive learning model.

The invention relates to the field of voice recognition and analysis, in particular to a voice interaction recognition and analysis method and system based on a Bluetooth headset. The method comprises the following steps: identifying environment audio information of a current wearer based on a Bluetooth earphone; carrying out noise space distribution modeling on the environment audio information of the current wearer, and constructing an environment noise real-time distribution field; carrying out noise distribution change trend analysis on the environment noise real-time distribution field, and carrying out dynamic audio receiving adaptation calculation to obtain an optimal audio adaptation receiving parameter; carrying out real-time interactive audio perception on the wearer according to the optimal audio adaptive receiving parameters, carrying out sound quality detail enhancement processing, and constructing a sound quality detail enhanced audio signal; and performing deep audio semantic analysis and instant semantic ambiguity analysis on the sound quality detail enhanced audio signal, and marking an audio segment of semantic ambiguity. According to the invention, accurate and efficient voice interaction recognition analysis is realized.

The invention discloses a video editing method and device, electronic equipment and a nonvolatile storage medium. The method comprises the following steps: acquiring an audio data stream of a target video, and segmenting the audio data stream into a plurality of audio clips; determining a classification result corresponding to the audio clip, and determining a time period corresponding to the audio clip as a candidate ball hitting time period when the classification result is that the audio clip contains the ball hitting sound; acquiring a video frame corresponding to the candidate ball hitting time period in the target video, and judging that the candidate ball hitting time period is a real ball hitting time period under the condition that the visual feature of the video frame is matched with a preset ball hitting rule; and determining an editing time point according to the real ball hitting time period, and editing the target video according to the editing time point to obtain a ball hitting round video clip. The technical problem that the accuracy of ball hitting detection segment detection is low due to the fact that ball hitting detection is conducted only through sound and is easily interfered by environmental noise in the prior art is solved.

The present disclosure provides a speech synthesis model training method. The method can comprise: acquiring initial training data; selecting a plurality of first text segments, second text segments, and third text segments from coherent text, and acquiring first audio segments, second audio segments, and third audio segments from coherent audio; detecting whether the plurality of audio segments and text segments meet a stitching condition; on the basis of the text sequence, stitching, by alternatively arranging text and audio, the plurality of text segments and audio segments meeting the stitching condition to generate combined training data, so as to generate a combined training data set; on the basis of the combined training data set, training the initial speech generation model to obtain a trained speech synthesis model, wherein during the training, prosodic, tonal, and / or emotional features from the second text segments and / or the third text segments are extracted. Further disclosed is a speech synthesis method. The present disclosure allows for use of the context of the text and audio to realize text comprehension to capture prosody, tonality, and emotion in the context.

The invention discloses a conference voice translation method and device and electronic equipment, the method is applied to the field of artificial intelligence, and the method comprises the steps: collecting original voice information in a conference through target equipment, and extracting an audio clip with sound activity from the original voice information through a voice activity detection model; performing voice recognition on the effective voice information in the audio clip with the sound activity to obtain first text information; and determining a to-be-translated target language type of the target user, and translating the first text information through the target translation model to obtain target text information. Through the conference recording method and device, the problem that the translated conference record is inaccurate due to the fact that traditional conference recording mainly depends on manual recording and translation and is easily influenced by subjective factors of a recorder in the prior art is solved.

The invention relates to a conference summary summarization method based on an AI large model, and belongs to the technical field of voice processing, and the technical scheme specifically comprises the following steps: obtaining audio information, recognizing all spokesmen involved in the audio information by using a preset spokesman separation algorithm, and splitting the audio information into a plurality of audio segments according to the spokesmen; analyzing and processing each audio segment by using a preset voice transcription algorithm to generate a transcription text; and based on a preset large language model, extracting and summarizing the transcriptional text, and generating and outputting summarized contents for describing the audio information, so that a user can know the summarized contents. The method has the effect of optimizing the information definition and the text readability of the transcribed text.

Techniques for performing automatic speech recognition (ASR) processing are described. The ASR processing may involve use of a segmenter and a decoder. The segmenter may be configured to identify audio segments, from audio data containing speech, based on word boundaries. The decoder may be configured to generate multiple word hypotheses for individual audio segments. A fixed-size representation of an audio segment may be generated. The described ASR processing techniques may be computationally less expensive than at least some other systems. Also, the described ASR processing techniques may maintain a larger number of word predictions per audio segment as compared to at least some other systems.

The embodiment of the invention discloses a target voice regulation and control method, device and equipment based on intelligent glasses, belongs to the technical field of intelligent wearing, and solves the problem that in the prior art, a user is difficult to obtain a voice signal meeting the own requirement due to the fact that standardization processing is generally performed on a voice signal. Comprising the following steps: acquiring audio information of a speaker in a current scene, and dividing the audio information into a plurality of audio clips; acquiring portrait information in the current scene, and associating the portrait information with the corresponding audio clip to determine a target speaker; determining corresponding sound preference information of the user in the current scene based on a mapping relation between the noise level and intelligent glasses use preference data; analyzing the sound of the target speaker, responding to a dynamic sound adjustment strategy based on the obtained sound characteristics, and optimizing the sound of the target speaker; and based on the sound preference information, performing frequency band adjustment on the optimized sound through a power amplifier arranged on the intelligent glasses to realize target voice regulation and control.

The invention discloses an audio processing method and device, electronic equipment and a computer readable medium, and relates to the technical field of audios, and the method comprises the steps: obtaining a target data segment of a to-be-processed audio, the target data segment comprising continuous audio data of a first number of frames; inputting the complex frequency spectrum features corresponding to the target data segment into a track division music extraction model to obtain the complex frequency spectrum features of the target audio track as first frequency spectrum features; and obtaining a sub-audio segment corresponding to the target audio track based on the first spectrum feature. Therefore, the complexity of processing the data each time is reduced, the time delay is reduced, the complex spectral features of the audio are used as input, the amplitude and the phase are predicted at the same time, and unnecessary time delay and computing power consumption are avoided.

Methods and apparatus for generating a synchronized video-audio pair. According to an example embodiment, a method of generating a synchronized video-audio pair includes: applying one or more text inputs to a generative model, the one or more text inputs including a speech text, the generative model including a neural network; with the generative model, converting the speech text into an audio segment for the synchronized video-audio pair; and with the generative model, generating a video segment for the synchronized video-audio pair, the video segment including a talking head having lip movements corresponding to the speech text and in synchronization with the audio segment.

The invention discloses a speech recognition method, a speech recognition device and a speech recognition system. The speech recognition method provided by the embodiment of the invention comprises the following steps: acquiring audio stream data; in the process of acquiring the audio stream data, acquiring a multi-modal endpoint signal corresponding to the audio stream data; segmenting the audio stream data based on the multi-modal endpoint signal and a voice activity detection model to obtain at least one audio segment; performing streaming speech recognition on the at least one audio segment based on an offline speech recognition model to obtain a recognition text; the recognized text is streamed. Therefore, the delay of outputting the recognition text is reduced, the occupation of hardware resources is reduced, and the method can be suitable for a real-time interaction scene and a scene with a relatively high recognition precision requirement.

The invention relates to a voice interaction large model family health assistant dialogue method and device, equipment and a medium. The method comprises the following steps: carrying out fragmentation processing according to an original voice stream of a user to generate an audio fragment with a medical mark, and carrying out voice recognition and entity extraction on the audio fragment to generate a dynamic entity map; generating an evidence-based decision prompt based on the map, inputting the prompt into a preset medical big model for processing, and outputting a result containing an essential symptom list; and according to the symptom matching degree of the necessary symptom list and the dynamic entity map, generating a diagnosis report or a question-asking list, if the diagnosis report is output, performing medical rule chain verification operation on the diagnosis report to generate a quality control report, and based on the question-asking list or the quality control report, generating a synthetic voice stream. According to the method, through medical intention directional screening, map entity analysis, large model diagnosis, voice synthesis and the like, the voice recognition accuracy, the diagnosis suggestion reliability and the inquiry interaction efficiency of the family health assistant in the medical scene are improved.

The invention discloses a speaker role separation method, related equipment and a computer program product, and the method comprises the steps: determining a transcription result of target audio data, the transcription result comprising a transcription text and speaker turning point information; and carrying out time alignment on the transliteration text and the target audio data, and segmenting to obtain a single-person audio clip. The voiceprint information is extracted by sliding the window in each single-person audio clip, and the single-person audio clip only contains the content of a single speaker, so that the window length can be increased without worrying about the risk that multiple persons speak in the window, and the voiceprint accuracy can be improved through the larger window length. All sliding window audios are clustered according to voiceprint information in a global range, text fragments corresponding to the sliding window audios of the same cluster are endowed with the same speaker identity, text fragments corresponding to the sliding window audios of different clusters are endowed with different speaker identities, the accuracy of speaker role separation on a transliteration text level can be improved, and the user experience is improved. The method is especially suitable for long audio scenes.

The invention discloses a bipartite graph matching-based speaker recognition method and electronic equipment, and belongs to the technical field of audio recognition. The method comprises the following steps: acquiring an audio stream, and segmenting the audio stream into continuous target duration audio clips in real time; determining a voiceprint embedding vector corresponding to each audio clip of the target duration; determining an embedding matrix based on the voiceprint embedding vector; determining a target voiceprint feature corresponding to each target duration audio clip according to a bipartite graph method; calculating the similarity between the voiceprint embedding vectors; and grouping the voiceprint embedding vectors based on the similarity between the voiceprint embedding vectors, and clustering the voiceprint embedding vectors corresponding to the same speaker to obtain a clustering result and generate a target recognition report. According to the invention, the calculation complexity can be reduced and the identification precision can be improved.

The invention discloses an audio processing method and device, a storage medium and an electronic device, and relates to the technical field of data processing, and the audio processing method comprises the steps: obtaining voice feature data and semantic feature data of original audio data; the original audio data is audio data extracted from the target audio and video data; generating segmentation position data according to the voice feature data and the semantic feature data; and generating audio paragraph data and paragraph visualization data according to the segmentation position data, and providing the paragraph visualization data to a user interaction interface. According to the technical scheme of the embodiment of the invention, the utilization rate of the audio information can be improved, the data processing cost in an application scene is remarkably reduced, and the data application efficiency and scene adaptability are improved.

Embodiments of the present disclosure relate to a method and apparatus for synthesizing a singing voice, an electronic device and a program product. The method comprises obtaining a musical score file with breath-taking identifiers, and segmenting the musical score file into a plurality of musical score segments based on the breath-taking identifiers. The method further comprises generating a plurality of audio segments corresponding to the plurality of musical score segments, and synthesizing a singing voice corresponding to the musical score file based on the plurality of audio segments. In an embodiment of the present disclosure, the musical score file is segmented into a plurality of musical score segments according to the breath-taking identifier, a plurality of audio segments corresponding to the plurality of musical score segments are generated, and finally, a singing voice corresponding to the musical score file is synthesized based on the plurality of audio segments.

The present invention relates to the field of computer vision technology, and discloses a virtual digital human lip synchronization optimization method, device, equipment and storage medium. The virtual digital human lip synchronization optimization method includes: obtaining the target audio segment to be output by the virtual digital human at the next moment; judging whether the target audio segment belongs to the audio type to be processed; if the target audio segment belongs to the audio type to be processed, then based on a preset lip synchronization optimization strategy, generating a 3D human face mouth shape parameter frame sequence corresponding to the target audio segment; based on the 3D human face mouth shape parameter frame sequence, generating a corresponding 3D human face mouth shape image frame sequence and rendering it into the virtual digital human. The present invention can adapt to various audio types and improve the fluency and naturalness of the virtual digital human's mouth shape under different audio types.

The invention relates to the technical field of computers, and discloses an audio processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a target original singing audio segment corresponding to a target song segment in a target song; extracting pitch feature information of the target original singing audio segment; according to the pitch feature information of the target original singing audio segment, through a target singing rating model suitable for the target object, predicting a target singing performance level of the target object for the target song segment; according to the target singing performance level of the target object for the target song segment, determining a target volume proportion suitable for the target object when singing the target song segment; and in response to the singing of the target object to start singing the target song segment, adjusting the volume ratio to the target volume ratio. According to the method, the singing playing effect can be ensured, and the unnatural problem caused by sound correction can be solved.

The invention relates to the field of intelligent customer service, in particular to an automatic intelligent customer service system and a method thereof, and the system comprises an image analysis unit which is used for determining whether to carry out image auxiliary analysis or not according to an image shielding reference value and an image defect reference value of a target area; the auxiliary analysis unit is used for executing image auxiliary analysis; the audio analysis unit is used for determining the category of each audio paragraph as a first-class audio paragraph or a second-class audio paragraph according to the speech speed fluctuation coefficient and the fundamental frequency mutation coefficient; the difficulty evaluation unit is used for determining difficulty coefficients of the audio paragraphs according to the number and the intensity of the first-class audio paragraphs in the target audio; the distribution processing unit is used for determining a task execution condition according to the difficulty coefficient balance degree and correspondingly determining that a task distribution mode is task distribution according to the task difficulty coefficient or the task repetition frequency; according to the invention, the voice recognition task distribution efficiency is improved.

According to the intelligent voice broadcast system and method for the elevator, elevator car environment data are collected in real time through a multi-source heterogeneous sensor set, and passenger density distribution and movement tracks are calculated through fusion of a millimeter wave radar and a 3D visual sensor; physiological features of a handrail contact area are detected through a biological impedance sensor so as to judge the health state of a passenger, and the current signal-to-noise ratio is dynamically calculated and the voice frequency band is optimized through an environment noise analysis module. Therefore, by constructing a technical system of'multi-modal perception-edge intelligent decision-cross-modal interaction 'and adopting innovative means such as millimeter-wave radar and biological impedance fusion sensing, deep reinforcement learning of a dynamic strategy and a phased array directional sound field, the problems of broadcast stiffness, poor scene adaptability and privacy potential safety hazards are solved. According to the system and method, accurate passenger feature recognition, dynamic voice content generation and multi-channel collaborative feedback are achieved through two-dimensional innovation, and the intelligent level and service efficiency of elevator voice interaction are improved.

The invention provides an audio processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a structured list pre-generated for an audio to be edited; analyzing the structured list to obtain metadata of each audio clip; in response to an editing operation for the first audio clip, updating the metadata of the first audio clip to obtain updated metadata; obtaining the first audio clip based on the index of the first audio clip, and decoding the first audio clip to obtain corresponding audio data; creating an audio buffer source node, configuring playing parameters of the audio buffer source node according to the updated metadata, and calling the configured audio buffer source node to play audio data; and in response to receiving the file export request, constructing an offline audio context based on the updated metadata, and generating an audio file through the offline audio context. According to the invention, the audio editing efficiency can be improved.

The embodiment of the invention discloses an animation generation method and device, computer equipment and a storage medium, and belongs to the technical field of computers. The method comprises the steps of performing semantic analysis through a large language model, and determining a first text segment and a first semantic tag in a speaking text; searching a first animation matched with the first semantic tag from an animation library, wherein the semantics expressed by the virtual character in the first animation is matched with the semantics expressed by the first semantic tag; the audio segment corresponding to the second text segment is processed through an animation generation model to obtain a second animation, the second animation comprises an animation corresponding to the second text segment, and the rhythm of the virtual character in the animation corresponding to the second text segment is matched with the speaking rhythm of the second text segment; and generating a target animation of the virtual character based on the first animation and the second animation. According to the method and the device, the animation which is semantic-matched and rich in rhythm sensation is generated, the amount of information contained in the animation is enriched, and the animation playing effect is improved.

The embodiment of the invention provides a multi-mode emotion recognition method and device, electronic equipment, a storage medium and a product, and relates to the technical field of emotion recognition. The method comprises the following steps: acquiring a to-be-recognized audio / video which comprises an audio stream and a video stream, segmenting the audio stream to obtain at least one audio segment, inputting each audio segment into an audio recognition model to obtain an audio recognition result, determining a corresponding video segment in the video stream according to a target audio segment of which the audio recognition result is an emotion result, and inputting the video segments into a video recognition model to obtain a video recognition result, and determining a target emotion result of the to-be-recognized audio and video based on the audio recognition result and the video recognition result. According to the embodiment of the invention, the video emotion recognition is used for assisting the audio emotion recognition to complete the emotion recognition of the audio and the video, errors possibly caused by single audio recognition are avoided, and the recognition accuracy can be improved.

The invention relates to an audio recognition method and device based on marking and backtracking correction, and the method comprises the steps: segmenting a target audio, and enabling adjacent segments after segmentation to have a partial overlapping region; if a slice point exists in a non-mute segment of the target audio and the acoustic feature similarity of a preset number of frames before and after the slice point is smaller than a set similarity threshold value, the slice point is marked as a cut-off risk point, and the cut-off risk point is used for indicating that continuous semantics before and after the slice point has a cut-off risk; after a text corresponding to the audio of each slice is recognized through the speech recognition model, the language confidence degree of an overlapping area where the truncation risk point is located is determined, and the language confidence degree is used for indicating context language logic of the overlapping area; and if it is determined that the language confidence is lower than a set confidence threshold, correcting the recognition texts of the slices before and after the truncation risk point. According to the invention, the accuracy of audio recognition is improved.

Systems and methods for multi-application audio splicing may include (1) receiving, from a third-party application installed on a user's device, an audio segment of an audio file selected by the user via an audio-segment-selection interface presented by the third-party application in association with the audio file, (2) loading the audio segment into a post-creation interface of a social media application installed on the user's device and prompting the user to add visual content via the post-creation interface, (3) creating a social media post that includes (i) the audio segment and (ii) visual content added by the user via the post-creation interface, and (4) posting the social media post to a social media consumption channel. Various other methods, systems, and computer-readable media are also disclosed.

The invention discloses an audio restoration method and device and electronic equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: performing feature extraction on a to-be-repaired first audio clip, a previous-text audio clip and a next-text audio clip of the first audio clip to obtain a first acoustic feature vector of the first audio clip, a second acoustic feature vector of the previous-text audio clip and a third acoustic feature vector of the next-text audio clip; and inputting the first acoustic feature vector, the second acoustic feature vector and the third acoustic feature vector into an audio large model, performing autoregression prediction through the audio large model to obtain a predicted discrete coding vector, and decoding the predicted discrete coding vector to obtain a repaired second audio clip.

The invention discloses an audio editing method and system, a terminal and a storage medium, and the method comprises the steps: determining an original text corresponding to an original audio, and obtaining a target text after part of the original text is modified; deleting an audio segment corresponding to the partial text in the original audio to obtain a reference audio; and based on the original text, the target text and the reference audio, generating a target audio corresponding to the target text. The invention provides an intelligent voice editing processing method which does not depend on a locally installed voice editing system and is high in usability. Moreover, the reference audio is generated after the audio segments corresponding to the partial text are deleted, and then the target audio is generated based on the reference audio, the original text and the target text after the partial text is modified, so that the calculation burden is reduced, and the voice editing effect is improved.