203 results about "Audio synthesis" patented technology

The invention provides a vehicle audio control method and device, a vehicle and a storage medium. The method comprises the following steps: acquiring environment information outside the vehicle and object information of a target object; wherein the object information comprises object type information and object position information; determining an analog audio according to the object type information, and determining an environment audio according to the environment information; synthesizing a target audio according to the analog audio and the environment audio; and performing spatial sound effect adjustment on the target audio according to the object position information, and playing the adjusted target audio. According to the invention, a driver can perceive multi-dimensional information through the target audio and make more accurate judgment and driving operation in time, so that the driving safety of the vehicle is effectively improved.

An audio synthesis method, an audio synthesis model training method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which relate to artificial intelligence technology. The method includes: invoking an audio synthesis model based on language information and preset style information of a target text to perform following processing, the audio synthesis model including a prior encoder and a waveform decoder: generating audio features corresponding to the target text based on the language information and the preset style information by using the prior encoder; performing normalizing flow processing on the audio features by using the prior encoder, to obtain a hidden variable of the target text; and performing waveform decoding on the hidden variable of the target text by using the waveform decoder, to obtain a synthetic waveform conforming to an audio style described in the preset style information and corresponding to the target text.

The invention provides a music generation method, a music generation device, electronic equipment and a storage medium, and the music generation method comprises the steps: after receiving a music generation instruction, generating an accompaniment audio and a singing audio according to the music generation instruction, carrying out the audio synthesis of the accompaniment audio and the singing audio, and generating and outputting target music. Therefore, when the method is applied to the vehicle intelligent cabin scene, the accompaniment audio and the singing audio are respectively generated by adopting an artificial intelligence technology, and the accompaniment audio and the singing audio are synthesized to obtain the target music, so that the adaptation degree of the accompaniment audio and the singing audio in music content creation can be effectively improved; the accompaniment audio and the singing audio are closely fused in multiple dimensions such as rhythm, tone and emotional expression, so that the user experience is greatly improved.

The embodiment of the invention provides a voice generation method and device, computer equipment, a computer readable storage medium and a computer program product, and belongs to the field of audio processing. The voice generation method comprises the following steps: acquiring a content text and an emotion description text; determining an emotion weight array according to the emotion description text; determining a target emotion vector according to the emotion weight array and a plurality of basic emotion vectors; the target emotion vector and the content text serve as model input, first target voice is generated through a pre-trained audio synthesis model, and the first target voice comprises text content in the content text and emotion features in the emotion description text. According to the technical scheme provided by the embodiment of the invention, emotion control on the first target voice can be realized by utilizing the emotion description text, so that the voice generation stability and the emotion controllability and accuracy in the voice generation process are improved.

A system and method are provided for dynamically generating interactive multimedia storytelling experiences using integrated artificial intelligence models. The system comprises a generative language model for producing narrative content in response to user input, a generative video synthesis module for visualizing story segments, and a generative audio synthesis module for producing synchronized speech, effects, and music. In alternative embodiments, a single multimodal generative model may perform both video and audio synthesis. A user interaction module accepts free-form input to evolve the story in real time, and a content generation coordinator manages orchestration, timing, and latency optimization between components. The system supports modular architecture, lip synchronization with character visuals, predictive pre-generation to reduce delay, personalization based on user profiles, and deployment across various platforms including desktop, mobile, and extended reality environments. The invention enables open-ended, user-driven narrative generation with seamless and adaptive audiovisual synthesis.

The invention discloses a hearing aid health analysis method based on a physiological signal monitoring function. The method comprises the following steps: collecting physiological signal data of a user in real time through a multi-mode biosensor array integrated on a hearing aid body; the hearing aid body comprises a digital signal processor and a wireless communication module; the physiological signal data at least comprises a brain wave signal, a skin electric response signal and a temporal artery pulse signal; performing motion artifact elimination processing on the physiological signal data through an adaptive filtering algorithm; performing feature extraction on the preprocessed physiological signal by using an embedded health analysis engine to generate a composite health feature vector containing a sympathetic nerve activity index, cognitive load measurement and a cardiovascular function index; and dynamically generating a health risk assessment report based on an association relationship between the composite health feature vector and a preset hearing scene, the bone conduction vibration unit and the audio synthesis module of the hearing aid body output graded health alarm signals.

The embodiment of the invention belongs to the technical field of research and development design and audio processing, is applied to pre-processing scenes of audio reconstruction and audio synthesis, and relates to an audio reconstruction method, device and equipment and a storage medium thereof. A potential feature representation is obtained through encoding by an audio encoder; inputting the potential feature representation into a single quantizer of which the codebook space size is expanded, and performing quantization processing to obtain discretized audio feature representation; carrying out decoding processing on the discretized audio feature representation by adopting an improved decoder after adversarial training to obtain a real value signal; converting the real value signal to obtain a signal spectrogram; and inverse Fourier transform is carried out according to the phase and the amplitude value of the signal spectrogram, and reconstruction of the original audio stream is completed. The codebook space of the quantizer is expanded, the decoder is improved, and the adversarial training mode is adopted to carry out audio coding and decoding training, so that the high availability of the reconstructed audio is ensured when the audio is actually reconstructed.

The embodiment of the invention provides an audio synthesis method and device, a medium and equipment, and relates to the technical field of speech synthesis. The method comprises the steps of obtaining a target text to be subjected to audio synthesis; splitting the target text into a plurality of text units with complete semantics to obtain a task sequence; for the ith text unit in the task sequence, scheduling influence factors are obtained before the audio synthesis task is executed, the scheduling influence factors comprise application layer semantic information and / or equipment real-time performance indexes, and i is a positive integer; determining a target synthetic link from a cloud synthetic link and an end-side synthetic link based on the scheduling influence factors; and completing audio synthesis of the ith text unit through the target synthesis link. According to the scheme provided by the embodiment of the invention, in the audio synthesis process of the target text, adaptive switching between the cloud audio synthesis link and the end-side audio synthesis link can be realized in a low-delay manner.

The invention provides a method and device for generating audio based on a large model, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence such as audio synthesis and large models. According to the specific implementation scheme, in the process of generating a text by a large model, characters generated in real time are obtained; based on the characters, adopting a pre-trained audio generation model to sequentially obtain audio features of each audio unit of the characters; the audio features of the audio units are discretized audio features, and the characters comprise audio features of a plurality of different audio units; and based on the audio features of the audio units, synthesizing corresponding audios by adopting a pre-trained vocoder. According to the technology disclosed by the invention, the time delay of generating the audio of the text output by the large model can be effectively shortened, and the naturalness and fluency of the synthesized audio are improved, so that the user experience of voice interaction based on the large model can be improved.

The invention discloses an interactive digital human generation method based on picture and audio synthesis, and belongs to the crossing field of artificial intelligence and computer graphics. According to the method, a full-link process of'feature extraction-model construction-emotion driving-real-time interaction-video output 'can be automatically completed only by uploading a character picture and a section of audio by a user: key point and semantic feature extraction is performed on the picture to obtain face / posture information; performing voice recognition, semantic analysis and emotion recognition on the audio to obtain a semantic tag and an emotion parameter; generating a personalized three-dimensional digital human based on the information, and driving the personalized three-dimensional digital human to generate facial expressions and limb actions which are synchronous with emotions; the user intention is analyzed in real time through natural language understanding and computer vision, and multi-modal interaction is achieved. According to the method, digital human creation can be completed without professional modeling and motion capture equipment, the generation cost is reduced, and the method can be widely applied to virtual anchors, online education, intelligent customer service and movie and television entertainment scenes.

The invention discloses a cross-culture melody feature extraction and musical instrument matching system based on deep learning, and belongs to the technical field of music information processing and artificial intelligence. The system comprises an audio acquisition module, an audio preprocessing module, a melody feature extraction module, a cultural context understanding module, an emotion semantic analysis module, a musical instrument timbre database, a musical instrument matching recommendation module, a distributor scheme generation module, a user interaction module and an audio synthesis module. Multi-dimensional melody features are extracted through a multi-scale convolutional neural network and a bidirectional long-short term memory network, cultural semantic understanding is realized through reasoning on a music knowledge graph by using a graph neural network, and musical instrument matching is performed by using a multi-objective optimization algorithm in combination with three dimensions of timbre integrating degree, cultural consistency and emotional expressive power. According to the system, automation of the whole process from humming melody to musical instrument configuration is achieved, the efficiency of the instrument is improved by 400%, the accuracy rate reaches 92%, and an intelligent tool is provided for cross-culture music creation and national music modernization reorganization.

The invention relates to the field of data processing, and provides a PDF video generation method, device and equipment and a storage medium, the method analyzes an uploaded PDF document based on an OCR library, identifies text content and image elements in each page, and retains a visual effect in the PDF document through an image processing technology, so that the generated video is more attractive; generating a video content script according to the text content and the image element; converting the text content into voice through a natural language processing algorithm; synthesizing the text content and the image elements into a preliminary video through an audio synthesis library and a video synthesis library; according to the video content script, the voice and the image in the preliminary video are synchronized step by step, the final video is generated, and the PDF document content is converted into the video, so that not only can characters be displayed, but also elements such as images and charts can be presented, displayed information is more vivid, a better visualization effect is achieved, and information transmission is convenient.

Various example user interface (UI) mechanisms are described herein, each of which enables a user to efficiently and intuitively manipulate a trained generative model at runtime. Among other things, the described techniques have applications in the field of game design or application design more generally, enabling a game or application developer to easily generate extended application (e.g., gameplay) sequences. Other applications include guided image or audio synthesis, or other forms of guided output generation (e.g., synthesized code, simulated or actual industrial outputs, engineering data, cybersecurity data etc.).

A method of generating audio synthesizer control signals, the method comprising: receiving at a processing component, from a multi-dimensional orientation sensor of a controller device, inputs denoting sensed changes in orientation of the control device in first and second angular dimensions; processing the inputs to generate audio synthesizer control signals; and outputting the audio synthesizer control signals to an audio synthesizer; wherein the audio synthesizer control signals cause a first audible characteristic of the audio signal to be varied responsive to changes in orientation in the first angular dimension, and a second audible characteristic of the audio signal to be varied responsive to changes in orientation of the control device in the second angular dimension.

The invention provides a Schrodinger bridge-based spatial audio synthesis method and system, and the method comprises the steps: obtaining a monaural sound source signal, and constructing a prior signal and a noisy representation based on the monaural sound source signal; inputting the prior signal and the noisy representation into a pre-trained spatial audio synthesis model, and generating a final dual-track spatial audio through the spatial audio synthesis model based on a stochastic differential equation iterative sampling path; wherein the spatial audio synthesis model is obtained by training a preset neural network model through a predefined dual-channel Schrodinger bridge parameterization target and boundary auxiliary supervision. According to the invention, the problems of slow synthesis speed and poor quality of the existing spatial audio are solved.

The invention discloses a mimicry audio generation method based on feature semantic distinguishing and model single-step screening, and belongs to the technical field of computer audio synthesis and signal processing. The implementation method comprises the following steps: 1, extracting Mel spectrum features from an audio sample, forming a frequency domain spectrogram of the audio through short-time Fourier transform, and performing frequency domain mapping and logarithmic compression on the frequency domain spectrogram by using a Mel filter bank to form a Mel spectrogram; 2, training an audio codec configured with a multi-stage residual quantizer by using fusion loss of semantic classification loss and supervised reconstruction loss to obtain audio features with semantic differentiation; 3, training the audio diffusion model with multi-task diffusion loss; and 4, performing single-step screening on the voice audio by using the trained audio diffusion model in combination with the language audio similarity to obtain a de-noised audio. Compared with the prior art, the technical problem of efficiently generating the high-quality audio consistent with the input semantics when the high-quality vivid scene sound effect is simulated is solved.

An audio synthesis method and apparatus, an audio synthesis model training method and apparatus, an electronic device, a computer readable storage medium, and a computer program product, relating to artificial intelligence technology. The audio synthesis method comprises: on the basis of language information of a target text and preset style information, calling an audio synthesis model to execute the following processing, wherein the audio synthesis model comprises a prior encoder and a waveform decoder: by means of the prior encoder and on the basis of the language information and the preset style information, generating audio features corresponding to the target text; by means of the prior encoder, performing normalizing flow processing on the audio features to obtain hidden variables of the target text; and by means of the waveform decoder, performing waveform decoding processing on the hidden variables of the target text to obtain a waveform that conforms to an audio style described by the preset style information and corresponds to the target text.

The embodiment of the invention belongs to the technical field of research and development design and audio processing, is applied to a pre-processing scene of audio synthesis, and relates to an audio signal generation method and device, equipment and a storage medium thereof. Audio feature extraction is carried out; obtaining an audio phase spectrum and an audio amplitude spectrum through a phase amplitude prediction network; and taking the audio phase spectrum and the audio amplitude spectrum as an audio signal generation reference spectrum, and carrying out audio signal generation on the audio features by adopting the audio signal generation network after the adversarial generation training is completed. According to the audio signal generation method, the phase amplitude prediction network is introduced, and phase information is fully considered during subsequent audio signal generation, so that the quality and definition of audio signal generation are improved. The method is applied to virtual voice synthesis scenes of intelligent voice customer service in the field of financial science and technology, phase information of reference voice is fully combined, and the quality and definition of synthesized voice are improved.

A device may receive and convert audio data to text data in real-time, and may detect a network fluctuation that causes missing voice packets. The device may process partial text and context of the text data, with a model, to generate a new phrase, and may generate a response phoneme for the new phrase. The device may utilize a text embedding model to generate a text embedding for the response phoneme, and may process the audio data, with the model, to generate a target voice sequence. The device may utilize an audio embedding model to generate an audio embedding for the target voice sequence, and may combine the text embedding and the audio embedding to generate an embedding input vector. The device may process the embedding input vector, with an audio synthesis model, to generate a final voice response, and may provide the audio data and the final voice response.

The present application provides an audio generation method, device, electronic device and computer-readable storage medium based on artificial intelligence; it relates to artificial intelligence technology; the method includes: encoding a phoneme sequence corresponding to a text to obtain a context representation of the phoneme sequence; determining the alignment position of the first frame implicit state relative to the context representation based on the first frame implicit state corresponding to each phoneme in the phoneme sequence; when the alignment position corresponds to a non-end position in the context representation, decoding is performed based on the context representation and the first frame implicit state to obtain a second frame implicit state; synthesis is performed based on the first frame implicit state and the second frame implicit state to obtain audio data corresponding to the text. Through the present application, the accuracy of audio synthesis can be improved.

The application discloses a hearing aid health analysis method based on physiological signal monitoring function, comprising the following steps: collecting physiological signal data of a user in real time through a multi-modal biological sensor array integrated in a hearing aid body; the hearing aid body comprises a digital signal processor and a wireless communication module; the physiological signal data at least comprises an electroencephalogram signal, a galvanic skin response signal and a temporal artery pulse signal; the physiological signal data is subjected to motion artifact elimination processing through an adaptive filtering algorithm; a preprocessed physiological signal is subjected to feature extraction through an embedded health analysis engine, and a composite health feature vector comprising a sympathetic nerve activity index, a cognitive load metric and a cardiovascular function index is generated; a health risk assessment report is dynamically generated based on the association between the composite health feature vector and a preset hearing scene; and a graded health alarm signal is output through a bone conduction vibration unit and an audio synthesis module of the hearing aid body.

The invention discloses an audio file construction method, audio processing equipment and a storage medium, an electrocardiogram corresponding to a target user is firstly acquired, then at least one sub-audio matched with the electrocardiogram is acquired according to the electrocardiogram, and then a commemorative audio file corresponding to the electrocardiogram is synthesized according to the sub-audio. Therefore, the electrocardiogram is converted into the commemorative audio file matched with the actual heartbeat sound at the electrocardiogram collection moment. Therefore, the expression ability of the information of the electronic souvenir is effectively improved, and the user experience is improved.

The invention discloses a multi-format document intelligent audio content generation system and method, and relates to the technical field of artificial intelligence and audio generation. Comprising the steps of 1, analyzing various formats of documents uploaded by a user to generate a structured plain text, 2, performing semantic analysis according to the content of the structured plain text, identifying different emotional colors and mood characteristics in the plain text, converting the plain text into a multi-role dialogue script according to the emotional colors and the mood characteristics, and storing the multi-role dialogue script in a database. 3, calling a text-to-audio model, and generating an audio clip of a corresponding role for each line segment of the dialogue script, and 4, splicing and synthesizing the audio clips in the audio stream list into a complete audio file according to a line sequence; automatic conversion of multi-format document content and multi-role audio synthesis are realized, dependence on cloud service is not needed, data privacy is guaranteed, and processing efficiency is improved.

This invention proposes a method, system, and computer device for generating text-based videos, relating to the technical field of video processing. The method includes: acquiring user input parameters; generating anchor point prompts and anchor point character diagrams based on the user input parameters; evaluating the anchor point character diagrams until they pass evaluation; extracting information from the anchor point prompts to obtain a storyboard; evaluating the storyboards until they pass evaluation; generating audio and storyboard video based on the storyboards; evaluating the storyboard video until it passes evaluation; post-processing the storyboard video to obtain a post-processed video; and combining the post-processed video with the audio to generate a target video. This invention effectively provides a closed-loop evaluation node for text-based video generation, improving the quality of generated text-based videos.

Embodiments of the present disclosure provide a vocoder training method. The vocoder training method comprises: obtaining a first fundamental frequency sequence of audio in an audio corpus; performing fundamental frequency perturbation processing on the first fundamental frequency sequence to obtain a second fundamental frequency sequence; performing mapping processing on the second fundamental frequency sequence to obtain a target tensor; and training the target tensor, an acoustic feature sequence corresponding to the audio, and the audio to obtain a vocoder used for audio synthesis. The method of the present disclosure improves the robustness of the vocoder to fundamental frequency prediction errors in actual application by introducing fundamental frequency perturbation, thereby significantly improving the accuracy and quality of audio synthesis and providing a better experience for users. In addition, embodiments of the present disclosure provide an audio synthesis method, a medium, a device and a computing device.

The present application provides a video processing method, device, electronic device and computer-readable storage medium; the method comprises: obtaining a video file to be edited and at least one special effect file for adding special effects to the video file to be edited; performing tree-like editing processing based on the video file to be edited and at least one special effect file to obtain a tree-like rendering structure including the video file and the special effect file; performing rendering processing on the tree-like rendering structure based on the video layer included in the video file to be edited and at least one special effect file to obtain a rendered video file; performing audio synthesis processing based on the rendered video file to obtain a target video file with special effects added. Through the present application, the flexibility and efficiency of adding special effects can be improved.

The application discloses a kind of wisdom management system and method based on video privacy information identification, it is related to information processing identification technical field, including: time stamp matching test is carried out to audio of different length, matching test model is established, audio in court video is extracted, audio is divided and optimization division result is obtained, each segment audio after division is converted into text separately, private content in text is identified, the location of private content in audio is found by time stamp matching and is carried out mute, complete audio is synthesized after mute processing, and the division process of subsequent audio to be muted is planned, after frame extraction is carried out to court video image, private information in each frame image is identified and is carried out coding, audio-video is synthesized after coding video and mute processing audio, the probability that mute is not complete even part of private information in audio is not muted due to that time stamp matching deviation is larger is reduced, the reliability of court audio private information processing is guaranteed.

The utility model discloses a sound scene enhancing device based on environmental perception, and relates to the technical field of acoustics. The system comprises a microphone array, a signal conditioning circuit, an analog-to-digital converter, a DSP processor, a sound scene database memory, a master control MCU, an FPGA beam controller, an audio synthesis chip, a power amplifier, a loudspeaker array and a noise classification coprocessor, the microphone array is electrically connected with the signal conditioning circuit, and the signal conditioning circuit is electrically connected with the analog-to-digital converter. The analog-to-digital converter is electrically connected with the DSP processor, the analog-to-digital converter is electrically connected with the master control MCU, the DSP processor is electrically connected with the sound scene database memory, the DSP processor is electrically connected with the noise classification coprocessor, the noise classification coprocessor is electrically connected with the master control MCU, the master control MCU is electrically connected with the sound scene database memory, and the master control MCU is electrically connected with the FPGA beam controller. The sound scene can be automatically adjusted according to environment changes.

The invention relates to a target video explanation method, system and terminal based on video and audio synthesis, and belongs to the technical field of video and audio processing, and the target video explanation method comprises the steps: obtaining a target video; dividing the target video into a plurality of adjustment sub-videos according to the target duration of the target video; decomposing each adjusted sub-video into a role layer, a scene layer and an interaction layer; matching a first audio according to the role layer, matching a second audio according to the scene layer, and matching a third audio according to the interaction layer; fusing the first audio, the second audio and the third audio to generate a sub-audio; associating the adjusted sub-videos with the corresponding sub-audios; and when the target video is played to a certain adjusted sub-video range, performing explanation according to the corresponding sub-audio. The method has the beneficial effect of improving the watching experience of the user.