Teleconference human voice recognition and filtering method and system based on large audio model
Through the deep learning technology of audio big model, the problem of distinguishing between vocals and background noise in remote meetings is solved, high-precision recognition and filtering is achieved, the error recognition rate is reduced, data security and system stability are ensured, and user experience is improved.
Patent Information
- Application Number
- CN202411383827.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art is difficult to accurately distinguish between vocals and background noise in remote meetings, resulting in the impact of communication quality, and there are problems such as delay in real-time processing, large resource consumption, poor adaptability and poor user experience.
Deep learning method based on audio big model is adopted, by pre-entering user sound samples, extracting acoustic features, training audio big models, identifying and filtering non-user noise in real time, providing encrypted storage to ensure data security, and iteratively optimize the model through user feedback.
It improves audio recognition accuracy, reduces misidentification, has adaptive capabilities, ensures the security of user sound data, and optimizes audio acquisition and model updates through integrated interfaces, improving user experience and system stability.
Smart Images

Figure CN119091897A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning of large audio models, and in particular to a method and system for remote conference human voice recognition filtering based on large audio models. Background Art
[0002] In modern teleconferencing systems, background noise often seriously affects the quality of communication. These noises include but are not limited to keyboard tapping, people talking around, air conditioning, etc. Although existing audio processing technologies can suppress these background noises to a certain extent, they often cannot distinguish between human voices and noise well, which affects the quality of effective communication. Therefore, there is an urgent need for an efficient system that can accurately identify and filter noise from non-conference participants.
[0003] 1. Sound enhancement technology
[0004] In the prior art, sound enhancement is usually used to improve the voice quality in conferences by reducing background noise, echo and other interference. These technologies include using digital signal processing (DSP) algorithms to increase the ratio of speech signals to noise. Common methods include spectral subtraction, Wiener filtering and minimum mean square error (MMSE) estimation.
[0005] 2. Beamforming Technology
[0006] Beamforming is a technology that uses multiple microphones to collect sound, and focuses on the sound source in a specific direction by adjusting the relative strength and phase of the signals from each microphone, thereby enhancing the desired signal while suppressing noise from other directions. This is particularly effective in multi-participant remote meetings, and can significantly improve the sound quality of specific participants in the meeting.
[0007] 3. Machine Learning and Deep Learning Models
[0008] In recent years, deep learning has made significant progress in sound recognition and separation. For example, models such as convolutional neural networks (CNN), recurrent neural networks (RNN), and long short-term memory networks (LSTM) are used to recognize and separate speech signals. These models can learn complex sound features from large amounts of audio data, thereby achieving more accurate sound recognition and filtering of background noise.
[0009] 4. Echo cancellation technology
[0010] Echo cancellation technology is used to eliminate echoes generated during calls, which is a common problem in remote conferences. By using adaptive filters and echo suppression algorithms, existing technologies can effectively reduce or eliminate echoes caused by audio feedback between microphones and speakers.
[0011] The current technical solution has the following defects:
[0012] 1. Insufficient differentiation: Existing technologies often have difficulty accurately distinguishing between human voices and background noise, especially when the noise types are complex and varied, such as other people’s conversations and the voices of meeting participants.
[0013] 2. Real-time processing delay: Complex sound processing technology may introduce considerable processing delay, which may cause communication interruption or incoherence in remote meetings that require real-time response.
[0014] 3. High resource consumption: Advanced sound processing techniques (especially deep learning-based models) require a lot of computing resources, which may limit their application on resource-constrained devices.
[0015] 4. Adaptability problem: Many existing systems perform well in specific environments, but may not be able to maintain the same performance level when faced with environmental changes and need to be readjusted or trained.
[0016] 5. Poor user experience: Sound processing may cause unnatural or distorted sound quality, affecting user experience. Summary of the invention
[0017] The purpose of the present invention is to provide a remote conference voice recognition filtering method and system based on a large audio model. By using a large audio model for deep learning, the voice characteristics of registered users can be learned and recognized more accurately, providing higher recognition accuracy for audio acquisition and transmission, and reducing misrecognition.
[0018] The purpose of the present invention can be achieved by the following technical solutions:
[0019] A remote conference voice recognition and filtering method based on a large audio model, the method comprising:
[0020] Pre-record user voice samples and process the voice samples;
[0021] Extracting acoustic features from the processed sound data and corresponding them with the pre-recorded sound samples to form a sound data set;
[0022] Use sound datasets to train large audio models;
[0023] In real-time conferences, the audio big model is used to receive and process the user's real-time voice;
[0024] Output the processed sound to the remote conference to make the audio communication clear;
[0025] Collect feedback information and iteratively optimize the sound recognition model.
[0026] Furthermore, the sound samples are stored through an encrypted storage protocol to ensure the security of the user's sound data.
[0027] Furthermore, the processing of the sound samples includes denoising, enhancement and data normalization.
[0028] Furthermore, the extracted acoustic features include pitch, volume, frequency, rhythm and tone of the user's voice.
[0029] Furthermore, the processing of the user's real-time voice by the audio big model includes:
[0030] Use a large audio model to recognize input sounds in real time, extract and enhance user voices based on their acoustic features;
[0031] Filter out noise other than the user's voice.
[0032] Furthermore, each user has an independent sound data set and audio big model. In a remote conference, the audio big model of the corresponding user is triggered to perform real-time sound processing based on the acoustic characteristics of the real-time sound.
[0033] Furthermore, the feedback information includes user feedback on sound quality and audio model performance, updates of sound samples, and user sound preference settings.
[0034] A computer-readable storage medium includes one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the remote conference human voice recognition filtering method based on a large audio model as described above.
[0035] A remote conference voice recognition and filtering system based on a large audio model, the system comprising:
[0036] Sound recording module, users register and record clear sound samples;
[0037] The sound processing module processes the sound samples and extracts acoustic features, which are then combined with the pre-recorded sound samples to form a sound data set;
[0038] Model training module, using sound dataset to train large audio models;
[0039] The voice recognition module uses the audio model to receive and process the user's real-time voice in a real-time conference, and outputs the processed voice to the remote conference module;
[0040] Remote conference module, which supports remote conferences, receives processed audio, and provides real-time communication to users;
[0041] Model update module, collects feedback information and iteratively optimizes the sound recognition model;
[0042] The user interface module provides an integrated interface for integrated operations.
[0043] Furthermore, the integrated operation includes pre-recording clear voice samples, updating voice samples, setting voice preferences and submitting user feedback.
[0044] Compared with the prior art, the present invention has the following beneficial effects:
[0045] 1. The present invention records user voice samples in advance and uses a large audio model for deep learning, which can more accurately learn and identify the voice features of registered users, provide higher recognition accuracy for audio collection and transmission, and reduce misidentification. At the same time, the large audio model has the ability of continuous learning, and can automatically adjust and optimize the sound processing algorithm according to environmental changes and user feedback, and has adaptive ability and long-term stability;
[0046] 2. The present invention takes user privacy into consideration when storing sound samples, and adopts encrypted data processing and storage protocols to ensure the security of user sound data, thus providing security for users;
[0047] 3. The present invention provides an integrated page for users to update sound samples, set preferences and submit feedback, making audio acquisition and model optimization more efficient and convenient. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 is a flow chart of the present invention;
[0049] Figure 2 It is a system structure diagram of the present invention. DETAILED DESCRIPTION
[0050] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0051] Example 1
[0052] This embodiment discloses a remote conference voice recognition and filtering method based on a large audio model. Figure 1 As shown, the method includes:
[0053] S1, recording a clear voice sample of the user and processing the voice sample;
[0054] S2, extracting acoustic features from the processed sound data and corresponding to the pre-recorded sound samples to form a sound data set;
[0055] S3, using sound datasets to train large audio models;
[0056] S4, in a real-time conference, uses the audio big model to receive and process the user's real-time voice;
[0057] S5, outputs the processed sound to the remote conference, making the audio communication clear;
[0058] S6, collect feedback information and iteratively optimize the sound recognition model.
[0059] The specific implementation process of the method is as follows Figure 2 As shown, first, the user registers an account through the user interface and submits a sound sample, which is the basic data for training the model.
[0060] The sound samples are stored through an encrypted storage protocol to ensure the security of the user's sound data.
[0061] Secondly, the audio acquisition system is responsible for recording the user's voice and transmitting it to the sound processing system.
[0062] Next, the sound processing system processes the audio and sends it to the feature extraction module to extract key acoustic features, which are then compared with the pre-recorded sound samples to form a user-specific sound dataset. That is, the unprocessed sound samples with noise and their corresponding processed and extracted sound features are compared one by one to form a sound dataset.
[0063] The processing of the sound samples includes denoising, enhancement and data normalization.
[0064] The extracted acoustic features include the pitch, volume, frequency, rhythm and tone of the user's voice, and the user's vocal habits and voiceprints are recorded one by one.
[0065] After the sound data set is prepared, the audio model is trained in the model training system. The audio model used in this embodiment is the Wavenet model. During training, the sound data set is used as the training data set. The training is performed using noisy unprocessed sound data and clear sound data. The training data is noisy sound data, and the training target data is clear sound data. The training is repeated until the model converges.
[0066] After the model training is completed, the trained model is deployed to the recognition engine.
[0067] Subsequently, the user conducts a real-time conference, and the audio acquisition module receives the real-time sound emitted by the user and sends it to the recognition engine. The recognition engine uses the deployed audio large model to identify and filter non-user sounds, leaving only the user's voice.
[0068] The processing of the user's real-time sound by the audio big model includes:
[0069] Use a large audio model to recognize input sounds in real time, extract and enhance user voices based on their acoustic features;
[0070] Filter out noise other than the user's voice.
[0071] Each user has an independent sound data set and audio big model. In a remote conference, the corresponding user's audio big model is triggered to perform real-time sound processing based on the acoustic characteristics of the real-time sound.
[0072] Finally, the filtered sound is transmitted to the remote conference system through the output system for playback, making the audio communication clear and unobstructed.
[0073] In addition, after the user ends the meeting, the user can also provide feedback to the system on the usage experience and improvement suggestions, including user feedback on sound quality and audio large model performance, update sound samples, and set sound preferences.
[0074] After receiving feedback information, the large audio model will be optimized and iteratively updated, and redeployed to the recognition engine after the update is completed so that the user can use it next time. Compared with models that require manual updates or cannot adjust themselves, the adaptive model can better cope with changing environments and maintain long-term performance stability.
[0075] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0076] Example 2
[0077] This embodiment discloses a remote conference voice recognition and filtering system based on a large audio model, the system comprising:
[0078] Sound recording module, users register and record clear sound samples;
[0079] The sound processing module processes the sound samples and extracts acoustic features;
[0080] Model training module, using sound dataset to train large audio models;
[0081] The voice recognition module uses the audio model to receive and process the user's real-time voice in a real-time conference, and outputs the processed voice to the remote conference module;
[0082] The remote conference module supports remote conferences, receives processed audio, and sends it to the remote conference system to provide clear calls;
[0083] Model update module, collects feedback information and iteratively optimizes the sound recognition model;
[0084] The user interface module provides an integrated interface for integrated operations.
[0085] The integrated operation includes: pre-recording clear voice samples, updating voice samples, setting voice preferences and submitting user feedback.
[0086] When using the remote conference voice recognition and filtering system based on the large audio model, the user first records clear sound samples through the integrated page. After the large audio model training is completed, the user uses the system through the integrated page to assist in real-time conferences and maintain clear calls.
[0087] If after use, the user needs to provide feedback to the system on the usage experience and suggestions for improvement, such as feedback on sound quality and audio large model performance, this can also be done through the integrated page.
[0088] If the user needs to update the sound sample and set the sound preference, he / she can record and set it through the integrated page, and send the updated sound sample to the remote conference voice recognition filtering system based on the audio big model, repeat the model training process, and update the audio big model.
[0089] The specific details of the above other modules can be understood by referring to the relevant descriptions and effects in Example 1.
[0090] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A remote conference voice recognition and filtering method based on a large audio model, characterized in that: The method comprises: Pre-record user voice samples and process the voice samples; Extracting acoustic features from the processed sound data and corresponding them with the pre-recorded sound samples to form a sound data set; Use sound datasets to train large audio models; In real-time conferences, the audio big model is used to receive and process the user's real-time voice; Output the processed sound to the remote conference; Collect feedback information and iteratively optimize the sound recognition model.
2. According to the method of claim 1, the method is characterized in that: The sound samples are stored via an encrypted storage protocol.
3. According to the method of claim 1, the method is characterized in that: The processing of the sound samples includes denoising, enhancement and data normalization.
4. According to the method of claim 1, the method is characterized in that: The extracted acoustic features include pitch, volume, frequency, rhythm and tone of the user's voice.
5. According to the method of claim 1, the method is characterized in that: The processing of the user's real-time sound by the audio big model includes: Use a large audio model to recognize input sounds in real time, extract and enhance user voices based on their acoustic features; Filter out noise other than the user's voice.
6. According to the method of claim 1, the method is characterized in that: Each user has an independent sound data set and audio big model. In a remote conference, the audio big model of the corresponding user is triggered to perform real-time sound processing based on the acoustic characteristics of the real-time sound.
7. According to claim 1, a remote conference voice recognition and filtering method based on a large audio model is characterized in that: The feedback information includes user feedback on sound quality and audio model performance, updates of sound samples, and user sound preference settings.
8. A computer-readable storage medium, characterized in that: It includes one or more programs for execution by one or more processors of an electronic device, and the one or more programs include instructions for executing the remote conference human voice recognition filtering method based on the audio large model as described in any one of claims 1-7.
9. A remote conference voice recognition and filtering system based on a large audio model, characterized in that: The system comprises: Sound recording module, users register and record clear sound samples; The sound processing module processes the sound samples and extracts acoustic features, which are then combined with the pre-recorded sound samples to form a sound data set; Model training module, using sound dataset to train large audio models; The voice recognition module uses the audio model to receive and process the user's real-time voice in a real-time conference, and outputs the processed voice to the remote conference module; Remote conference module, which supports remote conferences, receives processed audio, and provides real-time communication to users; Model update module, collects feedback information and iteratively optimizes the sound recognition model; The user interface module provides an integrated interface for integrated operations.
10. A remote conference voice recognition and filtering system based on a large audio model according to claim 9, characterized in that: The integrated operation includes pre-recording clear voice samples, updating voice samples, setting voice preferences and submitting user feedback.
