Method and apparatus for human voice sound effect matching, and audio-video device

Abstract

The application relates to the technical field of voice analysis, and discloses a method and device for human voice sound effect matching and a video-audio equipment, which comprises the following steps: collecting a human voice audio signal generated by a user in a accompaniment singing scene; performing acoustic feature extraction on the human voice audio signal to obtain a feature vector for representing sound characteristics of the user; inputting the feature vector into a pre-constructed sound effect matching model to output a sound effect configuration result matched with the sound characteristics of the user; the sound effect configuration result at least comprises a target sound effect type and a sound effect processing parameter corresponding to the target sound effect type; and performing sound effect processing on the human voice audio signal according to the sound effect configuration result to obtain a processed human voice audio signal. The application can improve the automation degree of sound effect configuration, make the sound effect processing result more suitable for the sound characteristics of different users, reduce the complexity of manual adjustment of the user, and improve the consistency and stability of human voice audio processing.

Description

Technical Field

[0001] This application relates to the field of speech analysis technology, such as a method and apparatus for matching human voice effects, and audio-visual equipment. Background Technology

[0002] Currently, accompaniment singing applications are becoming increasingly popular on smart TVs, mobile devices, and online audio and video platforms. When singing with accompaniment, users typically need to apply sound effects parameters such as reverb, equalization, and dynamic processing to the vocal audio to improve the singing effect and overall listening experience. However, different users have significant differences in timbre, vocal range, formant characteristics, and singing techniques, making it difficult for the same set of sound effect parameters to be suitable for all users simultaneously.

[0003] To address these needs, related technologies typically employ preset sound effect templates or fixed sound effect parameters to process human vocal audio. For example, the system provides several preset sound effect modes for users to manually select, or adjusts pitch or volume based on simple rules. Furthermore, some technologies perform acoustic analysis on user singing audio to provide suggestions for improvement or song recommendations, but the configuration of sound effect parameters still largely relies on human experience or fixed templates.

[0004] In the process of implementing the embodiments of this disclosure, it was found that at least the following problems exist in the related art: While related technologies can perform sound effects processing or sound analysis on human voice audio to a certain extent, they usually only recommend sound effects based on song style, completely ignoring the differences in individual singer's voice characteristics, resulting in low sound effect matching and failing to maximize the user's singing performance and listening satisfaction.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0007] This disclosure provides a method, apparatus, and audio-visual equipment for matching human voice effects, in order to improve the matching degree of human voice effects in accompaniment singing scenarios.

[0008] In some embodiments, the method for matching human voice effects includes: acquiring human voice audio signals generated by a user in a background music singing scenario; extracting acoustic features from the human voice audio signals to obtain feature vectors characterizing the user's voice characteristics; inputting the feature vectors into a pre-constructed sound effect matching model and outputting sound effect configuration results that match the user's voice characteristics; the sound effect configuration results include at least a target sound effect type and sound effect processing parameters corresponding to the target sound effect type; and performing sound effect processing on the human voice audio signals according to the sound effect configuration results to obtain processed human voice audio signals.

[0009] In some embodiments, the apparatus for human voice effect matching includes a processor and a memory storing program instructions, the processor being configured to execute the method for human voice effect matching as described above when the program instructions are executed.

[0010] In some embodiments, the audio-visual device includes a device body; and means for matching human voice effects, mounted on the device body.

[0011] The method, apparatus, and audio-visual equipment for human voice effect matching provided in this disclosure can achieve the following technical effects: By collecting the user's vocal audio signal in a accompaniment singing scenario and extracting acoustic features from the vocal audio signal to form a feature vector characterizing the user's voice characteristics, this feature vector is then input into a sound effect matching model to output a sound effect type and sound effect processing parameters that match the user's voice characteristics. Sound effect processing is then performed on the vocal audio signal based on the sound effect configuration results, thereby achieving automatic matching and application of sound effect parameters based on the user's voice characteristics. Compared to methods relying on fixed templates or manual adjustments, this application can improve the automation level of sound effect configuration, making the sound effect processing results more closely match the voice characteristics of different users, reducing the complexity of manual adjustments by users, and improving the consistency and stability of vocal audio processing.

[0012] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0013] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a flowchart illustrating a method for matching human voice effects provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating another method for matching human voice effects provided in an embodiment of this disclosure; Figure 3 This is a flowchart illustrating another method for matching human voice effects provided in an embodiment of this disclosure; Figure 4 This is a flowchart illustrating the method for constructing the sound effect matching model in this embodiment of the present disclosure; Figure 5 This is a flowchart illustrating another method for matching human voice effects provided in an embodiment of this disclosure; Figure 6 This is a flowchart illustrating another method for matching human voice effects provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of an apparatus for matching human voice effects provided in an embodiment of this disclosure. Detailed Implementation

[0014] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0015] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0016] Unless otherwise stated, the term "multiple" means two or more.

[0017] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0018] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0019] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0020] In the field of vocal accompaniment application technology, those skilled in the art, when solving the problem of matching and adjusting vocal effects, typically tend to rely on preset sound effect templates or human experience to configure sound effect parameters. For example, in vocal applications on television or mobile terminals, the system usually presets several reverb modes, equalization modes, or comprehensive sound effect schemes. Users manually select different sound effect modes or adjust parameters such as reverb time, equalization gain, and compression intensity using sliders to obtain the desired vocal effect. Some related technologies also perform simple pitch or volume analysis on the user's vocal audio, but the final determination of sound effect parameters still mainly relies on fixed rules or human experience. Those skilled in the art generally believe that matching vocal effects is highly dependent on subjective listening and human adjustment experience. Only through repeated listening and manual fine-tuning can a relatively suitable sound effect be obtained. Therefore, for a long time, related technical solutions and engineering practices have mostly revolved around preset templates and manual interactive adjustment processes.

[0021] However, the inventors discovered limitations in the aforementioned technological understanding during their research. While preset sound effect templates or manual adjustments can improve vocal quality to some extent, significant differences exist among users in timbre characteristics, vocal range, formant distribution, and singing techniques. Fixed templates or empirical rules cannot accurately reflect a user's true vocal characteristics. Even with repeated manual adjustments, it often relies on the user's subjective judgment and tuning experience, resulting in a time-consuming process and difficulty in ensuring consistent results across different devices or usage scenarios. Furthermore, existing technologies typically lack mechanisms to feed back actual user behavior to the model or parameter configuration process, making it difficult to continuously optimize sound effect matching results as user preferences and habits change.

[0022] Based on the above understanding, the inventors have broken through the technical inertia of existing technologies that rely on preset templates or manual adjustments for sound effect configuration, and proposed a technical approach to achieve automatic sound effect matching based on the user's multi-dimensional acoustic characteristics. The inventors recognize that the human voice audio signal generated by a user during accompaniment singing contains rich acoustic information. By extracting and structurally representing pitch features, spectral features, formant features, dynamic features, and singing technique features, a feature vector characterizing the user's voice characteristics can be formed. If a mapping relationship can be established between the feature vector and the sound effect type and sound effect processing parameters, and sound effect configuration results can be automatically generated based on this mapping relationship, personalized sound effect matching for different users' voice characteristics can be achieved without repeated manual adjustments.

[0023] Based on the aforementioned technical concepts, this application constructs an automatic human voice effect matching mechanism integrating acoustic feature extraction, model matching, and sound effect processing. Specifically, it collects human voice audio signals generated by users in accompaniment singing scenarios and extracts multi-category acoustic features from these signals to generate feature vectors characterizing the user's voice characteristics. Subsequently, the feature vectors are input into a pre-constructed sound effect matching model, which outputs sound effect types and processing parameters that match the user's voice characteristics. Finally, sound effect processing is performed on the human voice audio signals according to the sound effect configuration results, making the processed audio effect more closely match the user's voice characteristics. Through experiments and practical applications, the inventors have found that this method can significantly reduce reliance on manual adjustments, improve the automation level of sound effect matching, and maintain good consistency in performance across different users and device environments.

[0024] It should be noted that in actual accompaniment singing, the vocal effect is affected not only by the sound effect parameter settings, but also by a combination of factors such as differences in user acoustic characteristics, changes in singing state, and user behavior preferences. A single fixed template or static parameter configuration is difficult to adapt to these complex changes. This application establishes a parameter mapping relationship based on user acoustic characteristics and continuously optimizes it in conjunction with user behavior. This allows the sound effect matching process to be dynamically adjusted based on the user's real vocal characteristics and actual usage results, thereby more realistically reflecting and improving the vocal effect experience in accompaniment singing scenarios, rather than relying solely on a fixed template or the result of a single manual adjustment.

[0025] Figure 1 This is a flowchart illustrating a method for matching human voice effects provided in an embodiment of this disclosure.

[0026] like Figure 1 As shown, the method includes: Step S101: Collect the human voice audio signal generated by the user in the accompaniment singing scene.

[0027] Accompaniment singing scenario refers to the application scenario in which users sing with their voices accompanied by preset accompaniment audio, including but not limited to smart TV singing applications, mobile terminal singing applications, or other scenarios where voices and accompaniment are mixed and output.

[0028] Human voice audio signal refers to the digital representation of the sound signal generated during a user's singing. It can be pure human voice audio or human voice audio signal separated from a mixed audio that includes accompaniment.

[0029] Here, in the context of accompanying singing, the system acquires the sound signals generated during the user's singing through an audio acquisition device and converts the sound signals into digital audio signals. This allows the acquisition of audio data that reflects the user's true singing state, providing a foundation for subsequent acoustic feature analysis.

[0030] Step S102: Perform acoustic feature extraction on the human voice audio signal to obtain a feature vector that characterizes the user's voice characteristics.

[0031] Acoustic features refer to the quantitative parameters extracted from human voice audio signals to describe sound attributes, including pitch features, spectral features, formant features, dynamic features, and singing technique features.

[0032] A feature vector is a structured numerical representation composed of multiple acoustic features, used to uniformly characterize the user's voice characteristics and serve as input for subsequent models.

[0033] Here, the human voice audio signal can be segmented into frames, and multiple categories of acoustic features can be extracted in the time or frequency domain. Subsequently, the acoustic features are encoded, normalized, or concatenated to generate feature vectors that uniformly represent the user's voice characteristics. In this way, the raw audio signal is converted into a structured numerical representation, enabling the user's voice characteristics to be expressed in a computable and modelable manner, providing input data for subsequent model matching.

[0034] Step S103: Input the feature vector into the pre-built sound effect matching model and output the sound effect configuration result that matches the user's voice characteristics; the sound effect configuration result includes at least the target sound effect type and the sound effect processing parameters corresponding to the target sound effect type.

[0035] A sound effect matching model is a computational model used to establish a mapping relationship between a user's voice feature vector and sound effect configuration results. It can be a neural network model or other machine learning model.

[0036] The sound effect configuration result refers to the sound effect recommendation result output by the model.

[0037] Here, by performing model inference on the feature vectors, the type of sound effect that matches the user's voice characteristics and the corresponding sound effect processing parameters, such as reverb parameters, equalization parameters, or dynamic processing parameters, are obtained. This enables automatic sound effect matching based on the user's voice characteristics, avoiding reliance on manual selection or fixed templates for sound effect configuration, thereby improving the automation and personalization level of sound effect matching.

[0038] Step S104: Perform sound effect processing on the human voice audio signal according to the sound effect configuration result to obtain the processed human voice audio signal.

[0039] Audio processing refers to the process of digitally processing human voice audio signals based on audio processing parameters, including but not limited to reverb processing, equalization processing, and dynamic compression processing.

[0040] Here, the combination of sound effect processing modules can be determined according to the target sound effect type, and the parameters of each sound effect processing module can be set according to the sound effect processing parameters. Then, the human voice audio signal is digitally processed according to the preset processing order, and the sound effect configuration results predicted by the model are actually applied to the audio processing process, so that the processed human voice audio signal is more in line with the user's voice characteristics, thereby improving the human voice effect experience in the accompaniment singing scene.

[0041] Thus, through the coordinated execution of steps S101 to S104, the user's vocal audio signal generated in the accompaniment singing scenario is collected, and acoustic features are extracted from the vocal audio signal to form a feature vector representing the user's voice characteristics. This feature vector is then input into a sound effect matching model to output a sound effect type and sound effect processing parameters that match the user's voice characteristics. Based on the sound effect configuration results, sound effect processing is performed on the vocal audio signal, thereby achieving automatic matching and application of sound effect parameters based on the user's voice characteristics. Compared to methods relying on fixed templates or manual adjustments, this method improves the automation level of sound effect configuration, makes the sound effect processing results more closely match the voice characteristics of different users, reduces the complexity of manual adjustments by users, and enhances the consistency and stability of vocal audio processing.

[0042] The following describes how to collect human voice audio signals using specific examples.

[0043] Figure 2 This is a flowchart illustrating another method for matching human voice effects provided in this disclosure.

[0044] like Figure 2 As shown, the method includes: Step S201: Collect the sound signal generated by the user's singing; the sound signal includes the human voice and the accompaniment.

[0045] The sound signal generated by a user's singing refers to the acoustic signal produced when a user sings while the accompaniment is playing. This signal is usually collected by a microphone and converted into a digital audio signal.

[0046] Here, the acquired sound signal can be a mixed audio signal, which includes both the vocal part produced by the user singing and the sound component formed by the accompaniment audio played through the speaker. This mixed signal can be represented as vocals and accompaniment sounds superimposed on the same audio stream.

[0047] The system can acquire sound wave signals from the user's singing environment using an audio acquisition device; perform analog-to-digital conversion on the analog sound wave signals to generate digital audio signals; the digital audio signals are a mixture of vocals and accompaniment. In practical applications, the accompaniment may originate from the same device's speakers or from an external audio source.

[0048] Step S202: Perform human voice separation processing on the sound signal to obtain the human voice audio signal.

[0049] Vocal separation processing refers to the process of extracting the vocal component from a mixed audio signal containing both vocals and accompaniment, ensuring that the output signal primarily contains the user's vocals while minimizing accompaniment interference. The resulting audio signal, mainly containing the user's vocal information, is used for subsequent acoustic feature extraction and sound effect matching.

[0050] Here, voice separation can be achieved in various ways, such as: filtering separation methods based on spectrum analysis; separation methods based on time-frequency masks; voice separation models based on neural networks; and differential processing using accompaniment reference signals. The specific implementation method can be selected according to the system performance and real-time requirements.

[0051] In accompaniment singing scenarios, directly extracting features from the mixed audio may cause the spectral energy of the accompaniment sound to affect the calculation results of pitch, spectrum, or dynamic features, thereby reducing the accuracy of sound effect matching. By adding the human voice separation processing step S202, the interference of the accompaniment can be reduced at the source, allowing subsequent modeling to focus more on the user's own voice characteristics.

[0052] Step S203: Extract acoustic features from the human voice audio signal to obtain a feature vector that characterizes the user's voice characteristics.

[0053] Step S204: Input the feature vector into the pre-built sound effect matching model, and output the sound effect configuration result that matches the user's voice characteristics. The sound effect configuration result includes at least the target sound effect type and the sound effect processing parameters corresponding to the target sound effect type.

[0054] Step S205: Perform sound effect processing on the human voice audio signal according to the sound effect configuration result to obtain the processed human voice audio signal.

[0055] In one embodiment of this application, the acquisition process of human voice audio signals can be performed as follows.

[0056] In a singing scenario with accompaniment, the terminal device (such as a smart TV, set-top box, mobile terminal, or dedicated singing equipment) plays the accompaniment audio, while simultaneously capturing the sound signal generated by the user's singing through an audio acquisition device. The audio acquisition device can be an external microphone, a built-in microphone array, or other acoustic sensors capable of capturing ambient sound.

[0057] Specifically, when a user begins to sing, the audio acquisition device converts the ambient sound wave signals into analog electrical signals, and then converts the analog electrical signals into digital audio signals through an analog-to-digital converter. The digital audio signals can be sampled and quantized according to a preset sampling rate, such as 44.1kHz or 48kHz, and a quantization bit depth, thereby obtaining the original digital audio data stream.

[0058] In actual accompaniment singing scenarios, the collected audio signal typically contains a mixture of the user's voice and the accompaniment played by the device's speakers. Therefore, in this embodiment, the mixed audio signal can be further processed by voice enhancement or voice separation.

[0059] After obtaining the human voice audio signal with human voice as the main component, the human voice audio signal can be preprocessed, such as noise reduction, automatic gain control (AGC), silence detection (VAD), and amplitude normalization, to improve the stability of subsequent acoustic feature extraction.

[0060] In addition, in an alternative implementation, the system can guide the user to perform a short trial singing sample before the user's formal performance. By analyzing the volume range and signal-to-noise ratio of the trial singing segment, the system can adaptively adjust the acquisition gain to ensure that the subsequently acquired vocal audio signal has a suitable dynamic range.

[0061] Through the above acquisition and preprocessing process, a stable human voice audio signal with a high signal-to-noise ratio can be obtained, providing reliable data input for subsequent acoustic feature extraction and sound effect matching model inference.

[0062] The following section explains how to extract acoustic features and obtain feature vectors using specific examples.

[0063] Figure 3 This is a flowchart illustrating another method for matching human voice effects provided in this disclosure.

[0064] like Figure 3 As shown, the method includes: Step S301: Collect the human voice audio signal generated by the user in the accompaniment singing scene.

[0065] Step S302: Extract acoustic features of multiple categories from the human voice audio signal.

[0066] Acoustic features refer to the different dimensions of sound attribute features extracted from human voice audio signals. Each category is used to describe a certain aspect of the sound's characteristics.

[0067] Human voice audio signals can be segmented into frames for analysis in both the time and frequency domains. Different algorithms can then be used to extract various acoustic features. For example, fundamental frequency detection algorithms are used to extract pitch features, STFT or FFT to extract spectral features, LPC analysis to estimate formants, and energy calculation to extract dynamic features. These features are then output numerically.

[0068] Optionally, acoustic features include at least two of the following categories: pitch-related features, spectral-related features, formant-related features, dynamic-related features, and singing technique-related features.

[0069] Among them, pitch-related features are mainly used to characterize: the fundamental frequency, range, pitch stability, and pitch variation trend (e.g., glissando, staccato, etc.). Essentially, they reflect the frequency of vocal cord vibration and its variation over time. In this scheme, they are used to determine whether a user's voice is high-pitched, mid-pitched, or low-pitched, as well as to identify pitch control ability, singing style, and to provide a basis for matching parameters such as reverberation length and equalization high-frequency gain.

[0070] Spectral characteristics are used to characterize the energy distribution, timbre brightness or thickness, high-frequency to low-frequency ratio, and harmonic structure of sound at different frequencies. Typical characteristics include the Mel spectrum, MFCC, spectral centroid, and roll-off frequency. Essentially, they reflect the timbre characteristics of sound; in this scheme, they are used to determine whether a sound is too bright, too dark, too thick, or too thin, providing a basis for equalization (EQ) parameter adjustment, assisting in identifying the user's vocal type, and influencing the selection of sound effect types.

[0071] Formant correlation features are used to characterize duct morphology, vocal organ resonance structure, and individual timbre differences. Typical features include the first formant, second formant, and formant bandwidth. Essentially, they reflect the filtering characteristics of the vocal tract. In this solution, they are used to differentiate between users' vocal structures, identify sound penetration or nasal characteristics, influence mid-frequency equalization settings, and improve personalized matching capabilities.

[0072] Dynamic correlation features are used to characterize the amplitude of volume changes, energy fluctuations during performance, contrast in sound intensity, and dynamic control capabilities. Typical features include volume envelope, RMS energy, and dynamic range. Essentially, they reflect the characteristics of sound intensity changing over time. In this scheme, they are used to provide a basis for compressor parameters, determine whether dynamic enhancement is needed, identify vocal expressiveness, and control sound smoothness.

[0073] Singing technique-related characteristics are used to characterize vibrato frequency and amplitude, glissando features, pauses and legato patterns, and emotional expression. Essentially, they reflect the singing style and the application of techniques.

[0074] In other words, the five categories of features mentioned above comprehensively model the user's voice from multiple dimensions, including vocal frequency, timbre distribution, vocal tract resonance, dynamic control, and singing style. By integrating these multiple categories of acoustic features, the completeness of the sound representation can be improved, the discriminative power of the feature vectors can be enhanced, multi-dimensional inputs can be provided for the sound effect matching model, and the accuracy and personalization of sound effect parameter prediction can be improved.

[0075] Step S303: Generate sub-feature vectors corresponding to acoustic features of multiple categories.

[0076] A sub-feature vector is a vector representation formed by encoding or organizing acoustic features of a certain category. Each sub-feature vector corresponds to a feature category and is used to represent the acoustic information of that category. Sub-feature vectors usually have a fixed dimension to facilitate subsequent fusion.

[0077] Here, numerical processing and normalization of each category of acoustic features can eliminate dimensional differences. Feature embedding through linear transformations, feature mapping, or encoding networks generates vector representations for the corresponding categories. For example, the fundamental frequency sequence of consecutive frames can be statistically analyzed into mean, variance, etc., to form pitch sub-vectors; Mel spectral features can be encoded into spectral sub-vectors through convolutional networks. Unifying features of different categories into vector form achieves a structured representation of features, providing a unified input format for the fusion of features from different categories, and improving the consistency and computability of subsequent model inputs.

[0078] Step S304: Fuse the sub-feature vectors to obtain the feature vector.

[0079] Fusion refers to combining multiple sub-feature vectors to form a unified, comprehensive feature vector. Fusion methods can include concatenation, weighted summation, linear transformation, or attention fusion.

[0080] Feature vectors refer to the comprehensive vectors formed after fusion, which are used to represent the overall characteristics of the user's voice and serve as input to the sound effect matching model.

[0081] Multiple sub-feature vectors can be concatenated sequentially, or weighted and summed according to preset weights; or linearly combined through a fusion layer to output a feature vector of the same dimension. This approach integrates acoustic information from different dimensions, improving the completeness of sound representation, enhancing the discriminative power of feature vectors, and enabling the model to simultaneously consider multiple features such as pitch, timbre, and dynamics, thereby improving the accuracy and stability of sound effect matching.

[0082] Thus, through steps S302-S304, the complex time-series audio signal is transformed into a structured, multi-dimensional, and computable sound feature representation, constructing a high-dimensional representation space for user voices; providing high-quality input for establishing the mapping relationship between sound characteristics and sound effect parameters, and improving the sensitivity and prediction accuracy of the sound effect matching model to the differences in different user voices.

[0083] Step S305: Input the feature vector into the pre-built sound effect matching model, and output the sound effect configuration result that matches the user's voice characteristics. The sound effect configuration result includes at least the target sound effect type and the sound effect processing parameters corresponding to the target sound effect type.

[0084] Step S306: Perform sound effect processing on the human voice audio signal according to the sound effect configuration result to obtain the processed human voice audio signal.

[0085] For example, after obtaining the preprocessed human voice audio signal, acoustic features are extracted from the human voice audio signal in the following manner to generate a feature vector that characterizes the user's voice characteristics.

[0086] First, the human voice audio signal is processed by frame segmentation. The audio signal can be segmented into frames using a sliding window according to a preset frame length and frame shift, and each frame of the audio signal is then windowed, for example, using a Hamming window or Hanning window, to reduce the impact of spectral leakage on feature calculation. The preset frame length can be 20ms to 40ms, and the frame shift can be 10ms to 20ms.

[0087] After frame segmentation, acoustic features can be extracted from multiple dimensions. These acoustic features can include, but are not limited to, the following categories: Pitch-related features can be obtained by using fundamental frequency detection algorithms, such as autocorrelation, cepstral method, or pitch detection model based on neural networks, to extract the fundamental frequency value of each frame and further calculate features such as pitch range, pitch change trend, or pitch stability to characterize the pitch characteristics of the user's singing.

[0088] Spectral correlation features can be used to perform Short-Time Fourier Transform (STFT) on the framed audio signal and calculate frequency domain features such as Mel-spectrogram or Mel-MFCC. Spectral correlation features are used to characterize the energy distribution of sound in different frequency ranges. These spectral features can be input into a convolutional coding network for feature embedding to obtain spectral sub-feature vectors, which are used to characterize the user's timbre structure.

[0089] Formant correlation features can be used to estimate the frequency positions and bandwidths of the first to third formants based on linear predictive coding (LPC) or other spectral analysis methods, in order to reflect the morphological characteristics and timbre of the vocal tract.

[0090] Dynamic correlation features can be used to calculate the energy envelope, dynamic range, and rate of energy change of an audio signal, which can be used to characterize volume changes and control capabilities during performance.

[0091] Singing technique-related features can be extracted based on pitch curve analysis to extract vibrato frequency, vibrato amplitude, or glissando features, which can be used to characterize the user's singing technique performance.

[0092] After extracting acoustic features from at least two categories, the features from different categories can be normalized to eliminate the effects of differences in dimensions and amplitude scales. Subsequently, each category of features is converted into a corresponding sub-feature vector. Transformation methods can include direct concatenation, linear transformation, or feature embedding representation through an encoding network.

[0093] In one implementation, sub-feature vectors of different categories can be fused, for example, through vector concatenation, weighted fusion, or attention fusion, to generate a unified high-dimensional voiceprint feature vector V_voice as the feature vector. The feature vector comprehensively represents the user's voice characteristics in multiple dimensions such as pitch, timbre, resonance structure, dynamic performance, and singing skills, and serves as the input for subsequent sound effect matching models.

[0094] Through the above acoustic feature extraction and fusion process, the original audio signal can be transformed into a structured numerical representation, enabling the pitch, timbre, dynamic range, and singing technique features of the user's voice to be modeled in a unified manner, thus providing sufficient data support for establishing the mapping relationship between sound characteristics and sound effect parameters.

[0095] The construction process of the sound effect matching model will be explained below with reference to specific embodiments.

[0096] Figure 4 This is a flowchart illustrating the method for constructing a sound effect matching model in an embodiment of this disclosure.

[0097] like Figure 4 As shown, the sound effect matching model is constructed in the following way: Step S401: Obtain training sample data, which includes user voice feature vectors, corresponding audio effect processing parameter annotations, and satisfaction labels corresponding to the audio effect processing parameter annotations.

[0098] Training sample data refers to the dataset used to train a neural network model. Each sample describes the correspondence between a sound feature, sound effect parameter, and effect evaluation.

[0099] User voice feature vector refers to the structured vector representation of user voice characteristics generated through the aforementioned acoustic feature extraction and fusion steps.

[0100] Audio processing parameter annotation refers to the target audio effect parameter value corresponding to a certain sound feature vector, which may include: target audio effect type annotation, reverb parameter, equalization parameter, compression parameter and other specific numerical annotations.

[0101] Satisfaction rating refers to the labeled data used to reflect the subjective evaluation or quality of sound effect processing. It can be derived from user ratings, whether users adopt the recommended results, the extent of user adjustments to parameters, and expert tuning results.

[0102] Here, feature vectors are extracted from the singing audio of multiple users. Audio engineers manually tune the parameters to generate optimal sound effects, and users' subjective evaluations of the effect of these parameters are recorded. A training dataset in the form of a triplet consisting of feature vectors, sound effect parameter annotations, and satisfaction annotations is constructed. By establishing a foundation of related data, the model training is supervised, which improves the consistency between the model's learning results and the actual usage effect, providing reliable data support for achieving personalized sound effect matching.

[0103] Step S402: Construct a neural network model. The neural network model is used to establish the mapping relationship between the user's voice feature vector and the sound effect processing parameters.

[0104] A neural network model is a computational model with a multi-layered structure used to learn the non-linear mapping relationship between input feature vectors and output sound effect parameters.

[0105] The mapping relationship refers to the functional relationship from the input user voice feature vector to the output sound effect processing parameters, which is expressed through model parameters.

[0106] The neural network model includes: an input layer for receiving user voice feature vectors; multiple hidden layers for non-linear mapping of the feature vectors; and an output layer including at least two output branches: a classification output branch for predicting the target sound effect type; and a regression output branch for predicting the sound effect processing parameters corresponding to the target sound effect type.

[0107] The classification output branch can output the probability distribution of different sound effect types; the regression output branch can output specific parameter values ​​such as reverberation time, equalization gain, and compression ratio.

[0108] In one alternative implementation, satisfaction labels can be introduced into the training process to weight samples or to construct an auxiliary loss function, thereby enhancing the model's ability to fit users' subjective preferences.

[0109] Step S403: Train the neural network model based on the training sample data to obtain the sound effect matching model.

[0110] The training process refers to the process of adjusting model parameters through optimization algorithms, such as gradient descent, so that the model output results approximate the labeled results.

[0111] A sound effect matching model refers to a trained neural network model that has the ability to output matching sound effect parameters based on the input feature vector.

[0112] Here, after the model structure is built, the neural network model is trained based on the training sample data.

[0113] Specifically, the following steps can be taken: divide the training sample data into a training set and a validation set; Input the user's voice feature vector into the model to obtain the predicted sound effect type and sound effect processing parameters; Calculate the error between the prediction results and the annotation results; Construct a joint loss function, including: a classification loss function and a regression loss function; The model parameters are updated using the backpropagation algorithm; after multiple rounds of iterative training, the model parameters gradually converge.

[0114] Thus, after training, a sound effect matching model is obtained. This model can output the corresponding sound effect type and processing parameters based on the input user voice feature vector, thereby achieving automatic sound effect matching.

[0115] Furthermore, the system can continuously collect user feedback data during actual operation, such as users' adoption of recommendation results or parameter fine-tuning behavior, and add the feedback data to the training sample dataset to periodically perform incremental training or fine-tuning of the sound effect matching model, thereby achieving continuous optimization and personalized evolution of the model.

[0116] Specifically, the sound effect matching model is constructed and trained in the following way: We obtain user singing audio samples from actual usage scenarios or recording datasets of accompaniment singing applications and determine the corresponding vocal audio segments. For each user singing audio sample, we first perform preprocessing operations, such as noise reduction, automatic gain control, and silence detection, to obtain relatively stable vocal audio segments.

[0117] Multidimensional acoustic feature extraction is performed on each human voice audio segment to obtain a unified user voice feature vector V_voice. The feature vector can be formed by fusing pitch-related features, spectrum-related features, formant-related features, dynamic range-related features, and singing technique-related features.

[0118] The sound effect configuration results are used as supervisory labels for construction.

[0119] Target sound effect type tag: Select the type that best suits the sound characteristics from the preset sound effect type set, such as "recording studio quality", "live", "retro", etc.

[0120] Audio processing parameter labels: Record a set of parameter values ​​corresponding to the target audio effect type, such as reverberation time / attenuation, equalization gain, compression threshold / compression ratio, etc.

[0121] The aforementioned labels can be obtained through one of the following methods: The parameters are adjusted by professional audio engineers based on reference listening tests, and the final determined parameters are recorded. The annotation is based on the parameter combination that users ultimately adopt in actual use; If optional, the group with the highest satisfaction among multiple candidate parameters is used as the label.

[0122] For each audio sample, a satisfaction label S is generated. The satisfaction label can be derived from: explicit user ratings; or ratings mapped from behavioral indicators such as whether the user adopts the recommendation results, the degree of parameter fine-tuning, and the listening duration.

[0123] In this way, each training sample can form a triple: (V_voice, Y_effect, Y_param, S), where V_voice is the user's voice feature vector, Y_effect is the sound effect configuration annotation (including the target sound effect type Y_type and the corresponding sound effect processing parameter Y_param), and S is the satisfaction annotation.

[0124] In this embodiment, the sound effect matching model shares a backbone network and classification and regression output branches connected to the backbone network.

[0125] The input to the model in the input layer is V_voice. Its dimension is D, which can be 128, 256, or other dimensions, depending on the feature design. A normalization layer can be optionally configured in the input layer to improve training stability.

[0126] A shared backbone network is used to improve high-level semantic representation by mapping the input feature vector V_voice to a unified hidden representation H, which can be achieved using several fully connected layers or other feasible structures.

[0127] A shared backbone network can include several levels of feature mapping layers, such as a first feature mapping layer, a second feature mapping layer, and an activation layer. Taking a fully connected structure as an example, the first feature mapping layer maps V_voice to a first hidden vector, the second feature mapping layer further maps the first hidden vector to a second hidden vector, and the second hidden vector is output as an intermediate representation H to subsequent branch networks. Non-linear activation functions and regularization operations can be set between each feature mapping layer to improve training stability and generalization ability.

[0128] For example, the shared backbone network performs the following common feature transformation on the input V_voice to obtain the intermediate representation H: ; in f _i(·) represents the transformation function of the i-th level feature mapping layer. The transformation function can be a combination of linear transformation and nonlinear activation, and can further include normalization and regularization processing. The classification output branch takes H as input and outputs the target sound effect type prediction result, while the regression output branch takes H as input and outputs the sound effect processing parameter prediction result.

[0129] The sound effect type classification output branch is used to predict the target sound effect type based on the intermediate representation H. Specifically, the sound effect type classification output branch receives the intermediate representation H as input and outputs the category prediction result corresponding to the preset sound effect type set. The category prediction result can be a probability distribution vector P_type, where P_type[i] represents the probability that the input sound feature matches the i-th sound effect type; based on this, the sound effect type with the highest probability can be selected as the target sound effect type, or multiple candidate sound effect types can be output for subsequent processing. The sound effect parameter regression output branch is used to predict the sound effect processing parameters corresponding to the target sound effect type based on the intermediate representation H. Specifically, the sound effect parameter regression branch receives the intermediate representation H as input and outputs a parameter vector Y^_param. The parameter vector includes at least two types of parameter values ​​such as reverberation parameters, equalization parameters, and dynamic processing parameters, which are used for parameter settings in subsequent sound effect processing chains.

[0130] In one optional implementation, to ensure the consistency between the sound effect type prediction result and the parameter prediction result, the parameter vector output by the sound effect parameter regression output branch corresponds to the target sound effect type output by the sound effect type classification output branch. That is, the parameter vector is the parameter set corresponding to the target sound effect type, or it is the parameter adjustment amount superimposed on the basic parameters of the target sound effect type.

[0131] In another optional implementation, the sound effect parameter regression output branch may include multiple parameter output heads, each corresponding to a sound effect type; during the inference phase, the target sound effect type determined by the sound effect type classification output branch is selected, and the sound effect processing parameters are output from the parameter output head corresponding to the target sound effect type.

[0132] After the model structure is built, the neural network model is trained based on the training sample data. Specifically, the following steps can be taken: divide the training sample data into a training set and a validation set; input the user's voice feature vector into the model to obtain the predicted target sound effect type and sound effect processing parameters; calculate the error between the prediction result and the labeled result; construct a joint loss function, including a classification loss function, such as cross-entropy loss, and a regression loss function, and combine the classification loss and regression loss into a total loss through weighting; update the model parameters through the backpropagation algorithm; and gradually converge the model parameters through multiple rounds of iterative training. After training, a sound effect matching model is obtained. The sound effect matching model can output the corresponding sound effect type and sound effect processing parameters based on the input user voice feature vector, thereby achieving automatic sound effect matching.

[0133] For example, during model training, a joint loss function is constructed to simultaneously optimize classification and regression tasks: ; Where L_cls is the cross-entropy loss used for sound effect type classification, L_reg is the regression loss used for parameter fitting, and α and β are the first and second weight coefficients, respectively.

[0134] In one alternative implementation, satisfaction level labels can be used to weight or filter the training process, giving higher weights to samples with higher satisfaction levels during training, thereby enhancing the model's ability to fit user subjective preferences. To make the model biased towards configurations that better match user preferences, satisfaction level labels can be used to weight the regression loss: ; in, It increases with increasing satisfaction, making high-satisfaction samples contribute more to training.

[0135] In addition, the system can continuously collect user feedback data during actual operation, such as users' adoption of recommendation results or parameter fine-tuning behavior, and add the feedback data to the training sample dataset to periodically perform incremental training or fine-tuning of the sound effect matching model, thereby achieving continuous optimization and personalized evolution of the model.

[0136] Thus, after training, the model parameters are fixed to obtain the sound effect matching model. In actual operation: Input the feature vector V_voice of the user's current singing segment; The highest probability category of the target sound effect type P_type is used as the recommended type. Simultaneously output the corresponding sound effect processing parameter Y^_param; The parameters are then sent to the audio processing chain for processing.

[0137] The sound effect processing process is explained below with reference to specific embodiments.

[0138] Figure 5 This is a flowchart illustrating another method for matching human voice effects provided in this embodiment of the disclosure.

[0139] like Figure 5 As shown, the method includes: Step S501: Collect the human voice audio signal generated by the user in the accompaniment singing scene.

[0140] Step S502: Extract acoustic features from the human voice audio signal to obtain a feature vector that characterizes the user's voice characteristics.

[0141] Step S503: Input the feature vector into the pre-built sound effect matching model, and output the sound effect configuration result that matches the user's voice characteristics. The sound effect configuration result includes at least the target sound effect type and the sound effect processing parameters corresponding to the target sound effect type.

[0142] Step S504: Determine the combination of sound effect processing modules according to the target sound effect type.

[0143] Audio processing modules refer to processing units used to perform specific audio processing functions, such as equalization processing modules, reverb processing modules, dynamic compression processing modules, and delay processing modules.

[0144] The combination of modules refers to the set of sound effect processing modules selected to achieve a certain type of sound effect and their composition, such as selecting the combination of "equalization + reverb + compression" or the combination of "equalization + delay".

[0145] A mapping table between sound effect types and processing chain templates can be pre-established, where each sound effect type corresponds to at least one set of sound effect processing module combinations; after receiving the target sound effect type, the corresponding set of processing modules is determined by looking up the table or according to rules; the determined module combination is instantiated into an executable sound effect processing chain.

[0146] Step S505: Set the parameters of each sound effect processing module according to the sound effect processing parameters.

[0147] Audio processing parameters refer to the set of parameters output by the audio matching model that are used to control the behavior of the audio processing module. These parameters may include reverberation time / attenuation, equalization gain of each frequency band, compression threshold / ratio, delay length, etc.

[0148] Setting the parameters for each module involves writing the parameter values ​​into the control interface of the corresponding audio processing module to determine how and how intense the module processes the input audio.

[0149] The audio processing parameters are parsed into subsets corresponding to each module, such as reverb parameters, equalization parameters, and dynamic parameters. Each subset is then written into the parameter register / configuration interface of its corresponding module. Optionally, the parameter range can be constrained or pruned, such as limiting the gain range or the upper limit of the reverb time, to ensure processing stability.

[0150] Step S506: Perform digital signal processing on the human voice audio signal sequentially according to the preset processing order to obtain the processed human voice audio signal.

[0151] The preset processing order refers to the execution order of each processing module in the audio effect processing chain, such as "equalization → compression → reverb" or "equalization → reverb → delay". This order can be preset based on engineering experience or bound to the target audio effect type.

[0152] Digital signal processing refers to the processing operations performed on audio signals in the digital domain, such as filtering, convolution, gain adjustment, and dynamic control, to change the spectral, dynamic, or spatial characteristics of the audio.

[0153] The human voice audio signal to be processed is sequentially input into each sound effect processing module determined in step S504; each module processes the input audio according to the parameters set in step S505 and outputs intermediate results; the intermediate results are used as input for the next module until all modules have completed processing; the final processed human voice audio signal is output.

[0154] In this way, the abstract results output by the sound effect matching model are transformed into an executable audio processing flow, ensuring the consistency between sound effect style selection and specific processing, improving the automation, repeatability and stability of sound effect processing, and providing personalized and practical human voice effect output for different users.

[0155] For example, after the sound effect matching model outputs the sound effect configuration result, the sound effect rendering engine on the device terminal side automatically loads and applies parameters according to the sound effect configuration result, thereby performing sound effect processing on the user's voice audio signal. Specifically, this includes: The combination of sound effect processing modules is determined based on the target sound effect type. A mapping table between sound effect types and processing chains is pre-established on the device terminal side, allowing different sound effect types to correspond to different combinations of sound effect processing modules. For example, the following processing chain can be preset: Studio sound quality: Equalizer (EQ) + Compressor (Comp) + Reverb (Reverb) Live setup type: Equalizer (EQ) + Compressor (Comp) + Reverb (Reverb) + Delay (Delay); Retro phonograph type: Equalizer module (EQ) + bandpass / distortion effect module (optional) + reverb module (Reverb).

[0156] When the target sound effect type in the sound effect configuration result is a certain type, the sound effect rendering engine selects the corresponding processing chain template from the corresponding relationship table and instantiates it into a sound effect processing module combination, which is used as the object for subsequent parameter setting and execution processing.

[0157] The parameters for each module are set according to the sound effect processing parameters. The sound effect configuration result includes sound effect processing parameters corresponding to the target sound effect type, such as reverb level R, bass gain Bass, treble gain Treble, compression strength Comp, etc. In this embodiment, the sound effect rendering engine parses this parameter set into parameter subsets for different modules and writes them into the configuration interface of the corresponding module, realizing parameter distribution and implementation. For example: For example: Equalizer module parameter mapping: Map the Bass to a low-frequency gain G_low, such as a gain of 80–200Hz; map the Treble to a high-frequency gain G_high, such as a gain of 4–12kHz; and give the mid-frequency gain G_mid based on the sound feature vector or the default template, forming a three-band equalizer parameter set {G_low, G_mid, G_high}.

[0158] Reverberation module parameter mapping: Map the reverberation size R to one or a combination of parameters such as reverberation time T_60, predelay, and reverberation wet-dry ratio Mix; For example, R can be linearly or piecewise mapped to the range of T_60, and Mix can be adjusted synchronously to control the intensity of spatial perception.

[0159] Compression module parameter mapping: Map the compression intensity Comp to one or more of the compression ratio Ratio, threshold Threshold, start time Attack, and release time Release; For example, the compression ratio can be increased and the threshold lowered as Comp increases, making the dynamics of the human voice smoother.

[0160] After completing the module combination and parameter settings, the sound rendering engine performs digital signal processing on the human voice audio signal sequentially according to the preset processing order, and outputs the processed human voice audio signal. An example processing order is as follows: For "Studio Quality Type": EQ → Comp → Reverb; For "Live Scene Type": EQ→Comp→Delay→Reverb.

[0161] During processing, each frame or segment of human voice audio data can be sent to the first module for processing to obtain intermediate results; these intermediate results are then used as input for the next module until all modules have completed processing, outputting the final human voice audio signal. This processed human voice audio signal can be mixed with the accompaniment audio for playback or output, thus creating the final listening experience in a accompaniment singing scenario.

[0162] Through the above implementation method, the target sound effect type and sound effect processing parameters output by the model can be converted into a processing chain structure and parameter configuration that can be executed on the terminal side, and the sound effect rendering engine can automatically complete the sequential digital signal processing to achieve automatic application of sound effects without manual parameter adjustment.

[0163] In some embodiments, the above-described voice effect matching method further includes a feedback update scheme based on user actions.

[0164] Figure 6 This is a flowchart illustrating another method for matching human voice effects provided in this embodiment of the disclosure.

[0165] like Figure 6 As shown, the method includes: Step S601: Collect the human voice audio signal generated by the user in the accompaniment singing scene.

[0166] Step S602: Extract acoustic features from the human voice audio signal to obtain a feature vector that characterizes the user's voice characteristics.

[0167] Step S603: Input the feature vector into the pre-built sound effect matching model and output the sound effect configuration result that matches the user's voice characteristics; the sound effect configuration result includes at least the target sound effect type and the sound effect processing parameters corresponding to the target sound effect type.

[0168] Step S604: Perform sound effect processing on the human voice audio signal according to the sound effect configuration result to obtain the processed human voice audio signal.

[0169] Step S605: Obtain the user's operation behavior on the sound effect configuration result, and store the operation behavior as feedback data.

[0170] User actions related to sound effect configuration results refer to the interactive operations or usage behaviors performed by users in response to the sound effect configuration results, reflecting the user's acceptance level or preference for the configuration results. These actions may include, but are not limited to: whether to adopt the recommended sound effect type, such as whether to enable / switch; fine-tuning sound effect processing parameters, such as adjusting reverb intensity, equalization gain, compression intensity, etc.; evaluating the processing effect, such as rating, liking / disliking; and listening and comparing during performance, such as switching between original sound and effects.

[0171] Feedback data refers to structured records collected from user actions, used for subsequent model updates or user profile updates. Feedback data is written to storage media or a database, which can be local or cloud-based, for later training or statistical analysis.

[0172] Here, during the display of sound effect configuration results or the processing of sound effects, user interaction events are monitored; these interaction events are transformed into structured data items to form feedback records associated with this recommendation. These feedback records may include at least: User ID or session ID; Target sound effect type and its parameters; Whether the user accepts / switches the information; The parameter value or adjustment range after user fine-tuning; Optional subjective ratings or implicit preference signals; The feedback records will be stored in a user preference profile database or a log storage system.

[0173] Step S606: Update the sound effect matching model based on the feedback data to improve the matching degree between subsequent sound effect configuration results and user preferences.

[0174] Model update refers to the process of adjusting the parameters or configuration of a sound effect matching model based on newly acquired feedback data, including but not limited to incremental training, fine-tuning, or personalized parameter updates.

[0175] The degree of matching refers to the consistency between the sound effect configuration results output by the subsequent model and the user's actual preferences, which can be reflected in an increase in user adoption rate, a decrease in fine-tuning range, or an increase in satisfaction.

[0176] Here, model updates can be implemented in various ways, such as: incremental training / fine-tuning, where feedback data is transformed into training samples, and the final parameter combination adopted by the user is used as the new label; new samples are added to the training set periodically or in batches for incremental training or fine-tuning of the model; the updated model is then used for inference for all users or a specific user group. Another example is personalized parameter updates, where preference parameters or personalized vectors are maintained for individual users; their personalized parameters are updated based on the user's fine-tuning direction and magnitude; during inference, the personalized parameters are combined with the output of the base model to form a sound effect configuration result that better suits the user. Yet another example is satisfaction-weighted updates, where training samples are weighted based on ratings or adoption behavior, allowing high-satisfaction samples to contribute more to the update.

[0177] Thus, the S605 acquires and stores the user's true preference signal, and the S606 uses this preference signal to update the model or personalized parameters. This upgrades the one-off sound effect recommendation system into a continuously evolving personalized matching system, avoiding the insufficient generalization of static models across different users; and continuously improving matching accuracy and user satisfaction in real-world application environments.

[0178] Combination Figure 7 As shown, this disclosure provides an apparatus for human voice effect matching, including a processor 700 and a memory 701. Optionally, the apparatus may further include a communication interface 702 and a bus 703. The processor 700, communication interface 702, and memory 701 can communicate with each other via the bus 703. The communication interface 702 can be used for information transmission. The processor 700 can call logical instructions in the memory 701 to execute the human voice effect matching method of the above embodiment.

[0179] Furthermore, the logic instructions in the aforementioned memory 701 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0180] The memory 701, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 700 executes functional applications and data processing by running the program instructions / modules stored in the memory 701, that is, it implements the method for human voice effect matching in the above embodiments.

[0181] The memory 701 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 701 may include high-speed random access memory and may also include non-volatile memory.

[0182] This disclosure also provides an audio-visual device, including: a device body, and the aforementioned device for voice effect matching. The device for voice effect matching is installed in the device body. The installation relationship described herein is not limited to placement inside the device body, but also includes installation connections with other components of the audio-visual device, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the device for voice effect matching can be adapted to feasible device bodies to achieve other feasible embodiments.

[0183] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described method for human voice effect matching.

[0184] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0185] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0186] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0187] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0188] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for matching human voice effects, characterized in that, include: Collect the human voice audio signal generated by the user in a background singing scenario; Acoustic feature extraction is performed on the human voice audio signal to obtain a feature vector that characterizes the user's voice characteristics; The feature vector is input into a pre-built sound effect matching model, and the sound effect configuration result that matches the user's voice characteristics is output; the sound effect configuration result includes at least the target sound effect type and the sound effect processing parameters corresponding to the target sound effect type; The human voice audio signal is processed according to the sound effect configuration result to obtain the processed human voice audio signal.

2. The method according to claim 1, characterized in that, The acquisition of human voice audio signals generated by users in accompaniment singing scenarios includes: Collect the sound signal generated by the user's singing; the sound signal includes the human voice and the accompaniment. The sound signal is subjected to human voice separation processing to obtain the human voice audio signal.

3. The method according to claim 1, characterized in that, The step of extracting acoustic features from the human voice audio signal to obtain a feature vector characterizing the user's voice characteristics includes: The human voice audio signal is processed by frame segmentation; Extract acoustic features from multiple categories; Generate corresponding sub-feature vectors for the acoustic features of the multiple categories; The sub-feature vectors are fused to obtain the feature vector.

4. The method according to claim 3, characterized in that, The acoustic features include at least two of the following categories: pitch-related features, spectrum-related features, formant-related features, dynamic features, and singing technique-related features.

5. The method according to claim 1, characterized in that, The sound effect matching model is constructed in the following way: Acquire training sample data, which includes user voice feature vectors, corresponding audio effect processing parameter annotations, and satisfaction labels corresponding to the audio effect processing parameter annotations; A neural network model is constructed, which is used to establish the mapping relationship between the user's voice feature vector and the sound effect processing parameters; The neural network model is trained based on the training sample data to obtain a sound effect matching model.

6. The method according to claim 5, characterized in that, The neural network model includes a classification output branch for predicting the target sound effect type and a regression output branch for predicting sound effect processing parameters.

7. The method according to claim 1, characterized in that, The step of performing sound effect processing on the human voice audio signal according to the sound effect configuration result to obtain the processed human voice audio signal includes: The combination of sound effect processing modules is determined based on the target sound effect type; Set the parameters of each sound effect processing module according to the aforementioned sound effect processing parameters; The human voice audio signal is processed digitally in sequence according to a preset processing order to obtain the processed human voice audio signal.

8. The method according to any one of claims 1 to 7, characterized in that, After performing sound effect processing on the human voice audio signal according to the sound effect configuration result, the method further includes: The system acquires the user's actions regarding the sound effect configuration results and stores these actions as feedback data. The sound effect matching model is updated based on the feedback data to improve the matching degree between subsequent sound effect configuration results and user preferences.

9. An apparatus for human voice effect matching, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to, when executing the program instructions, perform the method for human voice effect matching as described in any one of claims 1 to 8.

10. An audio-visual device, characterized in that, include: Equipment body; The device for matching human voice effects as described in claim 9 is installed on the device body.

Images