Method and system for dynamic optimization of a vocal effects chain

Abstract

This invention discloses a method and system for dynamic optimization of vocal effect chains, belonging to the interdisciplinary field of audio signal processing and artificial intelligence. The method aims to solve the problems of existing automatic tone-adjusting schemes failing to accurately understand user semantic intent, struggling to collaboratively optimize the complete effect chain, and lacking closed-loop feedback optimization capabilities. The method includes: acquiring and extracting timbre feature vectors from user vocal audio; inputting the timbre feature vectors into a timbre classification model to obtain timbre classification results; obtaining the user's natural language tone-adjusting instructions and combining them with the timbre classification results, inputting them into a large language model to generate a tone-adjusting strategy; mapping the tone-adjusting strategy to specific digital signal processing parameter configurations and sending them to the edge digital signal processing effect chain to process the audio; collecting user feedback information on the output audio; and dynamically optimizing the tone-adjusting strategy generation capability of the large language model or the mapping relationship between the tone-adjusting strategy and parameter configurations based on the feedback information, forming a closed loop. This application achieves a deep integration of semantic understanding and acoustic analysis and constructs a closed-loop feedback optimization mechanism, balancing real-time performance and intelligence.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing and artificial intelligence, and in particular to an intelligent tone-tuning method and system for real-time human voice processing. Background Technology

[0002] In real-time audio applications such as karaoke, live streaming, and voice calls, to enhance vocal expressiveness and listening experience, it is usually necessary to process the raw vocal signal through an effects chain. This involves adjusting timbre, dynamics, and spatiality using digital signal processing modules such as equalizers, compressors, and reverb units. Traditional professional sound mixing processes heavily rely on the manual operation and subjective experience of sound engineers, making them difficult for ordinary users to master.

[0003] To lower the barrier to entry for users, the industry has proposed various automated or intelligent tuning solutions. Existing technologies mainly fall into the following categories: Rule-based automatic tuning solutions: These solutions adjust parameters by using preset acoustic feature detection rules (e.g., increasing the gain of the corresponding frequency band if high-frequency deficiency is detected). Their drawbacks include fixed adjustment logic, high dependence on the quality of preset rules, lack of adaptability to complex and diverse timbres and user-specific needs, and poor generalization.

[0004] Automatic tuning schemes based on timbre classification networks: These schemes use deep learning models to classify the user's timbre features and map the classification results to a set of preset effect parameters. Their drawback is that they typically only output a limited, discrete combination of parameters, lacking the ability to understand and reason about the user's complex and continuous semantic tuning intentions, making it difficult to achieve nuanced and dynamic personalized adjustments.

[0005] End-to-end neural network-based voice enhancement solutions: These solutions use deep learning models to directly map the original human voice into an enhanced voice signal. Their drawbacks include the fact that the model is typically a "black box," with poor interpretability, making it difficult for users to understand or intervene in the processing. Furthermore, the large model size can lead to high real-time inference latency and makes it difficult to work in conjunction with traditional, flexibly configurable DSP effects chains.

[0006] Furthermore, most existing automatic tuning solutions end the process after parameter generation, lacking a mechanism for continuous optimization and adjustment based on actual user listening feedback. This prevents the formation of a closed loop of "perception-decision-execution-optimization," making it difficult for the system to adapt to changes in user subjective preferences, different devices, and scenarios.

[0007] Therefore, how to design an intelligent tuning method that can accurately understand the user's semantic intent, collaboratively optimize multiple parameters of the complete effect chain, and has the ability to perform closed-loop dynamic optimization based on user feedback has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] The purpose of this invention is to provide a method, system, and computer-readable storage medium for dynamic optimization of vocal effect chains, aiming to solve the technical problems in the prior art where automatic tone adjustment schemes cannot accurately understand user semantic intent, are difficult to collaboratively optimize the complete effect chain, and lack the ability to perform closed-loop dynamic optimization based on user feedback.

[0009] To address the aforementioned technical problems, this invention provides a dynamic optimization method for a vocal effects chain, comprising the following steps: acquiring real-time vocal audio signals from users, preprocessing the audio signals and extracting acoustic features to obtain structured timbre feature vectors; inputting the timbre feature vectors into a timbre classification model to obtain the user's timbre classification results and continuous feature representations; acquiring the user's natural language tuning instructions, and combining the timbre classification results and historical tuning status information, inputting them into a large language model for joint inference to generate a tuning strategy for the user's current audio; mapping the tuning strategy to specific digital signal processing parameter configurations, and sending the DSP parameter configurations to the edge-side digital signal processing effects chain; using the DSP effects chain to process the real-time vocal audio signals according to the sent parameter configurations, and outputting the processed audio signals; acquiring user feedback information on the output audio; and based on the feedback information, dynamically optimizing the tuning strategy generation capability of the large language model and / or the mapping relationship between the tuning strategy and the DSP parameter configurations to form a closed-loop optimization mechanism.

[0010] This invention also provides a dynamic optimization system for vocal effects chains, used to implement any of the aforementioned methods. The system includes: an edge-side real-time audio processing module for acquiring and preprocessing real-time vocal audio signals, extracting timbre features, running a timbre classification model, and running a DSP effects chain to process the audio; a cloud-based large-model tuning decision module for receiving timbre features and user tuning instructions, generating tuning strategies and DSP parameter configurations through a large language model, and dynamically optimizing based on feedback information; an edge-cloud communication and interaction module for transmitting timbre features, tuning instructions, DSP parameter configurations, and feedback information between the edge-side real-time audio processing module and the cloud-based large-model tuning decision module; and a feedback acquisition module for acquiring user feedback information on the output audio and sending the feedback information to the cloud-based large-model tuning decision module.

[0011] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the dynamic optimization method for the human voice effects chain as described in any of the preceding claims.

[0012] Compared with existing technologies, the beneficial effects of this invention are as follows: 1. It achieves deep integration of semantic understanding and acoustic analysis: By combining the ability of a large language model to understand user natural language commands with the ability of a timbre classification model to analyze the objective acoustic characteristics of human voices, the tuning decisions can respond to users' personalized and ambiguous needs, and can also make precise adjustments based on objective timbre data, overcoming the shortcomings of traditional solutions that either rely on fixed rules or lack semantic understanding. 2. It achieves collaborative optimization and interpretability of the complete effects chain: Through a modular decomposition mechanism, the advanced tuning strategy is decomposed into adjustment tasks for multiple specific DSP effects modules such as equalizers and compressors, and parameter mapping is performed using a knowledge base, achieving overall and collaborative control of the effects chain, rather than isolated parameter adjustments. At the same time, the generated interpretable text enhances the transparency of the system and user trust. 3. It constructs a closed-loop feedback optimization mechanism: By collecting explicit or implicit user feedback, the strategy generation capability and parameter mapping relationship of the large model are continuously optimized, enabling the system to continuously learn user preferences, dynamically adapt to different users, devices, and environments, and possess the ability to self-evolve. 4. Balancing real-time performance and intelligence: Adopting an edge-cloud collaborative architecture, low-latency audio processing tasks are placed on the edge, while high-computing-power-requirement model inference tasks are placed in the cloud, effectively ensuring the user experience in real-time audio application scenarios, while fully utilizing the powerful intelligent decision-making capabilities of large cloud models. Attached Figure Description

[0013] Figure 1 This is a core flowchart of a dynamic optimization method for human voice effect chain according to an embodiment of the present invention.

[0014] Figure 2 This is a preferred end-cloud architecture diagram of a dynamic optimization system for human voice effect chain according to an embodiment of the present invention.

[0015] Figure 3 This is a detailed flowchart of interactive tuning according to an embodiment of the present invention.

[0016] Figure 4 This is a schematic diagram of a user interface according to an embodiment of the present invention.

[0017] Figure 5 This is a schematic diagram of parameter mapping data flow according to an embodiment of the present invention.

[0018] Figure labeling: 400: Smartphone interface; 401: Smart tuning assistant; 402: Tuning suggestion guidance module; 403: Explanatory text; 404: Text feedback input box; 405: Send; 406: User input area; 501: Tuning strategy; 502: Parameter mapping and knowledge base unit; 503: EQ knowledge base; 504: Compressor knowledge base; 505: DSP parameter configuration; 506: Reverb knowledge base. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by those skilled in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application. Before further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application are explained, and the nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0020] (1) DSP Effect Chain: This refers to an audio processing flow consisting of one or more digital signal processing modules (such as equalizers, compressors, reverb units, etc.) connected in series or parallel. In this invention, it is an execution entity that runs on the edge device and performs effect processing on real-time audio according to the parameter configuration sent from the cloud. These modules work together to refine the timbre, dynamics, spatial sense, and other dimensions of the human voice.

[0021] (2) Large Language Model (LMM): This refers to a deep learning model trained on massive amounts of text data, possessing powerful natural language understanding, generation, and reasoning capabilities. In this invention, its core function is to act as a "tuning brain," capable of understanding users' typically vague and subjective tuning desires (e.g., "make my voice brighter") described in natural language, and combining these with objective timbre analysis results to deduce the advanced "tuning strategies" required to achieve those desires.

[0022] (3) Timbre feature vector: refers to the structured numerical array extracted from the original audio signal that can characterize the timbre of the human voice (such as brightness, thickness, fundamental frequency, harmonic structure, formants, etc.). It is a mathematical abstract representation of the audio signal and is the objective basis for subsequent timbre classification models and LLM analysis and decision-making, enabling the machine to "understand" and quantify the characteristics of the human voice.

[0023] (4) Tuning strategy: This refers to an intermediate layer of instructions generated by the large language model, which is higher than the specific parameters. It describes "what to do" (e.g., "enhance the clarity and penetration of the human voice", "increase the spatiality and thickness of the sound"), rather than "how to do it" (e.g., "boost the equalizer by 3dB at the 8kHz frequency point"). This strategy serves as a bridge connecting human intentions and machine execution, and will be further mapped to specific DSP parameters.

[0024] (5) Modular decomposition mechanism: This refers to an advanced working method of large language models when generating tuning strategies. It first decomposes a complex and comprehensive tuning instruction (such as "make the sound more like the effect from the recording studio") into relatively independent adjustment tasks for multiple independent DSP effect modules (such as equalizer, compressor, reverb, dessibilizer, etc.), thereby achieving coordinated and multi-dimensional optimization of the entire effect chain, rather than adjusting a single module in isolation.

[0025] Please see Figure 1 This application provides a method for dynamic optimization of the vocal effects chain, aiming to solve the problems of existing automatic audio tuning schemes failing to deeply understand users' personalized semantic needs, lacking interpretability, and failing to form a feedback loop for continuous optimization. The core idea of ​​this method is to construct a complete closed loop of "perception-decision-execution-optimization," using the user's real-time audio signal as the perception input, making decisions through a cloud-based intelligent agent, sending the decisions to the edge device for execution, and continuously iterating and optimizing the decision model based on user feedback.

[0026] In a specific execution flow, the method first performs step S1, which involves capturing the user's voice audio signal in real time on the user's device (such as a smartphone, personal computer, or dedicated karaoke device) using an audio acquisition device such as a microphone. To facilitate subsequent processing, the system preprocesses the acquired raw audio signal, such as performing noise reduction to eliminate environmental noise interference, echo cancellation to avoid sound overlap, and normalization to unify the volume level. Subsequently, the system extracts a series of acoustic features from the preprocessed audio signal, such as Mel-frequency cepstral coefficients (MFCC), fundamental frequency, zero-crossing rate, and spectral centroid, and organizes these features into a structured timbre feature vector. The purpose of this step is to transform the unstructured audio waveform into quantifiable data that the machine can understand and analyze, laying the foundation for subsequent timbre classification and intelligent decision-making.

[0027] Next, step S2 is executed, inputting the timbre feature vector extracted in the previous step into a pre-trained timbre classification model. This model can be a deep neural network, trained to recognize various objective attributes of the human voice. The model outputs two parts: one part is a discrete classification result, such as classifying the user's timbre as "bright," "full," or "husky," etc.; the other part is a continuous feature representation, i.e., coordinates in a multi-dimensional feature space, which more precisely describes the specific location of the timbre. Through this step, the system gains an objective and in-depth understanding of the user's voice, solving the shortcomings of traditional solutions that rely solely on simple rules and cannot accurately grasp the essence of timbre.

[0028] Subsequently, in step S3, the system obtains the user's natural language tuning instructions. The user can express their subjective tuning needs through text input or speech recognition, such as "I want my voice to sound warmer" or "Make my voice more magnetic." At this point, the system inputs this natural language instruction, along with the timbre classification results and continuous feature representations obtained in step S2, and any existing historical tuning state information (such as the user's previous settings), into a large language model (LLM) deployed in the cloud for joint reasoning. Leveraging its powerful semantic understanding and logical reasoning capabilities, the LLM comprehensively analyzes this information to generate a high-level, abstract tuning strategy. For example, for the instruction of "warmer" and "brighter" timbre, the model might generate a strategy that "moderately attenuates the energy of the high-frequency components while slightly enhancing the fullness of the mid-low frequencies."

[0029] After generating the tuning strategy, the method proceeds to step S4. The core task of this step is to transform the abstract tuning strategy generated in step S3 into digital signal processing (DSP) parameter configurations that can be executed on specific hardware. This mapping process is completed in the cloud, translating strategy language such as "attenuate high frequencies" into precise instructions such as "apply a -2dB shelving filter to the frequency band above 8kHz in the equalizer module." The final result is a set of structured parameters, such as a JSON object, containing the name of the DSP module to be adjusted, the parameter name, and the specific parameter value. In this way, the present invention links human subjective perception with the precise execution of the machine.

[0030] Next, step S5 is executed, where the DSP parameter configuration generated in the cloud is sent to the edge device via the network. A DSP effects chain runs on the edge device, processing the user's vocal audio signal in real time based on the received parameter configuration. For example, the equalizer module in the effects chain loads new filter settings, and the compressor module adjusts its threshold and compression ratio. After processing, an enhanced and optimized audio signal is output, which the user can hear in real time through headphones or speakers. This step, through the edge-cloud separation architecture, ensures low latency in audio processing, meeting the needs of real-time applications.

[0031] Next, to enable the system to learn and evolve, the method executes step S6, which involves collecting user feedback on the output audio. This feedback can be multi-dimensional, ranging from direct, explicit user feedback to implicit feedback indirectly observed by the system. For example, a user can directly type "I feel there's too much reverb" on the interface, or express their preference through like / dislike buttons. These are all examples of explicit feedback. Simultaneously, the system can also obtain implicit feedback by analyzing user behavior, such as a user repeatedly listening to a particular effect, or manually fine-tuning the effect based on AI recommendations.

[0032] Finally, the method executes step S7, a crucial step in closed-loop optimization. The system uploads the feedback information collected in step S6 to the cloud. The cloud-based optimization module dynamically optimizes the entire system based on this feedback. The optimization goals can be twofold: first, to fine-tune the pitch strategy generation capability of the large language model, making the subsequently generated strategies more aligned with the user's preferences; and second, to adjust the mapping relationship from pitch strategies to DSP parameter configurations, enabling the same strategy to produce more accurate results. Through this continuous learning and iteration based on real user feedback, the system can continuously evolve, becoming increasingly "comprehensive" of the user, thus achieving personalized, adaptive, and dynamic optimization.

[0033] Furthermore, in a preferred embodiment, to improve user experience and system transparency, in step S3, the large language model generates an interpretable tuning description text along with the tuning strategy. (See reference...) Figure 4As shown, after tuning is completed, an explanatory text (403) can be displayed on the interface (400), such as: "The clarity of the mid-high frequency part has been appropriately improved for you, and some compression effects have been added to make your voice sound more penetrating and energetic." This approach makes the "black box" AI decision-making process transparent, which not only helps users understand the operation done by the system, but also plays a teaching role, enhancing users' sense of trust and control. In addition, when the user's natural language tuning instructions are unclear, such as when the user only says "tune it", the system will activate a tuning suggestion guidance module (402). This module will actively suggest some specific and selectable tuning directions to the user based on the current timbre analysis results, such as asking on the interface (400): "Do you want the voice to sound A. clearer and brighter, or B. more mellow and magnetic?" Through this interactive guidance, the system can help users clarify their tuning goals, solve the problem that users have difficulty accurately expressing their needs due to a lack of professional knowledge, and improve the success rate of tuning.

[0034] Furthermore, to achieve refined and coordinated control over complex effects chains, a modular decomposition mechanism is introduced into the reasoning process of the large language model for the tuning strategy in step S3. When a comprehensive tuning instruction is received, such as "make my voice sound like a radio host," the large language model does not attempt to generate a single, general set of parameters. Instead, it first decomposes this high-level goal into adjustment tasks for multiple different DSP effect modules in the effects chain. For example, it might generate the following task list: {Task 1: For the 'compressor' module, the goal is to increase the dynamic density and power of the sound}, {Task 2: For the 'equalizer' module, the goal is to enhance the resonance of the mid-low frequencies and slightly suppress sibilance}, {Task 3: For the 'reverb' module, the goal is to add a slight room reverberation to increase the sense of space}. In this way, complex tuning requirements are decomposed into a series of independent, more manageable, and executable sub-tasks, laying the foundation for subsequent precise parameter generation. This modular design enables the invention to handle highly complex tuning requirements and achieve synergistic optimization of the entire effects chain. Compared to solutions that can only adjust individual modules, the invention can achieve synergistic optimization of the effects chain.

[0035] Furthermore, such as Figure 5As shown, in step S4, which maps the tuning strategy (501) to specific DSP parameter configurations, the system employs a knowledge base-based query mechanism. For each parameter adjustment task generated for each DSP effect module in the previous step, the system queries a corresponding DSP module knowledge base. For example, for the equalizer (EQ) module, there is an EQ knowledge base (503); for the compressor module, there is a compressor knowledge base (504); and for the reverb module, there is a reverb knowledge base (506). These knowledge bases store a large amount of expert knowledge, acoustic rules, and device adaptation information. For example, when receiving an EQ adjustment task of "increasing sound brightness," the parameter mapping unit (502) queries the EQ knowledge base (503), which may return a rule: "For vocals, 'increasing brightness' typically corresponds to a shelving boost in the 6kHz to 10kHz frequency band, with a gain range of +1.5dB to +4dB." The parameter mapping unit (502) then combines the current user's timbre characteristics and device model (e.g., different microphone models have different frequency response curves) to select an optimal initial value from this range, ultimately generating a specific DSP parameter configuration (505). In this embodiment, the DSP effects module includes at least one or more combinations of equalizers, compressors, reverbs, and de-essing units. This knowledge-based mapping method, compared to the end-to-end black-box model, not only ensures the professionalism and rationality of the generated parameters but also enhances the interpretability and maintainability of the system.

[0036] In one optional implementation, to construct a comprehensive and accurate feedback system, the feedback information collected in step S6 may include multiple types. The first type is text feedback directly input by the user, such as... Figure 4 As shown, the user can enter "The sibilance is still a bit heavy" in the text feedback input box (404) of the user input area (406) and then click the send button (405) to submit. This feedback is the most direct and rich in semantic information. The second type is implicit feedback generated based on the user's operation behavior on the output audio. For example, the system provides the user with two tuning effects, A and B. If the user repeatedly plays and records on effect A, but only listens to effect B for a few seconds before switching, the system can infer that the user prefers effect A. This implicit feedback does not require the user to actively express it, reducing the user's operational burden. The third type is the user's history of manually adjusting DSP parameters. For example, the AI ​​automatically generates a reverb parameter of 30%, but the user manually adjusts it to 25% in the advanced settings. The system will record this -5% preference difference. By comprehensively analyzing these different dimensions of feedback information that combine subjective and objective factors, the system can more comprehensively and deeply understand the user's true preferences, thereby performing more accurate model updates in the optimization stage of step S7.

[0037] Furthermore, such as Figure 2 As shown, the technical solution of this invention can be implemented using a highly efficient edge-cloud collaborative architecture. In this architecture, steps with extremely high real-time requirements, such as audio acquisition, preprocessing, and feature extraction in step S1, timbre classification in step S2 (typically using a lightweight model), and DSP effects chain processing and audio output in step S5, are all executed on the user's edge device. Steps with higher computational resource requirements but less sensitivity to latency, such as large language model inference in step S3, parameter mapping in step S4, and feedback learning and model optimization in step S7, are executed on a cloud server. The timbre feature vector obtained in step S1 and the feedback information acquired in step S6 are uploaded to the cloud through an edge-cloud communication and interaction module. Correspondingly, the DSP parameter configuration generated in step S4 is also distributed from the cloud to the edge device through this communication module. This clearly defined architecture balances real-time performance with the complexity of intelligent decision-making. It avoids the high bandwidth consumption and privacy risks associated with uploading large amounts of raw audio data to the cloud, transmitting only lightweight feature and parameter data. This ensures a smooth user experience while fully utilizing the powerful computing capabilities of the cloud, enabling advanced intelligence that was previously difficult to achieve on edge devices.

[0038] Please see Figure 2 This application also provides a dynamic optimization system for the vocal effects chain, which serves as a hardware or software implementation of the aforementioned method. The system includes: a real-time audio processing module on the edge, a cloud-based large-scale model tuning decision module, an edge-cloud communication and interaction module, and a feedback acquisition module. These modules work together to fully realize a closed-loop process from perception, decision-making, execution to optimization.

[0039] Specifically, the edge-side real-time audio processing module is the core execution unit deployed on the user device, responsible for handling all tasks requiring low-latency response. Its main functions include: acquiring the user's real-time human voice audio signal and performing preprocessing such as noise reduction; extracting structured timbre feature vectors from the audio; running a lightweight timbre classification model to obtain timbre classification results; and most importantly, it has a built-in complete DSP effects chain for real-time rendering and processing of the audio signal based on parameter configurations received from the cloud, ultimately outputting enhanced human voice.

[0040] The cloud-based large-scale model tuning decision module is the core decision module of the system, deployed on a remote server. It is responsible for all complex intelligent calculations. Its functions include: receiving timbre features, timbre classification results, and users' natural language tuning commands uploaded from the client side; using its embedded large language model to perform comprehensive reasoning on this multimodal information to generate advanced tuning strategies; further mapping the tuning strategies to specific DSP parameter configurations; and continuously and dynamically optimizing and learning its own models (including the large language model and parameter mapping model) based on feedback information sent by the feedback acquisition module.

[0041] The edge-cloud communication and interaction module is used to securely and efficiently transmit data between the real-time audio processing module on the edge and the large-scale model tuning decision module in the cloud. It packages and sends the edge's timbre characteristics, tuning commands, and feedback information to the cloud, while simultaneously sending the DSP parameter configurations and interpretable text generated by the cloud back to the edge. By optimizing the communication protocol and data format, this module ensures smooth and uninterrupted collaboration throughout the entire system.

[0042] The feedback acquisition module is specifically responsible for collecting user feedback on the sound tuning effect. It can be a software module integrated into the user interface, used to receive explicit feedback such as user text input, ratings, likes / dislikes, etc. Simultaneously, it can also record user actions in the background, such as playback duration, number of re-recordings, and manual parameter adjustments, to generate implicit feedback. All collected feedback information is formatted and sent to the cloud via the edge-cloud communication and interaction module, providing data support for the system's closed-loop optimization.

[0043] Furthermore, to illustrate its internal structure more clearly, such as Figure 2 As shown, the edge-side real-time audio processing module can be composed of multiple sequentially connected units. First is the audio acquisition and preprocessing unit, which interfaces directly with the hardware microphone, responsible for converting analog sound signals into digital signals and performing preliminary signal cleaning. Its output is sent to the acoustic feature extraction unit, which uses various signal processing algorithms to calculate timbre feature vectors from the audio data. Next, the timbre classification unit receives this vector, embedding a pre-trained timbre classification model to quickly output the timbre category and continuous representation. Finally, the DSP effects chain processing unit is the endpoint of the entire chain. It contains one or more configurable DSP modules (such as equalizers, compressors, etc.) and processes the raw or pre-processed audio stream in real time based on the latest parameters sent from the cloud. This pipelined structure ensures the efficiency and orderliness of edge-side processing.

[0044] Similarly, the cloud-based large-scale model tuning decision module can also be subdivided into three functional units. First is the intent understanding and strategy generation unit, whose core is the embedded large language model. This unit is the starting point for decision-making, responsible for integrating timbre classification results and user commands, understanding the user's true intent, and generating high-level, abstract tuning strategies. After strategy generation, it is passed to the parameter mapping and knowledge base unit. This unit is responsible for "translating" the abstract strategy into machine-executable language. It maps the strategy to a set of specific DSP parameter configurations by querying the internally stored DSP module knowledge base. Finally, the feedback learning and optimization unit is key to the system's self-evolution. It receives and analyzes all feedback information from the user and uses this information to update and fine-tune the large language model in the intent understanding and strategy generation unit, and / or the mapping rules in the parameter mapping and knowledge base unit. The collaborative work of these three units constitutes the core logic of cloud-based decision-making.

[0045] In one embodiment, the technical solution of the present invention can be applied to a smart audio device, such as a smart microphone. The microphone hardware integrates a high-performance DSP chip and a low-power AI coprocessor, forming a complete end-side real-time audio processing module. It wirelessly connects to the user's smartphone app via Bluetooth 5.2 technology. This smartphone app not only provides a user interface (such as…) Figure 4 As shown, the microphone allows users to input tuning commands via voice or text, and also functions as an end-to-end cloud communication and interaction module, as well as a feedback acquisition module. The system's cloud-based large-scale model tuning decision module is deployed on the company's own cloud computing platform. When a user receives the microphone and completes pairing with the mobile app, they simply say into the microphone, "My singing is prone to cracking, please tune it for me." The app converts this voice command into text, along with the user's timbre feature vector extracted by the microphone in real time, and sends it to the cloud. After cloud-based LLM analysis, it determines that the "cracking" problem is related to an excessively large dynamic range, generating a tuning strategy of "applying appropriate compression to control peak levels." This strategy is mapped to specific parameters of the compressor module (such as a threshold of -10dB and a compression ratio of 3:1) and sent to the microphone. The microphone's DSP chip immediately applies these parameters, and when the user sings again, the dynamics of the sound are effectively controlled, the cracking problem is significantly improved, and the app interface displays the message: "Dynamic compression has been activated for you to protect your voice from overload, making singing easier." This process fully demonstrates the practical application of the invention.

[0046] Finally, this application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it can implement the steps described in any of the foregoing method embodiments. The computer-readable storage medium can be any electronic storage device capable of storing program code, such as a server hard drive, cloud storage, USB flash drive, read-only memory (ROM), random access memory (RAM), etc. When software or firmware containing the logic of the method of the present invention is loaded onto the processor of a general-purpose or special-purpose computing device (such as a smartphone, server, DSP chip) and executed, the device becomes a specific device capable of executing the method of the present invention, thereby achieving dynamic, intelligent, and closed-loop optimization of the vocal effects chain.

Claims

1. A method for dynamic optimization of vocal effect chains, characterized in that, Includes the following steps: S1. Collect the user's real-time human voice audio signal, and preprocess the audio signal and extract acoustic features to obtain a structured timbre feature vector; S2. Input the timbre feature vector into the timbre classification model to obtain the user's timbre classification result and continuous feature representation; S3. Obtain the user's natural language tuning instructions, and combine them with the timbre classification results and historical tuning status information, input them into the large language model for joint reasoning, and generate a tuning strategy for the user's current audio. S4. Map the tuning strategy to specific digital signal processing parameter configurations, and send the DSP parameter configurations to the digital signal processing effects chain on the edge. S5. Using the DSP effects chain, process the real-time human voice audio signal according to the issued parameter configuration, and output the processed audio signal. S6. Collect user feedback on the output audio; S7. Based on the feedback information, dynamically optimize the pitch strategy generation capability of the large language model and / or the mapping relationship between the pitch strategy and DSP parameter configuration to form a closed-loop optimization mechanism.

2. The method according to claim 1, characterized in that, In step S3, when the large language model generates a tuning strategy, it also generates an interpretable tuning description text; and / or, when the intent of the natural language tuning instruction is unclear, the tuning suggestion guidance module is activated to guide the user to clarify the tuning goal.

3. The method according to claim 1 or 2, characterized in that, In step S3, the reasoning process of the large language model for the tuning strategy includes: based on the natural language tuning instructions, determining one or more DSP effect modules that need to be adjusted through a modular decomposition mechanism, and generating parameter adjustment tasks for each DSP effect module that needs to be adjusted.

4. The method according to claim 3, characterized in that, In step S4, mapping the tuning strategy to specific DSP parameter configurations includes: querying the corresponding DSP module knowledge base for the parameter adjustment task of each DSP effect module, and generating specific parameter values ​​adapted to the current device and scene; wherein, the DSP effect module includes at least one of equalizer, compressor, reverb and deschisor.

5. The method according to claim 1, characterized in that, In step S6, the feedback information includes at least one of the following: text feedback directly input by the user, implicit feedback generated based on the user's operation behavior on the output audio, and historical records of the user's manual adjustment of DSP parameters.

6. The method according to claim 1, characterized in that, The acquisition steps in steps S1 to S5 and S6 are executed on the end device, the mapping steps in steps S3 and S4 and S7 are executed on the cloud server, the timbre feature vector obtained in step S1 and the feedback information acquired in step S6 are uploaded to the cloud through the communication module, and the DSP parameter configuration generated in step S4 is sent to the end device through the communication module.

7. A system for dynamically optimizing the vocal effect chain to implement the method of any one of claims 1-6, characterized in that, include: The edge-side real-time audio processing module is used to acquire and preprocess real-time human voice audio signals, extract timbre features, run a timbre classification model, and run a DSP effects chain to process audio. The cloud-based large model tuning decision module is used to receive the timbre features and user tuning instructions, generate tuning strategies and DSP parameter configurations through a large language model, and perform dynamic optimization based on feedback information. The edge-cloud communication and interaction module is used to transmit timbre characteristics, tuning instructions, DSP parameter configurations and feedback information between the real-time audio processing module on the edge side and the large-scale model tuning decision module in the cloud. The feedback acquisition module is used to collect user feedback information on the output audio and send the feedback information to the cloud-based large model tuning decision module.

8. The system according to claim 7, characterized in that, The end-side real-time audio processing module includes the following sequentially connected components: The audio acquisition and preprocessing unit is used to acquire and preprocess audio signals. The acoustic feature extraction unit is used to extract timbre feature vectors from the preprocessed audio signal; The timbre classification unit, which embeds the timbre classification model, is used to classify the timbre feature vector; The DSP effects chain processing unit is used to process audio signals in real time according to the received DSP parameter configuration.

9. The system according to claim 7 or 8, characterized in that, The cloud-based large-scale model tuning decision module includes: The intent understanding and strategy generation unit, which embeds the large language model, is used to generate tuning strategies based on timbre classification results and user commands. The parameter mapping and knowledge base unit is used to map the tuning strategy to specific DSP parameter configurations and stores the DSP module knowledge base. The feedback learning and optimization unit is used to update the intent understanding and strategy generation unit and / or the parameter mapping and knowledge base unit based on feedback information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images