Audio processing method and device, computer device and storage medium

Abstract

The application relates to an audio processing method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring an audio signal to be processed and an index value expected range of the audio signal under at least one quality index, and performing quality detection on the audio signal according to the quality index to obtain an index value corresponding to the quality index of the audio signal; in the case that the audio signal satisfies an optimization condition based on the index value and the index value expected range, constructing prompt information according to the index value, the index value expected range and tool calling guide information adapted to the quality index, so as to drive a large language model to call a skill tool adapted to the quality index, determine an adjustment value for the audio signal, and finally adjust the audio signal according to the adjustment value to obtain an optimized audio signal. The method can improve the audio processing effect.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an audio processing method, apparatus, computer device, and storage medium. Background Technology

[0002] With the rapid development of computer technology, speech-to-text technology has been widely applied in scenarios such as meeting minutes, caption generation, intelligent customer service, and voice assistants. The accuracy of speech-to-text recognition directly affects the user experience and business value of downstream applications. However, in practical applications, the inconsistent quality of input audio has become a major factor affecting recognition accuracy.

[0003] Traditional technologies typically rely on users' subjective descriptions of optimization needs for audio adjustment, or adjust audio according to fixed preset modes. These methods can easily lead to a disconnect between the audio processing results and the actual signal state, resulting in unstable audio processing effects and failing to consistently improve the recognition performance of speech-to-text systems. Summary of the Invention

[0004] Therefore, it is necessary to provide an audio processing method, apparatus, computer device, and storage medium that can improve the audio processing effect in response to the above-mentioned technical problems.

[0005] Firstly, this application provides an audio processing method. The method includes:

[0006] Obtain the audio signal to be processed, and the expected range of the audio signal's index value under at least one quality index;

[0007] The audio signal is subjected to quality detection according to the quality index to obtain the index value of the audio signal corresponding to the quality index;

[0008] If the audio signal meets the optimization conditions based on the indicator value and the expected range of the indicator value, a prompt message is constructed according to the indicator value, the expected range of the indicator value, and tool call guidance information adapted to the quality indicator.

[0009] Guided by the prompts, the large language model invokes skill tools adapted to the quality indicators to determine the adjustment value for the audio signal.

[0010] The audio signal is adjusted according to the adjustment value to obtain an optimized audio signal.

[0011] Secondly, this application also provides an audio processing apparatus. The apparatus includes:

[0012] The acquisition module is used to acquire the audio signal to be processed, and the expected range of the index value of the audio signal under at least one quality index.

[0013] The detection module is used to perform quality detection on the audio signal according to the quality index, and obtain the index value of the audio signal corresponding to the quality index;

[0014] The prompt information construction module is used to construct prompt information based on the indicator value, the expected range of the indicator value, and tool call guidance information adapted to the quality indicator when it is determined that the audio signal meets the optimization conditions based on the indicator value and the expected range of the indicator value.

[0015] The adjustment value determination module is used to determine the adjustment value for the audio signal by calling a skill tool adapted to the quality index under the guidance of the prompt information through a large language model.

[0016] An audio optimization module is used to adjust the audio signal according to the adjustment value to obtain an optimized audio signal.

[0017] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the aforementioned audio processing method.

[0018] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the above-described audio processing method.

[0019] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the aforementioned audio processing method.

[0020] The aforementioned audio processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire the audio signal to be processed and the expected range of the audio signal's index value under at least one quality index. They then perform quality detection on the audio signal according to the quality index to obtain the index value corresponding to the quality index. Based on the actual detected value of the audio signal and its corresponding expected range, they can proactively determine whether the audio signal meets the optimization conditions, thereby avoiding blind adjustments and allowing computing resources to be allocated to the processing of audio signals that truly need optimization, thus improving audio processing performance. Furthermore, when the audio signal is determined to meet the optimization conditions based on the index value and the expected range of the index value, prompt information is constructed according to the index value, the expected range of the index value, and tool invocation guidance information adapted to the quality index. This prompts the large language model to invoke skill tools adapted to the quality index, determine the adjustment value for the audio signal, and finally adjust the audio signal according to the adjustment value to obtain the optimized audio signal. The above process achieves automatic perception and adaptive adjustment of the audio signal's inherent quality, addressing the problem of discrepancies between audio processing results and the actual signal state inherent in traditional technologies. This results in a stable improvement in the quality of the optimized audio, providing a more reliable recognition foundation for downstream applications such as speech-to-text conversion. Therefore, this method can enhance audio processing performance. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a diagram illustrating the application environment of the audio processing methods in some embodiments;

[0023] Figure 2 This is a flowchart illustrating the audio processing method in some embodiments;

[0024] Figure 3 This is a schematic diagram of the overall loudness detection process in some embodiments;

[0025] Figure 4 This is a schematic diagram illustrating the optimization requirements under different acoustic modes in some embodiments;

[0026] Figure 5 This is a schematic diagram of the prompt word template in some embodiments;

[0027] Figure 6 This is a schematic diagram illustrating the optimization process for multiple quality indicators in some embodiments;

[0028] Figure 7 This is a schematic diagram illustrating the optimization process for multiple quality indicators in other embodiments;

[0029] Figure 8 This is a schematic diagram of the audio signal compression process in some embodiments;

[0030] Figure 9 This is a schematic diagram of the segmented loudness detection process in some embodiments;

[0031] Figure 10 This is a schematic diagram illustrating the adaptive adjustment process of the loudness detection method in some embodiments;

[0032] Figure 11 This is a schematic diagram of the audio segment splicing process in some embodiments;

[0033] Figure 12 This is a schematic diagram of the audio segment splicing process in some other embodiments;

[0034] Figure 13 This is a schematic diagram illustrating the process of determining audio adjustment parameters in groups in some embodiments;

[0035] Figure 14 The diagram illustrates the input and output of a large language model in some embodiments.

[0036] Figure 15 This is a schematic diagram of the self-learning optimization mechanism of the large language model in some embodiments;

[0037] Figure 16 These are schematic diagrams of audio processing systems in some embodiments;

[0038] Figure 17 This is a schematic diagram of the logical layered architecture of the audio processing system in some embodiments;

[0039] Figure 18 This is a flowchart illustrating the audio processing method in some other embodiments;

[0040] Figure 19 This is a schematic diagram of the closed-loop optimization process driven by the intelligent agent in some embodiments;

[0041] Figure 20 This is a schematic diagram illustrating the interaction timing between modules of an agent in a complete audio loudness optimization task in some embodiments;

[0042] Figure 21 This is a structural block diagram of the audio processing device in some embodiments;

[0043] Figure 22 These are internal structural diagrams of the computer device in some embodiments;

[0044] Figure 23 This is an internal structural diagram of a computer device in some other embodiments. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0047] The audio processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the application scenario may include a terminal 100 and a server 200. The terminal 100 and server 200 can communicate via a network, such as a wired or wireless network. A data storage system can store the data that the server 200 needs to process. The data storage system can be configured independently, integrated into the server 200, or located in the cloud or on other servers.

[0048] Optionally, terminal 100 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. A client application for the target application can be installed and run on terminal 100. This target application can be a smart audio optimization application or a tuning application specifically supporting audio processing, or it can be other applications providing tuning functions; this application does not limit its scope. Furthermore, this application does not limit the form of the target application; it can be a parent application running on an operating system, or a sub-application running within a parent application, such as a mini-program, or it can be a webpage.

[0049] Optionally, the client can utilize the microphone, audio playback channel, and computing resources of terminal 100 to achieve real-time acquisition, processing, and optimization of the input audio signal, and output the processed audio signal to the speech-to-text engine or other downstream modules. For example, in an enterprise meeting scenario, terminal 100 can be an audio recording device deployed in the meeting room, recording participants' speeches through the microphone of this device to obtain the audio signal to be processed. In a social application scenario, terminal 100 can be a user's handheld terminal, and the social application client can utilize the microphone of this handheld terminal to acquire voice information and obtain the audio signal to be processed.

[0050] Optionally, server 200 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. Optionally, server 200 can be a backend server for the aforementioned target application, used to provide backend services for the target application.

[0051] The audio processing method provided in this application embodiment can be executed by a computer device, which refers to an electronic device with data computing, processing, and storage capabilities. Figure 1 Taking the application environment shown as an example, the audio processing method can be executed by the terminal 100 alone (such as by the target application installed and running in the terminal 100), or by the server 200 alone, or by the terminal 100 and the server 200 interacting and cooperating to execute it. This application does not limit this.

[0052] In one exemplary embodiment, such as Figure 1 As shown, server 200 can acquire the audio signal to be processed from terminal 100. This audio signal can be acquired by the user through their own terminal 100. Server 200 can also acquire the expected range of the audio signal's index value under at least one quality index. This quality index and its corresponding expected range can be provided by the user or adapted by server 200 based on the acoustic mode corresponding to the audio signal. Then, server 200 can perform quality detection on the audio signal according to the quality index to obtain the index value corresponding to the quality index. For example, if the quality index is loudness, loudness detection can be performed on the audio signal to obtain its loudness value; if the quality index is signal-to-noise ratio (SNR), SNR detection can be performed on the audio signal to obtain its SNR. Next, server 200 can determine whether the audio signal meets the optimization conditions based on the index value and its expected range. If the audio signal does not meet the optimization conditions, it can be left unprocessed and directly output to downstream applications such as speech-to-text conversion.

[0053] Optionally, if the audio signal meets the optimization conditions, the server 200 can construct prompt information based on the index value, the expected range of the index value, and the tool invocation guidance information adapted to the quality index. This prompt information then drives the large language model to invoke the skill tool adapted to the quality index to determine the adjustment values ​​for the audio signal. For example, the skill tool adapted to the loudness index is a dynamic compression processing tool, used to determine the adjustment values ​​for multiple adjustment items such as threshold, compression ratio, attack time, release time, compensation gain, and inflection point; the skill tool adapted to the signal-to-noise ratio index is a noise reduction tool, used to determine the adjustment values ​​for multiple adjustment items such as noise reduction amount, sensitivity, and smoothness. Finally, the server 200 can adjust the audio signal according to the adjustment values ​​to obtain the optimized audio signal.

[0054] Optionally, the large language model can be deployed on server 200, or on the cloud or other servers. The embodiments of this application are not limited herein.

[0055] Optionally, if the computing power of terminal 100 meets the requirements, terminal 100 can also acquire the audio signal to be processed and perform quality detection and optimization on the audio signal. In this case, terminal 100 can acquire the audio signal to be processed from other terminals 100, or it can acquire the audio signal uploaded by the user through interaction with the user, or it can autonomously collect the audio signal to be processed through its own deployed microphone. Optionally, the large language model can be deployed on terminal 100 or server 200, and the application of the large language model can be realized through interaction between terminal 100 and server 200.

[0056] In some embodiments, such as Figure 2 As shown, an audio processing method is provided, which can be executed by a computer device, the computer device being... Figure 1 The terminal 100 or server 200 in the application. In this embodiment, the method includes the following steps:

[0057] Step S202: Obtain the audio signal to be processed and the expected range of the index value of the audio signal under at least one quality index.

[0058] Sound is a mechanical wave that propagates through the air. An audio signal is a sound signal that contains information about how its frequency and amplitude change over time. This audio signal can be either an analog signal or a digital signal.

[0059] There are multiple ways to acquire audio signals. Optionally, a microphone can be used to convert ambient sound into electrical signals, thus obtaining the audio signal to be processed. For example, in a corporate meeting setting, audio recording devices deployed in the meeting room can record participants' speech using their built-in or external microphones, thereby acquiring the audio signal. Similarly, in social application scenarios, the microphone on the terminal device where the social application is deployed can be used to collect users' voice information, obtaining the audio signal to be processed.

[0060] Optionally, audio signals can be obtained directly through network communication. For example, in a customer service call scenario, both the customer and the agent can transmit each other's voice signals in real time via the network, and both parties can use the received audio signals as audio signals to be processed; or, in a live streaming scenario, the client can download the live data stream from the server and extract the audio signals from it as audio signals to be processed.

[0061] Quality metrics are measures used to evaluate the quality of audio signals. These metrics can specifically include loudness, equality, and signal-to-noise ratio.

[0062] Different quality indicators are measured in different ways.

[0063] Optionally, loudness can be measured using LUFS (Loudness Units relative to Full Scale), which measures the perceived loudness of audio. Compared to the traditional dBFS peak measurement, LUFS better reflects the non-linear perception of sound loudness by the human ear.

[0064] Optionally, equalization can be measured using the spectral centroid. The spectral centroid describes the "balance point" or "brightness" of an audio signal in the spectrum, and is calculated by taking an amplitude-weighted average of the frequencies in each band. For example, audio rich in high-frequency components (such as cymbal sounds) has a higher spectral centroid and sounds brighter; while audio dominated by low-frequency components (such as bass sounds) has a lower spectral centroid and sounds more muffled.

[0065] Optionally, the signal-to-noise ratio (SNR) can be measured in dB (decibels), representing the ratio of signal power to noise power. A higher value indicates less noise in the signal and clearer audio.

[0066] The expected range of audio signal values ​​under a quality index refers to the expected numerical range of the audio signal's value corresponding to a specific quality index. Here, the index value of the audio signal corresponding to the quality index refers to the measurement value of the audio signal under the measurement method corresponding to the quality index. For the same quality index, the quality of the audio signal under that quality index can be represented by multiple index values. These multiple index values ​​can include at least one of the following: mean, variance, maximum value, minimum value, standard deviation, etc. Correspondingly, the expected range of index values ​​can include the expected numerical ranges corresponding to each of the multiple index values.

[0067] Optionally, the expected range of the index value can be determined based on the application scenario of the audio signal, relevant industry standards, or user-defined. For example, the expected range of the loudness index value for audio signals in broadcast television programs can be "-24LUFS to -22LUFS"; the expected range of the loudness index value for audio signals in corporate meeting scenarios can be "-24LUFS to -14LUFS"; the expected range of the equalization index value for audio signals in audiobooks can be "400Hz to 800Hz"; and the expected range of the signal-to-noise ratio index value for audio signals in daily voice calls can be "≥25dB".

[0068] Step S204: Perform quality detection on the audio signal according to the quality index to obtain the index value of the audio signal corresponding to the quality index.

[0069] Here, the index value corresponding to the quality index of the audio signal refers to the measurement value of the audio signal under the measurement method corresponding to the quality index. For example, the loudness value of the audio signal is the index value of the audio signal corresponding to the loudness index; the signal-to-noise ratio of the audio signal is the index value of the audio signal corresponding to the signal-to-noise ratio index.

[0070] Optionally, the audio signal corresponds to a quality index value, which may include at least one of the following: an overall index value or segmented index values. The overall index value characterizes the comprehensive level of the entire audio signal on a certain quality index. Examples include the integrated LUFS of the entire conference recording, the average spectral centroid of the entire frequency band, or the signal-to-noise ratio of the entire audio recording. Segmented index values ​​characterize the index value of the audio signal within a specific time period, facilitating the detection of local fluctuations, transient anomalies, or trends over time. For example, detecting whether the instantaneous loudness of a speaker is too low, or whether a sudden increase in background noise occurs in a certain segment. Optionally, the specific time period may be, for example, each frame, each sentence, or each fixed duration window.

[0071] Optionally, the computer device can use an audio quality testing tool adapted to the quality index to perform quality testing on the audio signal and obtain the index value of the audio signal corresponding to the quality index.

[0072] Optionally, the computer equipment can perform quality testing on the audio signal according to the measurement method of the quality index to obtain the index value of the audio signal corresponding to the quality index.

[0073] In one exemplary embodiment, quality detection targeting specific quality indicators can be achieved through collaborative interaction between an intelligent agent and an audio quality detection tool.

[0074] In this context, an intelligent agent, also known as an agent, is an intelligent program built on a Large Language Model (LLM) that possesses autonomous decision-making capabilities. It can invoke external tools based on contextual information, execute multi-step tasks, and adaptively adjust based on the execution results. In other words, an intelligent agent can perceive its environment and take actions to achieve a specific goal. It can be software, hardware, or a system, possessing autonomy, adaptability, and interactivity. An intelligent agent can perceive changes in the environment (such as through sensors or data input), make judgments and decisions based on its learned knowledge and algorithms, and then execute actions to influence the environment or achieve predetermined goals.

[0075] Large language models are large-scale pre-trained language models based on the Transformer architecture, possessing natural language understanding, reasoning, and generation capabilities. They can interact with external tools through a function calling mechanism. The function calling mechanism is the technical mechanism by which large language models interact with external tools. The model generates structured tool call instructions based on context, and after the system executes them, the results are returned to the model for subsequent reasoning. As the core engine driving intelligent agents in task planning and decision-making, large language models provide agents with the ability to understand natural language, reason about and decompose complex instructions, and select the tools to call.

[0076] Specifically, the agent can register audio quality detection tools adapted to quality metrics on the large language model. Optionally, the registration information for the audio quality detection tool can include a functional description, input and output formats, and calling permissions. Then, the large language model can activate the detection tool through a function call mechanism to perform quality detection on the audio signal, obtain the metric value corresponding to the quality metric, and feed it back to the agent for subsequent judgment.

[0077] In one exemplary embodiment, such as Figure 3As shown, the quality detection process for loudness indicators can include steps such as weighted filtering (S301), mean square value calculation (S302), and threshold processing (S303). Specifically, the computer device can perform weighted filtering on the audio signal to simulate the differences in human ear's sensitivity to different frequencies, obtaining a filtered signal. Then, the computer device can divide the audio signal into continuous and overlapping short frames, and calculate the mean square value for each frame as the instantaneous loudness (unit: LUFS) of that frame. That is, the instantaneous loudness of each frame is the average of the squares of all sampling points within the frame. Finally, threshold processing is performed. After threshold filtering, the computer device performs energy averaging on the instantaneous loudness of all retained frames to obtain the final integrated loudness, and outputs the indicator value in LUFS units.

[0078] Optionally, the weighted filtering can specifically be K-weighted filtering. K-weighted filtering can optionally include pre-filtering and RLB (Revision of Leq weighted filter) weighted filtering. Pre-filtering is used to simulate the frequency response of the human outer and middle ear, slightly boosting the frequency band from 1kHz to 3kHz; RLB weighted filtering is used to attenuate low frequencies (e.g., <100 Hz) and moderately boost high frequencies (e.g., 2kHz to 4kHz) to comprehensively simulate the differences in perceptual sensitivity of the human ear at different frequencies.

[0079] Optionally, the frame length of a short frame can be, for example, 0.3 seconds, 0.4 seconds, or 0.5 seconds, and is not limited here.

[0080] Optionally, for multi-channel audio, the computer device can perform a weighted summation of the mean square values ​​of each channel to obtain the mean square value calculation result. Taking stereo as an example, the weights of the left and right channels can both be 1; if there is a center channel, its weight can also be 1, and the weights of the left and right surround channels can be 1.41 to simulate the human ear's perception of the superposition of sound loudness from different directions.

[0081] Optionally, the threshold values ​​used in the thresholding process can include absolute thresholds and relative thresholds. Specifically, the computer device can first exclude frames with instantaneous loudness below the absolute threshold (typically silent or extremely low-level segments), as these segments are not included in the final loudness calculation. Then, a preliminary average loudness value is calculated for the remaining frames, and frames with instantaneous loudness significantly lower than the average loudness (i.e., low-energy segments far below the average loudness) are again excluded. Optionally, the absolute threshold could be, for example, -70 LUFS. Optionally, the relative threshold could be, for example, 10 LU below the average.

[0082] In an exemplary embodiment, the computer device can segment the audio signal into frames and window them, then perform a Fast Fourier Transform on each frame to obtain the spectral distribution of that frame. Next, for each frame, the computer device can calculate the spectral centroid of that frame and use the statistical value of the spectral centroids of all frames in the entire audio signal as an indicator of overall equalization. Optionally, this statistical value can be an average value or a time-weighted sum.

[0083] Step S206: If the audio signal meets the optimization conditions based on the index value and the expected range of the index value, construct a prompt message according to the index value, the expected range of the index value, and the tool call guidance information that is compatible with the quality index.

[0084] In this context, the audio signal meeting the optimization condition can mean that the audio signal's index value exceeds the expected range of index values. In an optional embodiment, the audio signal meeting the optimization condition can mean that the overall index value of the audio signal is within the expected range of index values, but the duration of the audio signal exceeds a duration threshold. Since audio signals exceeding the duration threshold have a higher probability of sudden changes, introducing the audio duration criterion into the optimization condition can ensure the accuracy of the judgment result and is conducive to further improving the audio processing effect.

[0085] A prompt is a text, instruction, or question inputted when interacting with a large language model to guide, instruct, or trigger the model to generate specific outputs. The core function of a prompt is to clarify the task objective, provide contextual constraints, and set the output format or style, thereby helping the model to more accurately and reliably generate the expected content. Examples of prompts include "Please translate the following sentence into English," "Write an apology email about the project delay," or "Summarize the central idea of ​​this article in one sentence."

[0086] Specifically, the computer device can determine whether an audio signal meets optimization conditions based on the indicator value and the expected range of the indicator value. If the audio signal does not meet the optimization conditions, the computer device can choose not to process the audio signal and directly output it to downstream applications such as speech-to-text conversion. If the audio signal meets the optimization conditions, the computer device can construct a prompt message based on the indicator value of the audio signal corresponding to the quality indicator, the expected range of the indicator value of the audio signal under the quality indicator, and the tool call guidance information adapted to the quality indicator.

[0087] Optionally, the computer device can determine system instructions representing the audio signal optimization task, and obtain prompt information by concatenating the system prompt, the index value of the audio signal corresponding to the quality index, the expected range of the index value of the audio signal under the quality index, and tool invocation guidance information adapted to the quality index. Taking the loudness index as an example, the system prompt could be something like, "You are a professional audio processing engineer, skilled in optimizing audio loudness using a dynamic range compressor. You need to generate the optimal combination of compressor parameters based on the audio loudness detection data to achieve the target loudness range while maintaining natural sound quality."

[0088] In an optional embodiment, step S206 includes: determining the acoustic mode corresponding to the audio signal when the audio signal meets the optimization conditions based on the index value and the expected range of the index value; obtaining a prompt word template that matches the acoustic mode under the quality index; and filling the variable placeholders of the prompt word template with the index value and the expected range of the index value to obtain the prompt information.

[0089] Among them, acoustic modes are used to characterize the foreground sound emission mode and background noise intensity of audio signals.

[0090] Foreground and background sounds are two fundamental concepts used to distinguish the primary and secondary relationships of sounds. Foreground sounds refer to the most prominent and attention-grabbing sounds in an audio scene, typically the core auditory object that the listener consciously or unconsciously focuses on. Examples of foreground sounds include the speaker's voice in a dialogue recording, the lead vocals or main instrument in a musical piece, or dialogue or key sound effects in a film. Background sounds, on the other hand, are secondary sounds in an audio scene that do not occupy the central attention, providing atmosphere, spatiality, or continuity for the foreground sounds. Examples of background sounds include the noise of an air conditioner or projector fan in a meeting recording, accompanying instruments in music, or the sounds of traffic, wind, and rain in outdoor recordings.

[0091] Foreground sound generation refers to the specific sound pattern presented by foreground sounds in an audio signal, which is usually characterized by the sound-producing object and its interaction. This foreground sound generation includes, but is not limited to: solo speech (a single speaker speaking continuously), multi-person dialogue (two or more speakers speaking alternately or overlapping), and speech with background music (such as a speech or song with musical accompaniment). Background noise intensity refers to the energy intensity of background noise in an audio signal. In practical applications, background noise intensity can be divided into several levels, such as high, medium, and low.

[0092] Specifically, when the audio signal meets the optimization conditions based on the index value and the expected range of the index value, the computer device can determine the acoustic mode corresponding to the audio signal.

[0093] Optionally, the computer device can use a trained audio classification model to identify the acoustic patterns of the audio signal. This audio classification model could be, for example, PANNs (Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition) or an AudioSet classifier.

[0094] Optionally, the computer device can, for example, support multimodal large models of audio input to directly understand the audio content and identify the acoustic patterns corresponding to the audio signals.

[0095] Optionally, the computer device can input metadata information such as audio source, file name, and associated business tags into the large language model, which will then infer possible acoustic patterns.

[0096] Optionally, computer devices can also determine the acoustic pattern of an audio signal by statistically analyzing its acoustic characteristics and using rule matching. These acoustic characteristics may include, for example, short-time energy, zero-crossing rate, spectral flatness, or the proportion of low-frequency energy.

[0097] For example, when the short-term energy is moderate and fluctuates little, the zero-crossing rate is stable between 20 and 80 Hz, the spectral flatness is below 0.2, and the low-frequency energy proportion is within the range of 0.3 to 0.5, the acoustic mode of the audio signal can be determined to be a single-person speech. When the short-term energy fluctuates significantly (e.g., the ratio of standard deviation to mean is greater than 0.5), the zero-crossing rate is still within the range of 20 to 80 Hz but changes intermittently, the spectral flatness is below 0.2, and the low-frequency energy proportion is 0.3 to 0.5, the acoustic mode of the audio signal can be determined to be a multi-person dialogue. When the short-term energy is high, the zero-crossing rate is between 30 and 100 Hz, the spectral flatness is between 0.2 and 0.5, and the low-frequency energy proportion is greater than 0.5, the acoustic mode of the audio signal can be determined to be a speech with background music. When the short-term energy is low or moderate but without significant fluctuations, the zero-crossing rate is above 100 Hz, the spectral flatness is greater than 0.6, and the low-frequency energy proportion has no obvious pattern, the acoustic mode of the audio signal can be determined to be a noisy environment.

[0098] Different foreground sound emission methods and background noise intensities have varying impacts and requirements on tasks such as audio acquisition, processing, and speech recognition. Based on this, computer equipment can configure different prompt word templates for different acoustic modes while maintaining the same quality index. A prompt word template refers to a pre-designed prompt message format that includes a fixed structure and variable placeholders.

[0099] Optionally, the fixed content of the prompt template may include tool invocation guidance information adapted to the quality metric. For example, for the signal-to-noise ratio metric, the tool invocation guidance information could be "Generate the optimal noise reduction parameters" or "Please generate noise reduction parameters by calling the tool."

[0100] Optionally, the cue word template may include audio optimization requirements for that acoustic mode in its fixed content.

[0101] Taking the balance as a quality indicator as an example. Optionally, in a single-person speech scenario, attention should be paid to the spectral balance within the audio frequency range (usually 300Hz to 3400Hz) to avoid excessive energy concentration in low or high frequencies, ensuring a natural and clear timbre. In a multi-person dialogue scenario, attention should be paid to the consistency of the balance among different speakers to avoid a decrease in the recognition rate of certain speakers by the transcription system due to differences in individual timbre. If necessary, spectral normalization processing can be performed on each speaker's speech. In a speech scenario with background music, attention should be paid to whether the audio frequency range is masked by the energy of the music, especially the vocal frequency range (200Hz to 4kHz) in the music should not have obvious spectral peaks covering the speech.

[0102] Taking the case where the quality index is used as the loudness index as an example. For example... Figure 4 As shown, optionally, in a single-person speech scenario, attention should be paid to the overall loudness uniformity to avoid loudness fluctuations due to changes in speaker distance or volume; in a multi-person dialogue scenario, attention should be paid to the loudness balance of each speaker to ensure that the loudness of each speaker's voice is basically consistent, and to avoid information loss due to someone's voice being too soft or covered; in a speech scenario with background music, attention should be paid to the loudness of the speech frequency band to ensure that the loudness of the frequency band where the speech is located is significantly higher than that of the background music in order to ensure speech intelligibility.

[0103] Optionally, in noisy environments where the background noise intensity is high, the audio optimization requirement can be met by first reducing noise and then optimizing other quality indicators.

[0104] Optionally, given a defined quality metric and acoustic mode, the computer device can obtain a prompt word template matching the acoustic mode under that quality metric. This prompt word template contains fixed content that defines the tool call guidance information for that quality metric and the audio optimization requirements for that quality metric under the corresponding acoustic mode. Then, the computer device can fill the variable placeholders in the prompt word template with personalized information related to a single audio signal, such as the metric value of the audio signal corresponding to the quality metric and the expected range of the metric value for that audio signal under that quality metric, to obtain the prompt information.

[0105] In the above embodiments, different prompt word templates are matched for different acoustic modes, and prompt words are constructed on this basis. The processing strategy of the large language model can be adaptively adjusted according to the specific content of the audio signal, which can ensure the fit between the adjustment value determined by the large language model and the actual audio content, thereby improving the audio processing effect.

[0106] In an exemplary embodiment, the prompt template includes system prompt content, input content, and tool invocation guidance content. The system prompt content includes audio optimization requirements; the tool invocation guidance content includes tool invocation guidance information; and the input content includes variable placeholders. In this embodiment, the prompt information is obtained by filling the variable placeholders with indicator values ​​and expected ranges of indicator values.

[0107] For example, such as Figure 5 As shown, the prompt template 500 includes system prompt content 510, input content 520, and tool invocation guidance content 530. The system prompt content 510 includes audio optimization requirements for quality indicators under a specific acoustic mode, and the tool invocation guidance content 530 includes tool invocation guidance information adapted to the quality indicators. See also... Figure 5 The system prompt 510 includes the audio optimization requirements for the loudness index in the single-person speech acoustic mode, namely "to make the overall loudness uniform". The tool call guidance content 530 includes tool call guidance information adapted to the loudness index, namely "please analyze the loudness characteristics and generate compressor parameters through tool call".

[0108] Optionally, the input content 520 includes variable placeholders 521 and 522, wherein the variable placeholder 521 is used to fill in the loudness value of the audio signal. The loudness value may include an overall loudness value and segmented loudness values. The variable placeholder 522 is used to fill in the expected range of the audio signal's index value under this loudness index.

[0109] Optionally, the input content 520 may also include variable placeholders corresponding to audio information and historical processing records, to fill in personalized information such as audio duration, sampling rate, and number of channels of the audio signal, as well as historical processing records for the audio signal.

[0110] Optionally, the prompt word template can include information such as role setting, context injection, constraints, output format specifications, and inference chain guidance. Taking loudness as the quality metric, for example, role setting allows the large language model to be configured as a professional audio engineer with extensive audio processing experience and compressor parameter adjustment knowledge; context injection allows loudness detection results, audio information, and expected loudness range to be injected into the context in the form of structured data; constraints can be used to clarify the reasonable value range and physical meaning constraints of each adjustment item, preventing the large language model from generating unreasonable adjustment values; output format specifications can be defined through the data pattern of function calls to ensure that the large language model outputs structured parameter JSON objects; inference chain guidance can be used to guide the large language model to first analyze loudness features, then determine the processing strategy, and finally generate specific parameters, forming a clear chain of thought.

[0111] In the above embodiments, fixed content is embedded in the system prompts and tool call guidance content, and variable placeholders are set in the input content to fill in personalized information such as the index value of the audio signal corresponding to the quality index and the expected range of the index value of the audio signal under the quality index. This can ensure the reusability of the prompt word template while meeting the audio optimization requirements, which is conducive to improving audio processing efficiency.

[0112] Step S208: Guided by prompts, the large language model invokes skill tools that are compatible with the quality indicators to determine the adjustment value for the audio signal.

[0113] The large language model can activate skill tools through function calling, obtain corresponding results, and feed them back to the agent for subsequent processing. A skill tool is a structured declaration describing the metadata of an externally callable function (such as an application programming interface, local function, database query, etc.). It includes at least core elements such as tool name, function description, and parameter pattern. The tool name uniquely identifies the skill tool; the function description, usually in natural language, helps the large language model determine when and why to use the skill tool; the parameter pattern defines the type, attributes, constraints, and required fields of the input parameters.

[0114] Specifically, a set of skill tools can be pre-registered for the large language model, each tool being adapted to a specific audio quality metric. During application, prompts containing tool invocation guidance information can be input into the large language model. The large language model can parse the tool invocation guidance information, autonomously decide and invoke the skill tool adapted to the current quality metric, and then generate adjustment values ​​for the audio signal.

[0115] Among them, the adjustment values ​​are used to optimize the actual performance of the audio signal on the corresponding quality indicators. For example, the adjustment values ​​for the loudness indicator may include the individual adjustment values ​​of each adjustment item such as threshold, compression ratio, attack time, release time, compensation gain, and inflection point; the adjustment values ​​for the signal-to-noise ratio indicator may include the individual adjustment values ​​of each adjustment item such as noise gate threshold, attenuation, noise reduction, compensation gain, and inflection point; and the adjustment values ​​for the equalization indicator may include the individual adjustment values ​​of each adjustment item such as frequency point, gain, bandwidth, and compensation gain.

[0116] In some embodiments, the audio processing method further includes: determining an adjustment method for a quality indicator and multiple adjustment items that affect the quality indicator under the adjustment method; and registering skill tools adapted to the quality indicator based on each adjustment item.

[0117] Among these, the adjustment methods for quality indicators refer to methods that use specific signal processing algorithms or operations to directionally modify the audio signal, thereby causing the audio signal's value corresponding to the quality indicator to approach the desired state from its current state. For example, the adjustment method for loudness is a compressor, the adjustment method for signal-to-noise ratio is noise reduction, and the adjustment method for equalization is an equalizer.

[0118] The multiple adjustment items affecting quality indicators under an adjustment method refer to a set of specific parameters that can be independently adjusted by the system in the signal processing algorithm or operation corresponding to that adjustment method. Each adjustment item controls the characteristics of the processing behavior in a certain dimension, and all adjustment items together determine the actual adjustment effect of the adjustment method on the target quality indicator. For example, for a loudness compressor, its adjustment items may include threshold, compression ratio, attack time, release time, compensation gain, inflection point, etc.; for signal-to-noise ratio noise reduction processing, adjustment items may include noise gate threshold, noise reduction depth, attack time, release time, frequency smoothing coefficient, etc.; for equalizers for equalization, adjustment items may include center frequency, gain value, bandwidth, filter type, etc. for each frequency band.

[0119] Taking loudness as an example. Optionally, the compressor can specifically be a dynamic range compressor (VRAM) used to reduce the dynamic range of the audio signal. When the input signal exceeds a set threshold, the compressor reduces the signal gain according to a set compression ratio, thereby making the audio that is too loud or too soft tend to be uniform. The threshold is the loudness critical point at which the compressor starts to work; when the loudness of the audio signal exceeds this value, the compressor begins to compress the signal. The unit of the threshold is dB. The compression ratio is the proportion of the signal that the compressor attenuates beyond the threshold. For example, a compression ratio of 4:1 means that for every 4dB increase in input, the output only increases by 1dB. Attack time is the time required for the compressor to fully apply compression from detecting that the signal exceeds the threshold, measured in milliseconds (ms). A shorter attack time allows for a faster response to transient signals. Release time is the time required for the compressor to completely stop compression from detecting that the signal has fallen back below the threshold, measured in milliseconds (ms). Makeup gain is the additional gain applied after compression to compensate for the overall loudness reduction caused by compression, measured in dB. The knee describes how smoothly the compressor transitions near the threshold. A hard knee abruptly begins compression at the threshold; a soft knee transitions gradually near the threshold, resulting in a more natural processing effect.

[0120] The output of the skill tool includes the adjustment value corresponding to each adjustment item.

[0121] Specifically, the computer device can determine the adjustment method for the quality indicator and the multiple adjustment items affecting the quality indicator under that method. Then, the adjustment methods for these items are encapsulated into a unified skill tool, and following the protocol specifications of the large language model, the adjustment value corresponding to each item is defined as the output of the skill tool, ultimately completing the registration of the skill tool. Furthermore, during application, the large language model can generate structured tool call instructions through a function call mechanism, which are parsed and executed by the agent scheduling layer. The execution result is returned to the large language model in a structured format for subsequent inference.

[0122] In the above embodiments, complex audio processing parameters are standardized into callable skill tools, enabling large language models to uniformly and efficiently control the adjustment process of quality indicators, which is beneficial to improving the scalability of the system.

[0123] In some embodiments, there are multiple quality indicators; the skill tools adapted to each quality indicator are invoked by the large language model in the order of invocation.

[0124] In an optional embodiment, when there are multiple quality metrics to be optimized, the order in which the appropriate tools for each quality metric are called can be determined based on the dependencies and interference relationships between the metrics. The basic quality metrics that affect other quality metrics are called earlier. For example, since compression changes the frequency domain envelope, dynamic range compression should precede equalization adjustment; that is, the order in which the loudness metric's appropriate tools are called is earlier than the order in which the equalization metric's appropriate tools are called.

[0125] In an optional embodiment, each metric can be pre-assigned a priority, and the appropriate skill tools for each quality metric can be called sequentially in descending order of priority.

[0126] In an optional embodiment, the process of determining the calling order includes: determining the quality assessment granularity of each quality indicator for the audio signal; and determining the calling order of the skill tools that each quality indicator is adapted to in order of quality assessment granularity from coarse to fine.

[0127] In audio signal quality assessment, the granularity of a quality indicator refers to the level of detail in which the indicator is used to analyze, measure, or evaluate the audio signal. This granularity can be represented by at least one dimension, such as time, frequency, or statistical methods. For example, whether the obtained indicator value is the global mean or a short-time frame sequence, a broadband indicator value or a sub-band or frequency point indicator value, or a mean or standard deviation.

[0128] In signal processing, adjustments to coarse-grained metrics often have a wider impact, potentially altering or even destroying the optimization results of fine-grained metrics. Therefore, the order in which the appropriate skill tools for each quality metric are invoked can be determined by prioritizing quality assessment granularity from coarse to fine.

[0129] For example, when quality metrics include loudness, signal-to-noise ratio, equalization, and intelligibility, the corresponding skill tools can be invoked in order of granularity from coarse to fine. For instance, loudness compression can be performed first to control the overall dynamics and energy distribution of the signal; secondly, noise suppression can be performed to reduce background noise, typically in the mid-frequency or broadband range with medium granularity; next, equalization processing can be performed to finely adjust multiple sub-band frequencies with even finer granularity; finally, speech enhancement targeting transient local features can be performed to improve speech intelligibility and detail.

[0130] In an exemplary embodiment, the computer device can perform quality detection on the audio signal according to each quality indicator to obtain the indicator value of the audio signal corresponding to each quality indicator; when each quality indicator meets the optimization conditions, a prompt message containing tool call guidance information adapted to each quality indicator is constructed; under the guidance of the prompt message, the large language model sequentially calls the skill tools adapted to each quality indicator in order of quality assessment granularity from coarse to fine, and determines the adjustment value of the audio signal corresponding to each quality indicator.

[0131] For example, such as Figure 6 As shown, the agent can utilize the function call mechanism of the large language model to perform comprehensive sound quality analysis on the input audio signal by calling various audio quality detection tools. This yields individual values ​​for multiple quality indicators such as loudness, signal-to-noise ratio (SNR), equalization, and intelligibility, resulting in a multi-dimensional sound quality report. Loudness detection provides the loudness value; noise detection determines the noise level, thus aiding in SNR assessment; spectral analysis assesses spectral equalization; and speech activity detection locates speech segments to aid in intelligibility evaluation. Based on the quality assessment granularity of each indicator in this multi-dimensional sound quality report, the agent can design a processing pipeline, determining the calling order of the appropriate skill tools for each quality indicator. Then, the agent sequentially calls each skill tool through the large language model according to the calling order, obtaining corresponding adjustment values ​​and adjusting the audio signal accordingly. Once all skill indicators are optimized, the optimized audio signal is obtained and output to downstream applications such as speech-to-text conversion.

[0132] In an exemplary embodiment, the computer device can use the coarsest quality metric as the current quality metric and perform quality detection, adjustment value generation, and audio optimization on this metric (i.e., sequentially execute steps S204 to S210). Then, it determines the coarsest quality metric from the remaining quality metrics and performs quality detection, adjustment value generation, and audio optimization on this metric (i.e., sequentially execute steps S204 to S210). That is, the audio object for which quality detection is performed on a later quality metric is the audio optimization result for the previous quality metric. This process continues until all quality metrics have been optimized, resulting in an optimized audio signal.

[0133] For example, such as Figure 7As shown, the intelligent agent can perform loudness detection and loudness compression on the input audio signal. If the loudness meets the requirements, it can further perform noise detection and noise suppression. Next, the audio signal with noise suppression is subjected to spectrum analysis and equalization processing. Finally, the equalized audio signal is subjected to speech activity detection and speech enhancement processing to obtain the processed audio signal, which is then output to downstream applications such as speech-to-text conversion.

[0134] In the above embodiments, the order in which the skill tools corresponding to each quality indicator are called is determined according to the order of quality assessment granularity from coarse to fine. This can avoid changing or even destroying the optimization results of fine-grained indicators, which is conducive to improving the overall processing effect of audio signals.

[0135] Step S210: Adjust the audio signal according to the adjustment value to obtain the optimized audio signal.

[0136] Specifically, computer equipment can adjust the audio signal according to the adjustment value to optimize the audio signal in terms of quality indicators, thereby obtaining an optimized audio signal.

[0137] Optionally, the computer device can invoke the skill tools responsible for performing audio optimization through the large language model to adjust the audio signal. For example, for loudness indicators, the audio signal can be dynamically compressed by invoking compression tools based on the adjustment values ​​generated by the large language model. The specific compression process can be encapsulated into the corresponding skill tools through instructions or code.

[0138] For example, adjustment values ​​for loudness metrics include threshold, compression ratio, attack time, release time, inflection point, and compensation gain. Figure 8 As shown, the compressor first calculates the instantaneous loudness L_n of the input audio signal x_n and compares it with a threshold: if L_n does not exceed the threshold, the gain attenuation is 0dB; if it exceeds the threshold, the required gain attenuation is calculated based on the compression ratio. Subsequently, the calculated gain attenuation is smoothed at the inflection point to soften it, and smoothed using time constants such as attack and release times to control the oscillation and release speeds. Next, the smoothed gain attenuation is applied to the original signal, and finally, the overall level is compensated by makeup gain, resulting in an output audio signal y_n with optimized loudness. The formula for calculating the gain attenuation can be G = (threshold + L_n) / (threshold / compression ratio - L_n).

[0139] In the aforementioned audio processing method, the audio signal to be processed and the expected range of the audio signal's index value under at least one quality index are obtained. The audio signal is then subjected to quality detection according to the quality index to obtain the index value corresponding to the quality index. Based on the actual detected value of the audio signal and its corresponding expected range, the method proactively determines whether the audio signal meets the optimization conditions, thus avoiding blind adjustments and allowing computational resources to be allocated to the processing of audio signals that truly need optimization, thereby improving audio processing performance. Furthermore, after determining that the audio signal meets the optimization conditions based on the index value and its expected range, prompts are constructed according to the index value, the expected range, and tool invocation guidance information adapted to the quality index. This prompts the large language model to invoke the skill tools adapted to the quality index, determine the adjustment value for the audio signal, and finally adjust the audio signal according to the adjustment value to obtain the optimized audio signal. This process achieves automatic perception and adaptive adjustment of the audio signal's own quality, addressing the problem of audio processing results being disconnected from the actual signal state in traditional techniques. This steadily improves the quality of the optimized audio, providing a more reliable recognition foundation for downstream applications such as speech-to-text conversion. Therefore, the above method can improve audio processing performance.

[0140] In some embodiments, performing quality detection on an audio signal according to a quality index to obtain an index value corresponding to the quality index includes: performing overall quality detection on the audio signal according to the quality index to obtain an overall index value corresponding to the quality index; if the audio signal does not meet the segmentation conditions, using the overall index value as the index value corresponding to the quality index; if the audio signal meets the segmentation conditions, splitting the audio signal into multiple sub-audio signals; performing segmented quality detection on each sub-audio signal according to the quality index to obtain a segmented index value corresponding to the quality index; and using the overall index value and each segmented index value as the index value corresponding to the quality index.

[0141] The specific methods for overall quality testing are described above. Segmentation conditions can be represented by the relationship between the overall indicator value and the expected range of the indicator value, or by the relationship between the audio duration and the overall indicator value and the expected range of the indicator value.

[0142] Specifically, the computer equipment can perform overall quality detection on the audio signal according to quality indicators, obtain the overall indicator value of the audio signal corresponding to the quality indicators, and then, based on the detection information obtained during the overall quality detection process, determine whether the audio indicators meet the segmentation conditions. Optionally, this detection information may include intermediate information and result information. Taking loudness as an example when the quality indicator is loudness, the intermediate information may include the instantaneous loudness of each frame, and the result information may include the overall loudness.

[0143] In an alternative embodiment, the computer device can determine that the audio signal meets the segmentation criteria if the overall index value of the audio signal exceeds the expected range of index values.

[0144] In an optional embodiment, the computer device can determine that the audio signal meets the segmentation criteria if the duration of the audio signal exceeds a duration threshold and the overall index value of the audio signal exceeds the expected range of index values. The duration threshold may be, for example, 30 seconds, 50 seconds, etc.

[0145] In an optional embodiment, the computer device may determine that the audio signal meets the segmentation conditions if the duration of the audio signal exceeds a duration threshold and the audio signal meets the segmentation difference prediction conditions.

[0146] In audio signal processing, segment difference prediction criteria refer to a set of quantifiable discrimination standards pre-defined based on the local variation patterns of audio signals. These criteria are used to determine whether there are significant non-stationary differences between different segments of the audio signal before the analysis of the entire audio signal is completed, thereby deciding whether segment quality testing should be performed on the audio signal.

[0147] Optionally, the audio signal's segmentation difference prediction criteria can be determined based on anticipated local variation patterns in time, frequency, energy, or statistical characteristics. These local variation patterns can be obtained from overall quality inspection process data; for example, the inter-frame variance obtained during overall quality inspection can be acquired, and if the inter-frame variance is greater than or equal to a threshold, the audio signal is determined to meet the segmentation difference prediction criteria.

[0148] Optionally, the short-time energy and zero-crossing rate of the audio signal can be detected by voice activity. If both exceed twice the variance of the background noise estimate, the audio signal can be determined to meet the segmentation difference prediction condition.

[0149] Optionally, the audio signal can be classified into different scenarios. If the audio signal is a scenario of multiple people speaking, then the audio signal can be determined to meet the segmentation difference prediction conditions.

[0150] In the above embodiments, by combining the audio duration, the overall detection results, and the segment differences predicted for the audio signal, it is determined whether the audio signal needs to be further segmented and segmented for quality detection. This avoids performing indiscriminate comprehensive quality detection on the entire audio segment, thereby significantly improving processing efficiency and reducing computational overhead while ensuring detection accuracy.

[0151] In an optional embodiment, if the audio signal does not meet the segmentation criteria, the computer device uses the overall index value as the index value of the audio signal corresponding to the quality index.

[0152] In an optional embodiment, when the audio signal meets the segmentation conditions, the computer device can split the audio signal into multiple sub-audio signals, and perform segmented quality detection on each sub-audio signal according to the quality index to obtain the segmented index value of the sub-audio signal corresponding to the quality index. Finally, the overall index value and the index values ​​of each segment are used as the index value of the audio signal corresponding to the quality index.

[0153] Optionally, the computer device can divide the audio signal into multiple sub-audio signals according to a time window. The length of this time window can be, for example, 3 seconds by default, and can be adjusted within the range of 1 to 10 seconds.

[0154] Optionally, the computer device can also divide the audio signal into multiple sub-audio signals according to the acoustic feature change points of the audio signal. These acoustic feature change points can be, for example, the local peak positions of parameters such as spectral flux or cepstral distance.

[0155] Optionally, the computer device can also divide the audio signal into multiple sub-audio signals according to the effective speech segment boundaries marked by the speech activity detection marker.

[0156] Optionally, the computer device can also divide the audio signal into multiple audio segments according to pre-trained audio event category boundaries. Audio event category boundaries can be, for example, music, noise, silence, or transitions between different speakers.

[0157] Optionally, the computer device can also divide the audio signal into multiple audio segments based on the moment when the inter-frame feature difference exceeds a dynamic threshold.

[0158] Taking loudness as an example, segmented loudness detection refers to a technique that divides audio into multiple segments along the time dimension and analyzes the loudness of each segment separately. This method can accurately capture the loudness variation characteristics of audio over time. For example... Figure 9As shown, a computer device can divide an audio signal into multiple sub-audio signals according to a time window. These sub-audio signals can specifically include segments 1 to N. Then, segment loudness detection is performed on each segment to obtain the corresponding loudness value. For example, the loudness value of segment 1 is LUFS_1, and the loudness value of segment 2 is LUFS_2. Next, the loudness values ​​are summarized to obtain a loudness analysis report and output it. Optionally, the output results of the segment detection can include a list of LUFS values ​​for each segment, statistical indicators, and a description of the loudness distribution characteristics. The statistical values ​​can include at least one of the following: maximum LUFS value, minimum LUFS value, average LUFS value, LUFS variance, etc. The loudness distribution characteristics description can be, for example, "low at the beginning and normal at the end" or "high overall with large fluctuations".

[0159] In the above embodiments, two modes are supported: overall quality detection and segmented quality detection for audio signals. The overall mode quickly evaluates the global features of the audio signal, while the segmented mode accurately captures the step-by-step features in the time dimension. This enables adaptive coordination between overall coarse screening and local fine detection, significantly improving the comprehensiveness and accuracy of audio quality detection while reducing computational overhead.

[0160] In an optional embodiment, the audio processing method further includes: determining that the audio signal meets the optimization conditions when the duration of the audio signal exceeds a duration threshold; and determining that the audio signal meets the optimization conditions when the duration of the audio signal is less than or equal to the duration threshold and the overall index value exceeds the expected range of the index value.

[0161] The duration threshold can be determined based on industry experience. Specifically, when the duration of an audio signal exceeds the duration threshold, the probability of it experiencing various acoustic states (such as different speakers, background noise, transient interference, etc.) on the time axis increases significantly, thus increasing the likelihood of local mutations. However, not all local mutations can be effectively identified through segmented difference prediction conditions. For example, some mutations have small changes in short-time energy, zero-crossing rate, or spectral characteristics, or the change process is relatively smooth, resulting in insufficient inter-frame differences to trigger the prediction threshold; other mutations, although obvious locally, are diluted in the overall statistical characteristics due to being surrounded by a longer, stable signal, and are also difficult to manifest as significant segmented differences. In this case, if judgment is based solely on the overall quality detection index values, even if these indicators do not exceed the expected range, there may still be local flaws in the audio signal that affect the listening experience. Therefore, to comprehensively ensure audio quality, regardless of whether the overall index values ​​meet the standards, it is necessary to further segment and refine the quality detection of long-duration audio signals to capture local anomalies overlooked by the overall assessment.

[0162] Optionally, the computer device may perform overall optimization of the audio signal if the duration of the audio signal exceeds a duration threshold and the audio signal does not meet the segment difference prediction conditions; and perform segment optimization of the audio signal if the duration of the audio signal exceeds a duration threshold and the audio signal meets the segment difference prediction conditions.

[0163] When the duration of an audio signal is less than or equal to a duration threshold, the probability of local mutations is relatively low. Based on this, the relationship between its overall index value and the expected range of index values ​​can be used to determine whether the audio signal needs optimization. Optionally, if the duration of the audio signal is less than or equal to the duration threshold and the overall index value exceeds the expected range of index values, the audio signal can be determined to meet the optimization conditions.

[0164] Taking loudness as an example, where the quality indicator is used. For instance, such as... Figure 10 As shown, the computer device can adaptively select the detection mode through an intelligent agent. Specifically, the agent receives audio information such as duration, scene, and source, and determines whether the audio duration exceeds a duration threshold. If the audio duration is less than or equal to the duration threshold, the overall loudness of the audio signal is detected, and if the overall LUFS does not exceed the expected loudness range, the audio signal is directly output to downstream applications such as speech-to-text conversion. Optionally, if the overall LUFS exceeds the expected loudness range, the overall loudness of the audio signal can be optimized.

[0165] Optionally, if the audio duration exceeds a duration threshold, first perform overall loudness detection on the audio signal. If the overall LUFS does not exceed the expected loudness range, further determine whether the audio signal meets the segmented difference prediction condition. If not, it indicates that the audio signal has uniform characteristics, and overall loudness optimization processing is sufficient. If the audio signal meets the segmented difference prediction condition, it indicates that the audio signal may have segmented differences. Then, further perform segmented loudness detection to obtain the corresponding segmented loudness values, and use a segmented optimization strategy to optimize the loudness. Optionally, if the overall LUFS of the audio signal exceeds the expected loudness range, segmented difference prediction is unnecessary, and segmented loudness detection and segmented loudness optimization can be performed directly.

[0166] In the above embodiments, setting different optimization conditions according to the audio duration can comprehensively ensure audio quality and help to further improve the audio processing effect.

[0167] In some embodiments, the audio signal includes multiple sub-audio signals; the adjustment value includes a sub-adjustment value of at least one target sub-audio signal among the sub-audio signals. In this embodiment, adjusting the audio signal according to the adjustment value to obtain an optimized audio signal includes: adjusting each target sub-audio signal according to the corresponding sub-adjustment value to obtain an updated sub-audio signal; and, if all target sub-audio signals have been adjusted, obtaining the optimized audio signal by splicing the current sub-audio signals.

[0168] The sub-audio signals can be obtained by segmenting the audio signal. Optionally, the audio signal can be segmented to obtain multiple sub-audio signals after acquisition. Optionally, the audio signal can be segmented during quality detection to obtain multiple sub-audio signals. Optionally, the large language model can also segment the audio signal by calling an audio segmentation tool during the determination of the adjustment value to obtain multiple sub-audio signals, and then determine the sub-adjustment value corresponding to each sub-audio signal.

[0169] During application, specific sub-adjustment values ​​can be generated for each segment of the audio signal, rather than using a single adjustment value to optimize the entire audio signal, in order to achieve refined quality optimization.

[0170] Optionally, the computer device can determine the individual sub-adjustment value for each sub-audio signal. Optionally, this sub-adjustment value can be a setting character, indicating that the corresponding sub-audio signal does not need optimization. This setting character can be, for example, zero, or a special character such as "NULL".

[0171] Specifically, the computer device can adjust each target sub-audio segment with a sub-adjustment value according to the corresponding sub-adjustment value to obtain an updated sub-audio signal. Then, with all target sub-audio segments adjusted, the optimized audio signal is obtained by splicing the current sub-audio signals.

[0172] Optionally, the spliced ​​sub-audio signals include non-target sub-audio signals other than the target sub-audio signals in each segment, and these non-target sub-audio signals do not need to be updated. The spliced ​​sub-audio signals also include the updated sub-audio signals obtained by adjusting the target sub-audio signals.

[0173] Optionally, after segment quality testing, the computer equipment can generate independent sub-adjustment values ​​for each segment of the audio signal by calling the corresponding skill tools, based on the segment index values ​​of each segment. Subsequently, each segment of the audio signal can be optimized according to each sub-adjustment value.

[0174] In the above embodiments, for at least one target sub-audio segment in each sub-audio signal, a sub-adjustment value corresponding to the target sub-audio signal is generated, and the target sub-audio signal is adjusted according to the sub-adjustment value. This avoids applying the same processing parameters to the entire audio segment, thereby achieving refined audio optimization and further improving the audio processing effect.

[0175] In some embodiments, an optimized audio signal is obtained by splicing together the current sub-audio signals, including: determining a smoothing window that is adapted to the sub-audio signals; splicing adjacent segments in the current sub-audio signals, and smoothly adjusting the segment boundaries of the adjacent segments according to the smoothing window to obtain the optimized audio signal.

[0176] The smoothing window is a finite-length window function used to weight and superimpose sampling points near the boundary when splicing adjacent audio segments. This results in a smooth amplitude change between the two signals in the transition region, eliminating abrupt noise caused by waveform discontinuities. The length of the smoothing window determines the duration of the transition region, and its shape determines the rate of amplitude change. Optionally, the smoothing window can be, for example, a cosine window, a triangular window, or a trapezoidal window.

[0177] Optionally, a uniform smoothing window can be used for each audio segment. This smoothing window could be, for example, a 50ms cosine window.

[0178] Optionally, a personalized smoothing window can be configured for each segment of the audio signal.

[0179] Optionally, the length of the smoothing window can be determined based on the local features or temporal resolution of the sub-audio signals. For example, for speech or monotone music signals, the smoothing window length can be set to one or two fundamental periods. This ensures that the glottal pulse phase is aligned at the splicing boundary, avoiding periodic amplitude abrupt changes caused by periodic misalignment. Alternatively, the spectral envelope distance between adjacent sub-audio signals near the boundary can be calculated. If the envelope heights are similar, the window can be appropriately shortened (e.g., using 5 ms); if the envelope differences are large, the window can be appropriately lengthened (e.g., using 30-40 ms) to more smoothly transition between different formant structures.

[0180] Optionally, the specific method for smooth transition adjustment may include at least one of crossfade in / out, linear interpolation, or low-pass filtering.

[0181] In an optional embodiment, with all target sub-audio segments already adjusted, the computer device can splice the current sub-audio signals to obtain a spliced ​​signal. Then, according to a smoothing window, the segment boundaries of each group of adjacent segments in the spliced ​​signal are adjusted for smooth transition to obtain an optimized audio signal.

[0182] Let's take the case where the audio signal consists of three audio segments as an example. For instance, such as... Figure 11 As shown, the updated sub-audio signal includes segment 1, segment 2, and segment 3. The computer device can splice these three sub-audio signals to obtain the spliced ​​signal P0. Then, according to the smoothing window adapted to segments 1 and 2, the boundary between segment 1 and segment 2 is smoothly adjusted, and according to the smoothing window adapted to segments 2 and 3, the boundary between segment 2 and segment 3 is smoothly adjusted to obtain the optimized audio signal P1.

[0183] In an optional embodiment, after all target sub-audio segments have been adjusted, the computer device can determine the two earliest sub-audio segments according to their current order in the audio signal, splice these two segments together, and smoothly adjust the segment boundaries of these two segments according to a smoothing window. Then, the computer device can determine the earliest sub-audio segment from the remaining sub-audio segments, splice this segment together with the previous splicing result, and smoothly adjust the segment boundaries according to a smoothing window. This process continues until the last sub-audio segment is spliced, resulting in an optimized audio signal.

[0184] Let's take the case where the audio signal consists of three audio segments as an example. For instance, as shown... Figure 12 As shown, the updated sub-audio signal includes segment 1, segment 2, and segment 3. The computer device can first splice segment 1 and segment 2, and then smoothly adjust the boundary between segment 1 and segment 2 according to a smoothing window adapted to segment 1 and segment 2 to obtain the spliced ​​signal P_1. Then, P_1 and segment 3 are spliced ​​together, and the boundary between spliced ​​signal P_1 and segment 3 is smoothly adjusted according to a smoothing window adapted to segment 2 and segment 3 to obtain the optimized audio signal P_2.

[0185] In the above embodiments, for each segment of audio signal, adjacent segments in each segment of audio signal are spliced ​​together, and the segment boundaries of adjacent segments are smoothly adjusted according to a smooth window to obtain an optimized audio signal. This can avoid signal abrupt changes or perceptible connection defects at the segment boundaries, which is conducive to further improving the audio processing effect.

[0186] In some embodiments, the audio signal includes multiple sub-audio signals. The adjustment value includes a sub-adjustment value for at least one target sub-audio signal among the sub-audio signals. In this embodiment, the adjustment value for the audio signal is determined by invoking a skill tool adapted to the quality index, guided by a large language model and prompting information. This includes: clustering each sub-audio signal based on its segmented index value to determine the audio cluster to which each sub-audio signal belongs; and for the target clusters that meet the optimization conditions within each audio cluster, the sub-adjustment value for each target sub-audio signal within the target cluster is determined by invoking a skill tool adapted to the quality index, guided by a large language model and prompting information.

[0187] Specifically, the sub-audio signals can be clustered according to segmented index values ​​to obtain multiple audio clusters, thereby achieving grouping of sub-audio signals. Then, reasoning is performed on the audio clusters to determine the adjustment value for the same audio cluster.

[0188] Optionally, each audio cluster may include target clusters that meet the optimization conditions and non-target clusters that do not meet the optimization conditions. Optionally, audio clusters whose segmentation index values ​​are within the expected range of the index values ​​may be identified as non-target clusters, and audio clusters whose segmentation index values ​​exceed the expected range of the index values ​​may be identified as target clusters.

[0189] Optionally, the computer device can, under the guidance of prompts, use a large language model to call skill tools adapted to the quality indicators to determine the sub-adjustment values ​​for each segment of target sub-audio in the target audio cluster that meets the optimization conditions.

[0190] For example, such as Figure 13 As shown, taking loudness as the quality index as an example, the computer device can analyze the loudness characteristics of each segment of the sub-audio signal using a large language model. These loudness characteristics are determined based on the segment loudness detection results of each segment of the sub-audio signal. When the expected range of the index value is "-24 LUFS to -14 LUFS", the computer device can cluster the loudness indices to obtain normal loudness groups with segment index values ​​greater than -24 LUFS and less than -14 LUFS, which are non-target clusters requiring no processing or only slight adjustment. It can also obtain low loudness groups with segment index values ​​less than -30 LUFS and high loudness groups with segment index values ​​greater than -10 LUFS, which are target clusters requiring optimization. Optionally, for different groups, the large language model can use skill tools to generate adjustment parameter sets for each segment of the sub-audio signal within that group, achieving a mapping between adjustment values ​​and sub-audio signals. These adjustment parameter sets can include adjustment values ​​corresponding to multiple adjustment items, working together to optimize the corresponding sub-audio signals.

[0191] Optionally, the low loudness group can be optimized for loudness using a low threshold and high compensation gain; the high loudness group can be optimized for loudness using a lower threshold and a high compression ratio.

[0192] Optionally, for each audio segment, the corresponding sub-adjustment value can be used to adjust the audio segment to obtain an optimized sub-audio signal. Finally, the computer device can merge the audio segments to obtain the optimized audio signal.

[0193] Optionally, unlike traditional solutions that generate compression parameters based on rule tables or fixed formulas, various embodiments of this application can utilize the contextual reasoning capabilities of large language models to intelligently generate adjustment values ​​for multiple adjustment items of quality indicators based on quality inspection data.

[0194] Taking loudness as an example, which is a quality indicator. Figure 14 As shown, the input information of a large language model can include loudness detection results, audio information, loudness expectation range, and historical processing records. Audio information can include audio duration, sampling rate, and number of channels. Based on this input information, the large language model can perform contextual reasoning to generate adjustment parameters, i.e., adjustment values, for the audio signal. The output information of the large language model can include a six-dimensional compressor parameter combination, including threshold, compression ratio, attack time, release time, compensation gain, and inflection point.

[0195] Optionally, the audio signal is adjusted according to the adjustment value to obtain an optimized audio signal, including: for each target sub-audio segment, adjusting the target sub-audio according to the sub-adjustment value of the target sub-audio segment to obtain an optimized sub-audio signal; and obtaining an optimized audio signal by splicing the current sub-audio segments together.

[0196] In the above embodiments, performing centralized inference on sub-audio signals with similar segment detection results can reduce the number of inferences in the large language model. This can reduce workload while ensuring audio processing quality and improve work efficiency.

[0197] In some embodiments, adjusting the audio signal according to the adjustment value to obtain an optimized audio signal includes: adjusting the audio signal according to the adjustment value to obtain an updated audio signal; returning to the step of performing quality detection on the audio signal according to a quality index; if the iteration termination condition is not met, generating an adjustment record for the current iteration round and adding the adjustment record to the prompt information for the next iteration process; if the iteration termination condition is met, obtaining the optimized audio signal.

[0198] The generated adjustment record includes the most recently used adjustment value and the most recently detected indicator value. This adjustment record clearly demonstrates the effectiveness of the most recent adjustment. The iteration termination condition can be either reaching a set number of iterations or the updated audio signal failing to meet the optimization conditions.

[0199] Optionally, a tolerance threshold can be set, and the index value detected for the updated audio signal can be compared with the expected range of the index value. If the latest obtained index value is within the expected range, the updated audio signal can be considered to meet the standard, i.e., it does not meet the optimization condition. If the latest obtained index value is not within the expected range, but the deviation of the index value from the boundary of the expected range is less than or equal to the tolerance threshold, the updated audio signal can also be considered to meet the standard, i.e., it does not meet the optimization condition. If the latest obtained index value is not within the expected range, but the deviation of the index value from the boundary of the expected range is greater than the tolerance threshold, it is considered unqualified, and the process proceeds to the next iteration. For example, for the loudness index, a tolerance threshold of 2LU can be set, and when the deviation of the loudness value of the updated audio signal from the boundary of the expected loudness range is less than or equal to 2LU, it is considered to meet the standard, and the audio optimization process ends.

[0200] Optionally, a maximum number of iterations can be set to prevent infinite loops. This maximum number of iterations could be, for example, 3.

[0201] Optionally, if the maximum number of iterations is not reached, the computer device can select the version output with the smallest deviation from the expected range of the indicator value from all historical results, ensuring that the system can always give the best result.

[0202] Specifically, an iterative verification and optimization mechanism can be used to ensure that the processing results meet the requirements. During each iterative inference process, the context received by the large language model can include the adjustment values ​​applied in all previous iterations, as well as the index values ​​detected after adjustment. This allows the large language model to clearly understand the adjustment effects of previous iterations, gain historical experience, and learn from this historical experience to adjust the subsequent optimization direction, avoiding the output of duplicate or invalid adjustment value combinations.

[0203] In the above embodiments, audio processing is performed through an iterative verification and optimization mechanism, which ensures that the audio processing results can meet the needs of actual application scenarios and is conducive to further improving the audio processing effect.

[0204] In some embodiments, the audio processing method further includes: performing text conversion on the optimized audio signal to obtain text information; generating audio optimization experience containing various adjustment values ​​if the conversion quality meets the requirements based on the text information; determining the cause of failure if the conversion quality does not meet the requirements based on the text information; updating at least a portion of the prompt information if the cause of failure is related to a quality indicator, and returning to the step of calling the skill tool through a large language model if the cause of failure is related to other indicators besides the quality indicator; and using the other indicators as new quality indicators and returning to the step of obtaining the expected range of indicator values ​​for the audio signal under the new quality indicators.

[0205] Among them, audio optimization experience is used as a reference in the reasoning process of large language models, specifically as a reference when large language models determine adjustment values.

[0206] Optionally, the optimized audio signals obtained in the various embodiments of this application can be used as input for downstream technologies such as Speech-to-Text (STT) to improve recognition accuracy. In this case, an STT result feedback loop can be added, enabling the system to continuously optimize the generation strategy of adjustment values ​​based on the actual transcription effect.

[0207] Optionally, the text conversion quality can be determined based on the confidence score output by the STT model. A higher confidence score indicates better conversion quality.

[0208] Optionally, the text information obtained from audio conversion can be scored using a Natural Language Inference (NLI) model to determine the conversion quality.

[0209] Specifically, computer equipment can perform text conversion on the optimized audio signal to obtain text information, and then test the conversion quality of this text information. If the conversion quality of the text information meets the requirements, audio optimization experience containing various adjustment values ​​is generated as a reference for subsequent inference in the large language model. If the conversion quality of the text information does not meet the requirements, the cause of failure can be determined by analyzing the performance of the audio signal under multiple quality indicators.

[0210] Optionally, if the cause of failure is related to the quality metrics initially optimized, the computer device may update at least a portion of the prompt message and return to the steps of invoking the skill tool through the large language model.

[0211] Optionally, the updated content in the prompt message may include at least one of the following: the expected range of indicator values ​​or historical adjustment records. For example, by narrowing the expected range of indicator values, an audio signal with better performance under a specific quality indicator can be obtained, thereby improving the conversion quality.

[0212] Optionally, if the failure is related to indicators other than the quality indicator, then the other indicator is used as the new quality indicator, and the step of obtaining the expected range of the indicator value of the audio signal under the new quality indicator is returned.

[0213] Taking the case where the quality index is used as the loudness index as an example. For instance, such as... Figure 15 As shown, the optimized audio signal obtained from the audio loudness optimization process is sent to the STT module to be converted into text information. Next, the conversion quality of the text information is evaluated. If the conversion quality meets the requirements, the mapping relationship between the audio features and the optimal adjustment value is recorded as a successful case and added to the reasoning experience knowledge base of the large language model for reference in subsequent tasks. If the conversion quality does not meet the requirements, the cause of failure is analyzed to determine if it is related to audio loudness. If it is related to audio loudness, the audio loudness can be re-optimized. If the cause of failure is unrelated to audio loudness, the task can be transferred to other processing modules to optimize other indicators that caused the failure.

[0214] Optionally, the reasoning experience knowledge base can store successful "audio feature → optimal adjustment value" mapping relationships in the form of a vector database. For newly acquired audio signals to be processed, historical cases with similar features can be retrieved from the reasoning experience knowledge base, and the historical optimal parameters can be used as the reference context (RAG enhancement) for the large language model reasoning to accelerate parameter convergence.

[0215] Optionally, conversion quality can be automatically evaluated using metrics such as confidence scores and word error rate estimates, without the need for manual annotation.

[0216] In an optional embodiment, an intelligent agent can be introduced as the core scheduler for the entire audio processing process. The agent can autonomously decide on processing strategies based on audio detection results, specifically including whether optimization is needed, whether to use overall processing or segmented processing, what adjustment method to use for optimization, and what adjustment values ​​to generate for that method. The agent can interact with external modules such as quality detection tools and audio optimization tools through function call mechanisms, forming a closed-loop processing flow of "perception-decision-execution-verification". In this scenario, when the quality metric is loudness, if the failure is related to the loudness metric, feedback can be provided to the agent used for audio loudness optimization, triggering a re-optimization of the audio loudness.

[0217] For example, such as Figure 16As shown, the entire audio processing system can include a user side 1601, a server side 1602, and an output side 1603. The user side 1601 supports user-uploaded audio files and provides an audio preprocessing entry point. The specific format of the audio file can include common formats such as WAV, MP3, and FLAC. The audio signal contained in the audio file is the audio signal to be processed. The server side 1602 deploys an intelligent agent scheduling center, which interacts with various skill modules through a function call mechanism to achieve quality detection and optimization of the audio to be processed. The optimized audio signal can be received by the output side 1603 for text conversion and output of the converted text information.

[0218] Optionally, the system may selectively display a loudness analysis report to the user, including the original loudness distribution, processing parameters, and a comparison of the optimized loudness distribution.

[0219] In one exemplary embodiment, such as Figure 17 As shown, the audio processing system may include an access layer 1701, an agent scheduling layer 1702, a storage layer 1703, a large language model inference layer 1704, and a tool execution layer 1705. The access layer 1701 provides audio file interfaces and real-time audio stream interfaces. The agent scheduling layer 1702 is responsible for context management and tool registration, and also performs inference and decision-making through the large language model. The storage layer 1703 stores audio files, caches processing parameters, and records logs. The large language model inference layer 1704 manages prompt word targets and performs function call parsing to invoke the tools registered by the agent. The tool execution layer 1705 specifically provides several registered skill tools, such as quality detection tools, audio processing tools, audio format conversion tools, and audio segmentation tools, for the large language model to call and complete the corresponding processing flow.

[0220] In the above embodiments, a feedback loop for the text conversion results is added, allowing the system to continuously accumulate successful cases during operation. This leads to a continuous improvement in the accuracy of the large language model's inference results as the amount of data increases, which is beneficial for further improving the audio processing effect.

[0221] In some embodiments, such as Figure 18 As shown, an audio processing method is provided, which can be executed by a computer device, the computer device being... Figure 1 In this embodiment of the application, the method may include the following steps: (The terminal 100 or server 200 in the example are mentioned.)

[0222] Step S1801: Obtain the audio signal to be processed and the expected range of the audio signal's index value under at least one quality index;

[0223] Step S1802: Perform overall quality detection on the audio signal according to the quality index to obtain the overall index value of the audio signal corresponding to the quality index;

[0224] Step S1803: If the audio signal does not meet the segmentation conditions, the overall index value is used as the index value of the audio signal corresponding to the quality index.

[0225] Step S1804: If the audio signal meets the segmentation conditions, the audio signal is split into multiple sub-audio signals.

[0226] Optionally, the computer device may determine that the audio signal meets the segmentation conditions if the duration of the audio signal exceeds the duration threshold and the overall index value exceeds the expected range of index values.

[0227] Optionally, the computer device may determine that the audio signal meets the segmentation conditions if the duration of the audio signal exceeds the duration threshold and the audio signal meets the segmentation difference prediction conditions.

[0228] Step S1805: For each segment of sub-audio signal, perform segmented quality detection on the sub-audio signal according to the quality index to obtain the segmented index value of the sub-audio signal corresponding to the quality index.

[0229] Step S1806: The overall index value and the index values ​​of each segment are used as the index values ​​of the audio signal corresponding to the quality index.

[0230] Step S1807: If the audio signal meets the optimization conditions based on the index value and the expected range of the index value, determine the acoustic mode corresponding to the audio signal.

[0231] Among them, acoustic modes are used to characterize the foreground sound emission mode and background noise intensity of audio signals;

[0232] Optionally, the computer device can determine that the audio signal meets the optimization conditions if the duration of the audio signal exceeds a duration threshold;

[0233] Optionally, the computer device may determine that the audio signal meets the optimization conditions if the duration of the audio signal is less than or equal to the duration threshold and the overall index value exceeds the expected range of the index value.

[0234] Step S1808: Obtain the prompt word template that matches the acoustic pattern under the quality index;

[0235] The prompt template includes fixed content and variable placeholders; the fixed content includes audio optimization requirements for quality indicators in acoustic mode, as well as tool call guidance information adapted to quality indicators;

[0236] Optionally, the prompt template includes system prompts, input content, and tool invocation guidance content; the system prompts include audio optimization requirements; the tool invocation guidance content includes tool invocation guidance information; and the input content includes variable placeholders.

[0237] Step S1809: Fill the variable placeholders in the input content with the index value and the expected range of the index value to obtain the prompt information;

[0238] Step S1810: Based on the segmentation index values ​​of each segment of audio signal, perform clustering processing on each segment of audio signal to determine the audio cluster to which each segment of audio signal belongs;

[0239] Step S1811: For the target sub-audio segments that meet the optimization conditions in each audio sub-cluster, the large language model, guided by the prompt information, calls the skill tools that are adapted to the quality indicators to determine the sub-adjustment values ​​for each target sub-audio segment in the target sub-cluster.

[0240] Step S1812: For each target sub-audio segment, adjust the target sub-audio according to the sub-adjustment value corresponding to the target sub-audio to obtain the updated sub-audio signal;

[0241] Step S1813: With all target sub-audio segments adjusted, determine a smooth window that matches the sub-audio signal.

[0242] Step S1814: Segment adjacent segments in each audio signal, and adjust the segment boundaries of adjacent segments smoothly according to the smooth window to obtain the updated audio signal.

[0243] Step S1815: Perform quality detection on the updated audio signal according to the quality indicators to obtain the quality detection results;

[0244] The quality inspection results may include overall index values ​​and segmented index values;

[0245] Step S1816: If the iteration termination condition is not met, generate the adjustment record for the current iteration round and add the adjustment record to the prompt information for the next iteration round; return to step S1810.

[0246] The adjustment record includes the most recently used adjustment value and the most recently obtained quality inspection result;

[0247] Step S1817: If the iteration termination condition is met, the optimized audio signal is obtained;

[0248] Step S1818: Perform text conversion on the optimized audio signal to obtain text information;

[0249] Step S1819: Based on the text information, if the conversion quality meets the requirements, generate audio optimization experience containing each adjustment value.

[0250] Among them, audio optimization experience is used as a reference in the reasoning process of large language models;

[0251] Step S1820: If the conversion quality does not meet the requirements based on the text information, determine the reason for the failure.

[0252] Step S1821: If the failure reason is related to the quality index, update at least part of the prompt information and return to step S1810.

[0253] In step S1822, if the failure is related to indicators other than the quality indicators, then the other indicators are used as new quality indicators, and the process returns to step S1801 to perform quality testing and optimization for the new quality indicators.

[0254] The following section uses the case where the loudness index is the quality index as an example to introduce the audio processing method provided in this application.

[0255] In one optional embodiment, the audio processing method provided in this application can be applied to various scenarios such as enterprise meeting speech transcription, customer service calls, multimedia content subtitle generation, and mobile voice assistants. In enterprise meeting scenarios, participants vary in distance from the microphone and speaking volume, resulting in significant loudness differences across different time periods in the recorded meeting audio. Using the solution provided in this application, the loudness characteristics of each time period can be automatically analyzed before the audio is input into the STT system, and different compression parameters can be applied segment by segment to make the overall audio loudness more uniform, significantly improving the transcription accuracy of meeting minutes.

[0256] In customer service call recordings, there may be significant differences in volume between the customer and agent sides, and network transmission may cause audio volume attenuation. The solution provided in this application utilizes intelligent loudness optimization to ensure that both parties' voices are within the optimal recognition range of the STT system, thereby improving the completeness and accuracy of call recording transcription.

[0257] In multimedia content such as videos and podcasts, the audio may interweave with background music and sound effects, resulting in uneven audio volume. The solution provided in this application allows for refined segmented loudness optimization of the audio, improving the accuracy of subtitle generation.

[0258] In mobile device recording scenarios, factors such as environmental noise and handheld stability cause significant fluctuations in voice loudness. The solution provided in this application allows for real-time loudness optimization of voice input on the terminal or in the cloud, improving the voice recognition performance of voice assistants.

[0259] In traditional techniques, the main methods for optimizing loudness are as follows.

[0260] Solution 1 is a fixed-parameter audio normalization method, which adjusts the overall loudness to a fixed target value by performing peak normalization or LUFS normalization on the input audio. This solution is simple to implement, but its "one-size-fits-all" approach cannot handle situations where different segments of the same audio have significant loudness differences. For example, in an audio clip containing quiet and loud sections, fixed normalization can only balance the overall loudness and cannot optimize each segment individually.

[0261] Option two is a dynamic range compression method based on fixed rules, specifically using a compressor with fixed parameters (such as a fixed threshold of -12dB, a compression ratio of 4:1, etc.) to perform dynamic range compression on the audio. While this method can reduce loudness differences to some extent, because the parameters are fixed, it cannot be adjusted according to the actual loudness characteristics of the audio. For audio with low loudness, the fixed threshold may not trigger compression at all, rendering the processing ineffective; for audio with complex loudness characteristics, fixed parameters may lead to over-compression or under-compression.

[0262] Option 3 is an adaptive processing method based on a rule engine. Specifically, it uses a pre-defined rule engine (if-else logic or lookup table) to select the corresponding compression parameters from a pre-defined parameter table based on the detected loudness value. This option has some adaptability, but the rule coverage is limited, making it difficult to handle complex and ever-changing real-world audio scenarios. Furthermore, rule formulation relies on the experience of audio engineers, and there may be conflicts or incomplete coverage between rules.

[0263] Based on this, in various embodiments of this application, by introducing an intelligent agent architecture and the reasoning capabilities of a large language model, a complete closed-loop processing system of "detection-analysis-generation-execution-verification-optimization" is constructed, effectively solving all the above-mentioned problems. Specifically: the reasoning capabilities of LLM are used to replace fixed rules to achieve intelligent dynamic generation of compression parameters; segmented detection and segmented processing are supported to finely address loudness changes in audio; an iterative verification mechanism is designed to ensure that the processing effect meets the standards before outputting to the STT system; the semantic understanding capabilities of LLM are used to understand audio features from a higher dimension and generate optimal parameter combinations (i.e., the adjustment values ​​corresponding to multiple adjustment items); joint optimization is performed in the six-dimensional parameter space to maximize the improvement of processing effect.

[0264] In one exemplary embodiment, such as Figure 19 and Figure 20 As shown, the agent-driven closed-loop optimization process includes the following steps:

[0265] Step S1901: Receive the input audio signal.

[0266] The audio signal can be extracted from audio files submitted by the user or the upstream system.

[0267] Step S1902: The agent analyzes the task context.

[0268] Step S1903: The agent selects a detection mode; if the selected detection mode is the segmented detection mode, then proceed to step S1904; if the selected detection mode is the overall detection mode, then proceed to step S1906.

[0269] Optional, such as Figure 20 As shown, the agent can input the task context and audio information into the large language model and then make a decision on the detection mode by calling the large language model.

[0270] Step S1904: Call the audio segmentation tool to segment the audio.

[0271] Step S1905: Call the loudness detection tool for each segment to detect the loudness of each segment.

[0272] Step S1906: Call the loudness detection tool to detect the overall loudness.

[0273] Optional, such as Figure 20 As shown, overall loudness detection can be performed first. If the overall loudness detection result is within the expected range, no processing is required, and the original audio can be directly output to the downstream STT system. If the overall loudness detection result is not within the expected range, segmented detection can be further performed.

[0274] Step S1907: The agent receives the loudness detection result.

[0275] Optional, such as Figure 20 As shown, the loudness detection results can be further fed back to the large language model so that the large language model can make subsequent decisions.

[0276] In step S1908, the agent infers whether the loudness is within the expected range. If yes, proceed to step S1915; otherwise, proceed to step S1909.

[0277] Step S1909: The agent calls the large language model to generate compressor parameters.

[0278] Step S1910: The agent calls the audio compression tool to perform compression processing according to the generated compressor parameters;

[0279] Step S1911: The agent re-detects the loudness of the processed audio.

[0280] In step S1912, the agent verifies whether the processing effect meets the standard based on the detection result; if it does not meet the standard and the maximum number of iterations has not been exceeded, then step S1913 is executed; if it does not meet the standard and the maximum number of iterations has been reached, then step S1914 is executed; if it meets the standard, then step S1915 is executed.

[0281] Step S1913: The agent analyzes the causes of the deviation and adjusts the parameter strategy;

[0282] Step S1914: Select the best historical result for output;

[0283] Step S1915: Output audio to the speech-to-text system;

[0284] Step S1916: Convert the speech in the audio into text;

[0285] Step S1917: Output the transcribed text.

[0286] The above-described audio processing method can bring at least the following beneficial effects:

[0287] (1) Significantly improves the accuracy of STT recognition.

[0288] Through refined loudness optimization processing, the loudness of the input audio is adjusted to the optimal recognition range of the STT system, effectively solving problems such as speech being submerged by noise due to excessively low loudness, signal distortion due to excessively high loudness, and recognition instability due to loudness fluctuations. This significantly improves the recognition accuracy of the STT system. In actual tests, for audio samples with abnormal loudness, the STT recognition accuracy can be improved by 10%-40% after processing with the solution provided in this application (depending on the degree of deviation of the original loudness of the audio).

[0289] (2) Achieve intelligent dynamic generation of compression parameters, and say goodbye to the "one-size-fits-all" processing.

[0290] Leveraging the contextual reasoning capabilities of large language models, the system dynamically generates the optimal combination of six-dimensional compression parameters based on the actual loudness characteristics of each audio segment, completely overcoming the limitations of fixed parameters and rules in traditional solutions. The system can handle various complex audio loudness scenarios and tailor the optimal processing solution for different audio types.

[0291] (3) Segmented detection and segmented processing to achieve refined optimization.

[0292] It supports segmenting audio by time, independently detecting loudness for each segment, and generating targeted compression parameters. This refined processing method is particularly suitable for scenarios with large loudness variations, such as multi-person meetings and interviews, and can effectively balance loudness differences between different speakers and at different times.

[0293] (4) Closed-loop iterative optimization to ensure processing effect.

[0294] A closed-loop iterative mechanism of "processing-verification-adjustment" was designed. After each processing step, the agent automatically verifies the effect. If the result is not satisfactory, it automatically adjusts the parameters and reprocesses to ensure that the final output audio loudness is always within the target range. This mechanism significantly improves the robustness and processing success rate of the system.

[0295] (5) Zero-configuration user experience.

[0296] The entire processing flow is driven autonomously by the intelligent agent, requiring no audio engineering knowledge or manual compressor parameter configuration from the user. Users simply input the audio file to obtain optimized audio and transcription results, significantly lowering the barrier to entry.

[0297] (6) The system’s scalability and evolution capability.

[0298] The agent-based architecture gives the system excellent scalability: processing capabilities can be expanded by registering new tool modules (such as noise reduction and equalization), inference quality can be improved by updating the LLM model, and parameter generation strategies can be continuously optimized by accumulating processing experience. Furthermore, the system can continuously evolve with the development of artificial intelligence technology.

[0299] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0300] Based on the same inventive concept, this application also provides an audio processing apparatus for implementing the audio processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more audio processing apparatus embodiments provided below can be found in the limitations of the audio processing method described above, and will not be repeated here.

[0301] In some embodiments, such as Figure 21 As shown, an audio processing apparatus is provided, comprising:

[0302] The acquisition module 2101 is used to acquire the audio signal to be processed and the expected range of the index value of the audio signal under at least one quality index.

[0303] The detection module 2102 is used to perform quality detection on the audio signal according to the quality index and obtain the index value of the audio signal corresponding to the quality index.

[0304] The prompt message construction module 2103 is used to construct prompt messages based on the indicator value, the expected range of the indicator value, and tool call guidance information adapted to the quality indicator, when the audio signal meets the optimization conditions based on the indicator value and the expected range of the indicator value.

[0305] The adjustment value determination module 2104 is used to determine the adjustment value for the audio signal by calling the skill tool adapted to the quality index under the guidance of the prompt information through the large language model.

[0306] The audio optimization module 2105 is used to adjust the audio signal according to the adjustment value to obtain the optimized audio signal.

[0307] In one embodiment, the detection module 2102 is specifically used for: performing overall quality detection on the audio signal according to the quality index to obtain the overall index value of the audio signal corresponding to the quality index; if the audio signal does not meet the segmentation conditions, using the overall index value as the index value of the audio signal corresponding to the quality index; if the audio signal meets the segmentation conditions, splitting the audio signal into multiple sub-audio signals; for each sub-audio signal, performing segmented quality detection on the sub-audio signal according to the quality index to obtain the segmented index value of the sub-audio signal corresponding to the quality index; and using the overall index value and each segmented index value as the index value of the audio signal corresponding to the quality index.

[0308] In one embodiment, the audio processing apparatus further includes a segmentation determination module, configured to perform at least one of the following two: determining that the audio signal meets the segmentation conditions when the duration of the audio signal exceeds a duration threshold and the overall index value exceeds the expected range of the index value; and determining that the audio signal meets the segmentation conditions when the duration of the audio signal exceeds a duration threshold and the audio signal meets the segmentation difference prediction conditions.

[0309] In one embodiment, the audio processing apparatus further includes an optimization judgment module, configured to: determine that the audio signal meets the optimization conditions when the duration of the audio signal exceeds a duration threshold; and determine that the audio signal meets the optimization conditions when the duration of the audio signal is less than or equal to the duration threshold and the overall index value exceeds the expected range of the index value.

[0310] In one embodiment, the audio signal includes multiple sub-audio signals; the adjustment value includes a sub-adjustment value of at least one target sub-audio signal among the sub-audio signals. In this embodiment, the audio optimization module 2105 includes: an audio optimization unit, configured to adjust the target sub-audio signals according to the corresponding sub-adjustment values ​​for each target sub-audio signal to obtain an updated sub-audio signal; and a splicing unit, configured to obtain an optimized audio signal by splicing the current sub-audio signals after all target sub-audio signals have been adjusted.

[0311] In one embodiment, the splicing unit is specifically used to: determine a smooth window that is adapted to the sub-audio signal; splice adjacent segments in each sub-audio signal for the current sub-audio signal, and adjust the segment boundaries of the adjacent segments smoothly according to the smooth window to obtain an optimized audio signal.

[0312] In one embodiment, the audio signal includes multiple sub-audio signals; the adjustment value includes a sub-adjustment value for at least one target sub-audio signal in each sub-audio signal. In this embodiment, the adjustment value determination module 2104 is specifically used to: perform clustering processing on each sub-audio signal based on its respective segmentation index value to determine the audio cluster to which each sub-audio signal belongs; for the target cluster that meets the optimization conditions in each audio cluster, under the guidance of prompt information, use a large language model to call a skill tool adapted to the quality index to determine the sub-adjustment value for each target sub-audio signal in the target cluster.

[0313] In one embodiment, the prompt information construction module 2103 includes: an acoustic pattern determination unit, used to determine the acoustic pattern corresponding to the audio signal; an acoustic pattern, used to characterize the foreground sound emission mode and background noise intensity of the audio signal; a prompt word template acquisition unit, used to acquire a prompt word template that matches the acoustic pattern under the quality index; the prompt word template includes fixed content and variable placeholders; the fixed content includes the audio optimization requirements of the quality index under the acoustic pattern, and tool call guidance information adapted to the quality index; and a prompt word construction unit, used to fill the variable placeholders with index values ​​and the expected range of index values ​​to obtain prompt information.

[0314] In one embodiment, the prompt word template includes system prompt content, input content, and tool invocation guidance content; the system prompt content includes audio optimization requirements; the tool invocation guidance content includes tool invocation guidance information; and the input content includes variable placeholders. In this embodiment, the prompt word construction unit is specifically used to: fill the variable placeholders in the input content with indicator values ​​and expected ranges of indicator values ​​to obtain prompt information.

[0315] In one embodiment, there are multiple quality indicators; the skill tools corresponding to each quality indicator are invoked by the large language model in an invocation order. In this embodiment, the audio processing apparatus further includes an invocation order determination module, used to: determine the quality assessment granularity of each quality indicator for the audio signal; and determine the invocation order of the skill tools corresponding to each quality indicator according to the quality assessment granularity from coarse to fine.

[0316] In one embodiment, the audio processing apparatus further includes a tool registration module for: determining an adjustment method for a quality indicator and multiple adjustment items that affect the quality indicator under the adjustment method; registering skill tools adapted to the quality indicator based on each adjustment item; wherein the output of the skill tool includes the adjustment value corresponding to each adjustment item.

[0317] In one embodiment, the audio optimization module 2105 is specifically used to: adjust the audio signal according to the adjustment value to obtain an updated audio signal; return to the step of performing quality detection on the audio signal according to the quality index; if the iteration termination condition is not met, generate the adjustment record of the current iteration round and add the adjustment record to the prompt information of the next iteration process; the adjustment record includes the adjustment value used most recently and the index value obtained most recently; and if the iteration termination condition is met, obtain the optimized audio signal.

[0318] In one embodiment, the audio processing apparatus further includes a self-learning module for: performing text conversion on the optimized audio signal to obtain text information; generating audio optimization experience including various adjustment values ​​if the conversion quality meets the requirements based on the text information; using the audio optimization experience as a reference in the inference process of the large language model; determining the cause of failure if the conversion quality does not meet the requirements based on the text information; updating at least a portion of the prompt information if the cause of failure is related to a quality indicator, and returning to the step of calling the skill tool through the large language model; and using the other indicator as a new quality indicator if the cause of failure is related to an indicator other than the quality indicator, and returning to the step of obtaining the expected range of indicator values ​​of the audio signal under the new quality indicator.

[0319] Each module in the aforementioned audio processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0320] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 22As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data involved in the audio processing process. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an audio processing method.

[0321] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 23 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an audio processing method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0322] Those skilled in the art will understand that Figure 22The structure shown in Figure 23 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0323] In some embodiments, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the audio processing method described above.

[0324] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the audio processing method described above.

[0325] In some embodiments, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the audio processing method described above.

[0326] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0327] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logic devices, etc., and are not limited to these.

[0328] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0329] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An audio processing method, characterized in that, The method includes: Obtain the audio signal to be processed, and the expected range of the audio signal's index value under at least one quality index; The audio signal is subjected to quality detection according to the quality index to obtain the index value of the audio signal corresponding to the quality index; If the audio signal meets the optimization conditions based on the indicator value and the expected range of the indicator value, a prompt message is constructed according to the indicator value, the expected range of the indicator value, and tool call guidance information adapted to the quality indicator. Guided by the prompts, the large language model invokes skill tools adapted to the quality indicators to determine the adjustment value for the audio signal. The audio signal is adjusted according to the adjustment value to obtain an optimized audio signal.

2. The method according to claim 1, characterized in that, The step of performing quality detection on the audio signal according to the quality index to obtain the index value of the audio signal corresponding to the quality index includes: The audio signal is subjected to overall quality detection according to the quality index to obtain the overall index value of the audio signal corresponding to the quality index. If the audio signal does not meet the segmentation conditions, the overall index value is taken as the index value of the audio signal corresponding to the quality index. If the audio signal meets the segmentation conditions, the audio signal is split into multiple sub-audio signals; For each segment of the sub-audio signal, segment quality detection is performed on the sub-audio signal according to the quality index to obtain the segment index value of the sub-audio signal corresponding to the quality index; The overall index value and each segment index value are used as the index values ​​of the audio signal corresponding to the quality index.

3. The method according to claim 2, characterized in that, The method further includes at least one of the following: If the duration of the audio signal exceeds a duration threshold and the overall index value exceeds the expected range of the index value, the audio signal is determined to meet the segmentation conditions. If the duration of the audio signal exceeds the duration threshold and the audio signal meets the segmentation difference prediction condition, then the audio signal is determined to meet the segmentation splitting condition.

4. The method according to claim 2, characterized in that, The method further includes: If the duration of the audio signal exceeds a duration threshold, the audio signal is determined to meet the optimization conditions. If the duration of the audio signal is less than or equal to the duration threshold and the overall index value exceeds the expected range of the index value, the audio signal is determined to meet the optimization conditions.

5. The method according to claim 1, characterized in that, The audio signal includes multiple sub-audio signals; the adjustment value includes the sub-adjustment value of at least one target sub-audio signal in each of the sub-audio signals; The step of adjusting the audio signal according to the adjustment value to obtain the optimized audio signal includes: For each target sub-audio segment, adjust the target sub-audio according to the sub-adjustment value corresponding to the target sub-audio to obtain the updated sub-audio signal; With all target sub-audio segments adjusted, the optimized audio signal is obtained by splicing the current sub-audio signals.

6. The method according to claim 5, characterized in that, The process of obtaining an optimized audio signal by splicing together the current audio segments includes: Determine a smooth window that is adapted to the sub-audio signal; For each segment of the current sub-audio signal, adjacent segments in each sub-audio signal are spliced ​​together, and the segment boundaries of the adjacent segments are smoothly adjusted according to the smoothing window to obtain the optimized audio signal.

7. The method according to claim 1, characterized in that, The audio signal includes multiple sub-audio signals; the adjustment value includes the sub-adjustment value of at least one target sub-audio signal in each of the sub-audio signals; The step of using a large language model, guided by the prompt information, to invoke a skill tool adapted to the quality index and determine the adjustment value for the audio signal includes: Based on the segmentation index values ​​of each segment of the sub-audio signal, clustering processing is performed on each segment of the sub-audio signal to determine the audio cluster to which each segment of the sub-audio signal belongs; For each target audio cluster that meets the optimization conditions, the large language model, guided by the prompt information, calls upon skill tools adapted to the quality indicators to determine the sub-adjustment values ​​for each target sub-audio segment within the target audio cluster.

8. The method according to claim 1, characterized in that, When the audio signal is determined to meet the optimization conditions based on the indicator value and the expected range of the indicator value, a prompt message is constructed according to the indicator value, the expected range of the indicator value, and tool call guidance information adapted to the quality indicator, including: If the audio signal satisfies the optimization conditions based on the index value and the expected range of the index value, the acoustic mode corresponding to the audio signal is determined; the acoustic mode is used to characterize the foreground sound emission mode and background noise intensity of the audio signal. Obtain a prompt word template that matches the acoustic mode under the quality indicator; the prompt word template includes fixed content and variable placeholders; the fixed content includes the audio optimization requirements of the quality indicator under the acoustic mode, and tool call guidance information adapted to the quality indicator; The variable placeholder is filled with the indicator value and the expected range of the indicator value to obtain the prompt information.

9. The method according to claim 8, characterized in that, The prompt template includes system prompts, input content, and tool invocation guidance content; the system prompts include the audio optimization requirements; the tool invocation guidance content includes tool invocation guidance information; and the input content includes variable placeholders. The variable placeholder is filled with the indicator value and the expected range of the indicator value to obtain the prompt information, including: The indicator value and the expected range of the indicator value are filled into the variable placeholders of the input content to obtain the prompt information.

10. The method according to claim 1, characterized in that, There are multiple quality indicators; the skill tools that each quality indicator is adapted to are called by the large language model in the order of invocation. The process of determining the calling order includes: Determine the granularity of quality assessment for each of the aforementioned quality indicators for the audio signal; The order in which the appropriate skill tools are called for each quality indicator is determined according to the quality assessment granularity from coarse to fine.

11. The method according to claim 1, characterized in that, The method further includes: Determine the adjustment method for the quality indicator, and the multiple adjustment items that affect the quality indicator under the adjustment method; Based on each of the aforementioned adjustment items, a skill tool adapted to the quality indicator is registered; wherein the output of the skill tool includes the adjustment value corresponding to each of the aforementioned adjustment items.

12. The method according to any one of claims 1 to 11, characterized in that, The step of adjusting the audio signal according to the adjustment value to obtain the optimized audio signal includes: Adjust the audio signal according to the adjustment value to obtain an updated audio signal; return to the step of performing quality detection on the audio signal according to the quality index; If the iteration termination condition is not met, an adjustment record for the current iteration round is generated and added to the prompt information for the next iteration round; the adjustment record includes the most recently used adjustment value and the most recently detected index value; If the iteration termination condition is met, the optimized audio signal is obtained.

13. The method according to any one of claims 1 to 11, characterized in that, The method further includes: The optimized audio signal is then converted into text to obtain text information. If the conversion quality meets the requirements based on the text information, an audio optimization experience containing each of the adjustment values ​​is generated; the audio optimization experience is used as a reference in the inference process of the large language model. If the conversion quality does not meet the requirements based on the text information, determine the reason for the failure; If the failure reason is related to the quality indicator, then update at least a portion of the prompt information and return to the step of calling the skill tool through the large language model; If the cause of failure is related to other indicators besides the quality indicator, then the other indicators are used as new quality indicators, and the process returns to the step of obtaining the expected range of the audio signal's value under the new quality indicator.

14. A data migration device, characterized in that, The device includes: The acquisition module is used to acquire the audio signal to be processed, and the expected range of the index value of the audio signal under at least one quality index. The detection module is used to perform quality detection on the audio signal according to the quality index, and obtain the index value of the audio signal corresponding to the quality index; The prompt information construction module is used to construct prompt information based on the indicator value, the expected range of the indicator value, and tool call guidance information adapted to the quality indicator when it is determined that the audio signal meets the optimization conditions based on the indicator value and the expected range of the indicator value. The adjustment value determination module is used to determine the adjustment value for the audio signal by calling a skill tool adapted to the quality index under the guidance of the prompt information through a large language model. An audio optimization module is used to adjust the audio signal according to the adjustment value to obtain an optimized audio signal.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 13.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 13.

Images