Audio processing method and device, electronic equipment and storage medium

Abstract

The application provides an audio processing method and device, electronic equipment and storage medium. The method comprises: obtaining configuration parameters of a selected sound mode of a target device, the configuration parameters comprising: intelligent equalization parameters, reverberation parameters and sound field expansion parameters; collecting environmental information of an environment in which the target device is located, the environmental information comprising room acoustic characteristic information and user viewing distance information; calibrating the reverberation parameters according to the room acoustic characteristic information, and calibrating the sound field expansion parameters according to the user viewing distance information; performing frequency response adjustment on input audio of the target device according to the intelligent equalization parameters, superimposing a reverberation effect on the input audio according to the calibrated reverberation parameters, and performing virtual sound field expansion processing on the input audio according to the calibrated sound field expansion parameters to obtain target audio. Thus, the practical value and user experience of the sound mode function are effectively improved.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an audio processing method, apparatus, electronic device and storage medium. Background Technology

[0002] Currently, televisions typically offer multiple sound modes to adapt to the sound playback needs of different program scenarios, such as standard mode, music mode, news mode, cinema mode, sports mode, and game mode. Standard mode is generally the factory default mode and is suitable for most playback scenarios; other modes are optimized for specific scenarios. For example, cinema mode enhances the immersive experience of watching movies, music mode restores the original sound quality of music recordings, and news mode improves voice clarity. These sound modes provide users with choices, allowing them to switch to the appropriate mode based on the currently playing content for a better listening experience.

[0003] In existing technologies, the design of the aforementioned sound modes typically relies on only one set of equalizers. During the design process, several center frequencies representing different frequency ranges are selected within the human hearing range (20Hz–20kHz), and a gain value corresponding to each center frequency is set for each sound mode. By differentiating the gain values ​​across various frequency bands between different modes, distinct frequency response curves are formed, thereby achieving timbre differences between modes.

[0004] However, in the aforementioned existing technologies, each sound mode is distinguished only by adjusting the frequency response characteristics through an equalizer. The human ear has limited perception of subtle differences in frequency response curves, and the differences in gain curves between different modes often make it difficult to form a clear auditory distinction, resulting in low differentiation between modes and reducing the practical value of the sound mode function. Summary of the Invention

[0005] This application provides an audio processing method, apparatus, electronic device, and storage medium to solve the problem that in the prior art, the sound modes are distinguished by adjusting the frequency response characteristics with an equalizer, resulting in indistinct differences in the listening experience between the modes and reduced practical value of the sound mode function.

[0006] In a first aspect, this application provides an audio processing method, including: Obtain the configuration parameters of the currently selected sound mode of the target device, including: intelligent equalization parameters, reverberation parameters, and sound field expansion parameters; Collect environmental information about the environment in which the target device is located, including room acoustic characteristics information and user viewing distance information; The reverberation parameters are calibrated based on the room acoustic characteristics information, and the sound field extension parameters are calibrated based on the user viewing distance information; The frequency response of the input audio to the target device is adjusted according to the intelligent equalization parameters, a reverberation effect is superimposed on the input audio according to the calibrated reverberation parameters, and a virtual sound field expansion is performed on the input audio according to the calibrated sound field expansion parameters to obtain the target audio.

[0007] In one possible implementation, the intelligent equalization parameters include a preset target frequency response curve and filter parameters, and the method further includes: Monitor the real-time frequency response distribution of the target audio; The real-time frequency response distribution is compared with the preset target frequency response curve to obtain the comparison result; The filter parameters are dynamically adjusted based on the comparison results, and the adjusted filter parameters are used to adjust the frequency response of subsequent input audio.

[0008] In one possible implementation, calibrating the reverberation parameters based on the room acoustic characteristics information includes: Based on the room acoustic characteristics information, calculate the actual reverberation time of the room where the target device is located; The actual reverberation time is compared with the preset reference reverberation time to obtain the comparison result, and a reverberation calibration coefficient is generated based on the comparison result. The default values ​​of the reverberation parameters are scaled using the reverberation calibration coefficient to obtain the calibrated reverberation parameters.

[0009] In one possible implementation, calculating the actual reverberation time of the room where the target device is located based on the room acoustic characteristics information includes: The target device's image acquisition device is used to acquire image information of the room, and the room's geometric dimensions and the material information of each wall surface are extracted from the image information. The material information of each wall surface is input into a pre-trained neural network model, and the neural network model outputs an estimated value of the sound absorption coefficient of each wall surface. The actual reverberation time of the room is calculated based on the room's geometric dimensions and the estimated sound absorption coefficients of each wall surface.

[0010] In one possible implementation, generating reverberation calibration coefficients based on the comparison results includes: When the comparison result indicates that the actual reverberation time is greater than the preset reference reverberation time, a reverberation calibration coefficient less than 1 is generated; When the comparison result indicates that the actual reverberation time is less than the preset reference reverberation time, a reverberation calibration coefficient greater than 1 is generated; When the comparison result indicates that the actual reverberation time is equal to the preset reference reverberation time, a reverberation calibration coefficient equal to 1 is generated.

[0011] In one possible implementation, calibrating the sound field extension parameters based on the user viewing distance information includes: Retrieve head-related transfer function data corresponding to the user's viewing distance information from the head-related transfer function database; Based on the head-related transfer function data, determine the sound field spread intensity calibration value; The default values ​​of the sound field expansion parameters are calibrated using the sound field expansion intensity calibration value to obtain the calibrated sound field expansion parameters.

[0012] In one possible implementation, collecting environmental information about the environment in which the target device is located includes: The target device acquires image information of the room using its image acquisition device, obtains acoustic characteristic information of the room based on the image information, and detects the user's viewing distance relative to the target device using the image acquisition device and / or distance sensing device of the target device, thereby obtaining the user's viewing distance information.

[0013] Secondly, this application provides an audio processing apparatus, comprising: The acquisition module is used to acquire the configuration parameters of the currently selected sound mode of the target device. The configuration parameters include: intelligent equalization parameters, reverberation parameters, and sound field expansion parameters. The acquisition module is used to acquire environmental information of the environment in which the target device is located, including room acoustic characteristics information and user viewing distance information; The calibration module is used to calibrate the reverberation parameters based on the room acoustic characteristics information, and to calibrate the sound field extension parameters based on the user viewing distance information; The processing module is used to adjust the frequency response of the input audio of the target device according to the intelligent equalization parameters, add a reverberation effect to the input audio according to the calibrated reverberation parameters, and perform virtual sound field expansion processing on the input audio according to the calibrated sound field expansion parameters to obtain the target audio.

[0014] In one possible implementation, the intelligent equalization parameters include a preset target frequency response curve and filter parameters, and the device further includes a monitoring module for: Monitor the real-time frequency response distribution of the target audio; The real-time frequency response distribution is compared with the preset target frequency response curve to obtain the comparison result; The filter parameters are dynamically adjusted based on the comparison results, and the adjusted filter parameters are used to adjust the frequency response of subsequent input audio.

[0015] In one possible implementation, the calibration module is specifically used for: Based on the room acoustic characteristics information, calculate the actual reverberation time of the room where the target device is located; The actual reverberation time is compared with the preset reference reverberation time to obtain the comparison result, and a reverberation calibration coefficient is generated based on the comparison result. The default values ​​of the reverberation parameters are scaled using the reverberation calibration coefficient to obtain the calibrated reverberation parameters.

[0016] In one possible implementation, the calibration module is further configured to: The target device's image acquisition device is used to acquire image information of the room, and the room's geometric dimensions and the material information of each wall surface are extracted from the image information. The material information of each wall surface is input into a pre-trained neural network model, and the neural network model outputs an estimated value of the sound absorption coefficient of each wall surface. The actual reverberation time of the room is calculated based on the room's geometric dimensions and the estimated sound absorption coefficients of each wall surface.

[0017] In one possible implementation, the calibration module is further configured to: When the comparison result indicates that the actual reverberation time is greater than the preset reference reverberation time, a reverberation calibration coefficient less than 1 is generated; When the comparison result indicates that the actual reverberation time is less than the preset reference reverberation time, a reverberation calibration coefficient greater than 1 is generated; When the comparison result indicates that the actual reverberation time is equal to the preset reference reverberation time, a reverberation calibration coefficient equal to 1 is generated.

[0018] In one possible implementation, the calibration module is further configured to: Retrieve head-related transfer function data corresponding to the user's viewing distance information from the head-related transfer function database; Based on the head-related transfer function data, determine the sound field spread intensity calibration value; The default values ​​of the sound field expansion parameters are calibrated using the sound field expansion intensity calibration value to obtain the calibrated sound field expansion parameters.

[0019] In one possible implementation, the acquisition module is specifically used for: The target device acquires image information of the room using its image acquisition device, obtains acoustic characteristic information of the room based on the image information, and detects the user's viewing distance relative to the target device using the image acquisition device and / or distance sensing device of the target device, thereby obtaining the user's viewing distance information.

[0020] Thirdly, this application provides an electronic device, including: a processor and a memory, wherein the processor is configured to execute an audio processing program stored in the memory to implement the audio processing method described in any one of the first aspects.

[0021] Fourthly, this application provides a storage medium storing one or more programs that can be executed by one or more processors to implement the audio processing method described in any one aspect.

[0022] Fifthly, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the audio processing methods described above.

[0023] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application, by acquiring the intelligent equalization parameters, reverberation parameters, and sound field expansion parameters of the currently selected sound mode, achieves the shaping of sound effects from multiple dimensions such as timbre, spatial sense, and surround sound. This overcomes the limitation of the prior art, which relies solely on a single equalizer to adjust frequency response characteristics, and significantly improves the sound performance of each mode. Based on this, by collecting environmental information of the target device's environment (including room acoustic characteristics and user viewing distance information), and calibrating the reverberation parameters using the room acoustic characteristics information and the sound field expansion parameters using the user viewing distance information, the preset sound mode parameters can adaptively match the user's actual listening environment and viewing distance, ensuring that each mode presents the optimal auditory effect in different user environments. Finally, based on the calibrated reverberation parameters and sound field expansion parameters, the input audio is superimposed with a reverberation effect and subjected to virtual sound field expansion processing, resulting in significant auditory differences between each sound mode in multiple dimensions. Users can clearly perceive the changes in sound effects when switching modes, thereby effectively improving the practical value of the sound mode function and the user experience. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0027] Figure 1 A flowchart illustrating an embodiment of an audio processing method provided in this application; Figure 2 A flowchart illustrating an embodiment of another audio processing method provided in this application; Figure 3 A flowchart illustrating another embodiment of the audio processing method provided in this application; Figure 4 A block diagram illustrating an embodiment of an audio processing apparatus provided in this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0030] To address the technical problem in existing technologies that rely solely on equalizer adjustments to frequency response characteristics to differentiate sound modes, resulting in unclear auditory differences between modes and reduced practical value of the sound mode function, this application provides an audio processing method that enables distinct auditory differences between sound modes across multiple dimensions. Users can clearly perceive changes in sound effects when switching modes, thereby effectively enhancing the practical value of the sound mode function and improving the user experience.

[0031] Figure 1 This is a flowchart illustrating an embodiment of an audio processing method provided in this application. Figure 1 As shown, the method includes the following steps: S101. Obtain the configuration parameters of the currently selected sound mode of the target device. The configuration parameters include: intelligent equalization parameters, reverberation parameters, and sound field expansion parameters.

[0032] Target device: refers to an electronic device equipped with audio playback function and supporting multiple sound mode selection, such as a television, smart speaker, projector, etc. This application takes a television as an example for explanation.

[0033] Sound modes refer to a set of preset audio processing schemes designed to adapt to the sound playback needs of different program scenarios. Each sound mode corresponds to a set of configuration parameters. Common sound modes include Standard mode, Music mode, News mode, Cinema mode, Sports mode, and Game mode. Among them, Standard mode is the factory default mode and is suitable for most playback scenarios; other modes are optimized for specific scenarios, such as Cinema mode to enhance the immersive experience of movie viewing, Music mode to restore the original sound quality of music recordings, and News mode to enhance speech clarity.

[0034] Configuration parameters: refers to the set of sound effect parameters corresponding to the sound mode. Each sound mode corresponds to a set of configuration parameters, which include at least three dimensions of parameters: intelligent equalization parameters, reverberation parameters, and sound field expansion parameters.

[0035] Intelligent equalization parameters: These refer to the configuration information used to control the intelligent equalization processing stage, including preset target frequency response curves and filter parameters. The preset target frequency response curve can subjectively be categorized as balanced, warm, open, or clear. The filter parameters are the adjustable parameters that actually participate in the calculations during intelligent equalization processing.

[0036] Reverb parameters: These are configuration details used to control the reverb processing stage. Reverb refers to the continuous echo produced when sound undergoes multiple reflections within a closed or semi-closed space, before the direct sound disappears. It adds a sense of space and depth to sound, making otherwise dry, reverb-free sounds more three-dimensional and natural. Reverb parameters control the intensity of the reverb effect.

[0037] Sound field expansion parameters: These are configuration information used to control the sound field expansion processing stage. Sound field expansion refers to using digital signal processing algorithms to make sound appear wider, more three-dimensional, and more immersive in the listening environment, thus enhancing the virtual surround sound effect. Sound field expansion parameters control the intensity of the virtual sound field expansion.

[0038] In this embodiment, the target device has multiple preset sound modes and their corresponding configuration parameters. When a user switches or selects a sound mode via a remote control, device panel, or voice command, the target device responds to the user's operation command and reads the configuration parameters corresponding to that sound mode from its local memory, namely, the intelligent equalization parameter, reverberation parameter, and sound field extension parameter. These three parameters correspond to different audio processing dimensions in the sound mode design: the intelligent equalization parameter is used for timbre adjustment, the reverberation parameter for spatial adjustment, and the sound field extension parameter for surround sound adjustment. The configuration parameters of different sound modes are distinguished by combinations of different intensity levels across these three dimensions, resulting in unique auditory characteristics for each mode across multiple dimensions. After obtaining these parameters, they are sent to the corresponding processing modules for use in subsequent processing stages.

[0039] In one example, based on the intelligent equalization parameters, reverberation parameters, and sound field expansion parameters obtained by S101 above, the default intensity levels of each sound mode in the three dimensions can be set as shown in the table below (the numbers in the table represent intensity levels, with larger numbers indicating higher intensity, and "×%" representing a percentage coefficient based on the user's actual environment calibration): Table 1

[0040] In this embodiment, the default values ​​for each mode were obtained through adjustments by a professional sound engineer in a standard listening room environment. In a standard listening room environment, each sound mode can achieve optimal sound effects using the above default values. As shown in Table 1, the cinema mode uses a combination of open intelligent equalization, strong reverberation, and strong sound field extension to create an immersive viewing experience; the music mode uses only balanced intelligent equalization, turning off reverberation and sound field extension to maintain the purity and naturalness of the original recording; there are at least two dimensions of intensity level differences between any two sound modes, making the auditory distinction between modes obvious, and allowing users to clearly perceive changes in sound effects when switching modes.

[0041] S102. Collect environmental information of the environment in which the target device is located, including room acoustic characteristics information and user viewing distance information.

[0042] Environmental information refers to various parameters related to the audio playback effect in the physical environment in which the target device is currently located, including at least room acoustic characteristics and user viewing distance information.

[0043] Room acoustic properties information: refers to the set of parameters used to characterize the acoustic properties of the room where the user is currently located. It includes relevant parameters that affect sound reflection and reverberation characteristics, such as the geometric dimensions of the room and the material information of each wall.

[0044] User viewing distance information: refers to the user's position parameters relative to the target device, that is, the straight-line distance between the user and the target device.

[0045] In this embodiment, the target device uses its equipped environmental sensors to collect the aforementioned environmental information. For room acoustic characteristics, the sensors can collect relevant physical information about the user's current room, extracting acoustic parameters such as the room's geometric dimensions and wall material information. For the user's viewing distance, the sensors can detect the user's distance relative to the target device. These sensors can be image acquisition devices (such as cameras), distance sensing devices (such as radar sensors), or a combination of both. The collected environmental information is used for subsequent adaptive calibration of reverberation parameters and sound field spread parameters.

[0046] S103. The reverberation parameters are calibrated according to the room acoustic characteristics information, and the sound field extension parameters are calibrated according to the user viewing distance information.

[0047] In this embodiment, the environmental information collected in S102 is used to adaptively calibrate the reverberation parameters and sound field expansion parameters so that the preset sound mode parameters can match the user's current actual listening environment.

[0048] For reverberation parameters, the default values ​​are calibrated based on the room's acoustic characteristics. Since different rooms have different geometric dimensions and wall finishes, their reverberation characteristics vary. Directly applying the default reverberation parameters obtained from calibration in a standard listening room to the user's actual room may not achieve the desired reverberation effect. Therefore, this step uses the collected room acoustic characteristics information to adaptively adjust the reverberation parameters, ensuring that the calibrated parameters adapt to the actual acoustic characteristics of the user's current room, avoiding issues of excessive or insufficient reverberation due to room differences.

[0049] For the sound field extension parameters, the default values ​​are calibrated based on the user's viewing distance information. The perceived intensity of the sound field extension is directly related to the user's viewing distance. A stronger sound field extension is needed to create a sense of immersion when the viewing distance is far, while a moderate sound field extension is sufficient for a surround effect when the viewing distance is close. Since the default values ​​of the sound field extension parameters obtained under standard listening room conditions are based on a fixed viewing distance and cannot adapt to the actual viewing positions of all users, this step uses the collected user viewing distance information to adaptively adjust the sound field extension parameters so that the calibrated sound field extension parameters can adapt to the user's current viewing distance.

[0050] Through the above calibration, the preset sound mode parameters can adaptively match the user's actual listening environment, ensuring that the best listening experience can be obtained in different user environments.

[0051] S104. Adjust the frequency response of the input audio of the target device according to the intelligent equalization parameters, add a reverberation effect to the input audio according to the calibrated reverberation parameters, and perform virtual sound field expansion processing on the input audio according to the calibrated sound field expansion parameters to obtain the target audio.

[0052] Input audio: refers to the audio signal to be played received by the target device, such as the audio source signal obtained by a TV from the HDMI interface, network streaming media, USB interface, or internal player.

[0053] Target audio: refers to the final audio signal output after processing through three dimensions of audio processing.

[0054] In this embodiment, the input audio is processed according to the three calibrated dimensions. The processing of the three dimensions can be performed in parallel or in any order. Finally, the audio signals processed by the three dimensions are synthesized to output the target audio.

[0055] Intelligent equalization processing: The input audio frequency response is adjusted according to intelligent equalization parameters to match the frequency response characteristics of the audio signal with a preset target frequency response curve. Specifically, the intelligent equalization parameters include the preset target frequency response curve and filter parameters. During processing, the gain of each frequency band of the input audio is adjusted through the filter so that the frequency response distribution of the output audio approximates the preset target frequency response curve, thereby achieving subjective listening sensations such as balanced, warm, open, or clear.

[0056] Reverb Processing: This process adds a reverb effect to the input audio based on calibrated reverb parameters. Reverb refers to the continuous echo produced when sound undergoes multiple reflections in a closed or semi-closed space, before the direct sound disappears. It can add a sense of space and depth to sound. Reverb processing uses digital signal processing algorithms to simulate reflected sound sequences under different spatial environments, then superimposes the generated reverb signal onto the original audio signal, making the processed audio sound more three-dimensional and natural.

[0057] Sound field expansion processing: Virtual sound field expansion is applied to the input audio based on calibrated sound field expansion parameters. Sound field expansion refers to the reprocessing of binaural audio signals using digital signal processing algorithms (such as crosstalk cancellation and virtual surround sound techniques) to create a wider, more three-dimensional, and more immersive virtual sound image in the listening environment. During processing, the corresponding head-related transfer function or virtual surround algorithm is called based on the sound field expansion parameters to filter and spatially remap the input audio, enabling the sound played through the target device's speakers to produce a virtual sound field effect that exceeds the physical width of the speakers.

[0058] The technical solution provided in this application, by acquiring the intelligent equalization parameters, reverberation parameters, and sound field expansion parameters of the currently selected sound mode, achieves a multi-dimensional shaping of sound effects from timbre, spatiality, and surround sound. This overcomes the limitations of existing technologies that rely solely on a single equalizer to adjust frequency response characteristics, significantly improving the sound performance of each mode. Furthermore, by collecting environmental information about the target device's environment (including room acoustic characteristics and user viewing distance), and calibrating the reverberation parameters using the room acoustic characteristics and the sound field expansion parameters using the user viewing distance, the preset sound mode parameters can adaptively match the user's actual listening environment and viewing distance, ensuring that each mode presents the optimal auditory effect in different user environments. Finally, based on the calibrated reverberation and sound field expansion parameters, a reverberation effect is superimposed on the input audio, and virtual sound field expansion processing is performed. This creates significant auditory differences between the various sound modes in multiple dimensions, allowing users to clearly perceive changes in sound effects when switching modes, thereby effectively enhancing the practical value and user experience of the sound mode function.

[0059] In one embodiment, the intelligent equalization parameters include a preset target frequency response curve and filter parameters. To maintain the tonal balance stability of the output audio, the method may further include the following steps: monitoring the real-time frequency response distribution of the target audio; comparing the real-time frequency response distribution with the preset target frequency response curve to obtain a comparison result; dynamically adjusting the filter parameters according to the comparison result, and using the adjusted filter parameters to adjust the frequency response of subsequent input audio.

[0060] Preset target frequency response curve: This refers to the target frequency response characteristic curve that the intelligent equalization processing aims to achieve, used as a benchmark for comparing the actual output audio frequency response distribution. Subjectively, the preset target frequency response curve can include types such as balanced, warm, open, and clear, each corresponding to different tonal styles.

[0061] Filter parameters: These are the adjustable parameters that actually participate in the calculations during intelligent equalization processing. By adjusting these parameters, the gain values ​​of each frequency band of the audio signal can be changed, thereby achieving different frequency response characteristics.

[0062] In this embodiment, based on the configuration parameters obtained above, the intelligent equalization parameters further include a preset target frequency response curve and filter parameters. The intelligent equalization process is not a one-time adjustment that remains fixed; instead, it continuously monitors the real-time frequency response distribution of the final output target audio, compares the real-time frequency response distribution with the preset target frequency response curve band by band, calculates the deviation between the real-time frequency response value and the target frequency response value for each band, and determines the gain value to be adjusted for each band based on the deviation. Specifically, if the real-time frequency response value of a certain band is higher than the target frequency response value for that band, the gain value of that band is reduced accordingly; if the real-time frequency response value of a certain band is lower than the target frequency response value for that band, the gain value of that band is increased accordingly; if the real-time frequency response value of a certain band is equal to the target frequency response value for that band, the gain value of that band remains unchanged. The filter parameters are dynamically adjusted based on the adjusted gain values ​​for each band, and the adjusted filter parameters are used to adjust the frequency response of subsequent input audio. Thus, intelligent equalization processing forms a continuous feedback adjustment mechanism, which enables the filter parameters to be continuously updated according to the deviation between the actual output effect and the preset target, ensuring that the frequency response distribution of the output audio continuously approaches the preset target frequency response curve.

[0063] By using the above methods, we can avoid timbre shift caused by changes in input audio characteristics or environmental interference, maintain the timbre balance and stability of the output audio, and further improve the listening consistency of each sound mode under different playback content.

[0064] In one embodiment, to obtain room acoustic characteristics information and user viewing distance information, S102 may specifically include the following steps: acquiring image information of the room using the image acquisition device of the target device, obtaining room acoustic characteristics information based on the image information, and detecting the user's viewing distance relative to the target device using the image acquisition device and / or distance sensing device of the target device to obtain the user viewing distance information.

[0065] Distance sensing device: refers to hardware device used to detect the distance between a target and a sensor, such as a radar sensor, which can directly measure the distance between the user and the target device by emitting and receiving reflected waves.

[0066] In this embodiment, the specific process of collecting environmental information in S102 is as follows: The image acquisition device (e.g., a camera) of the target device is used to acquire image information of the room where the user is currently located. Image recognition processing is performed on the acquired image information to extract the geometric dimensions of the room (e.g., the length, width, and height of the room) and the material information of each wall surface (e.g., the decoration materials of the walls, floor, and ceiling), thereby obtaining the room's acoustic characteristics information. Simultaneously, the image acquisition device and / or distance sensing device of the target device are used to detect the user's viewing distance relative to the target device: when using an image acquisition device, the distance between the user and the device can be calculated through facial recognition or human posture estimation algorithms; when using a distance sensing device (e.g., a radar sensor), the distance between the user and the device can be directly measured by transmitting and receiving reflected waves; both devices can also be used simultaneously to improve detection accuracy or achieve mutual verification, thereby obtaining the user's viewing distance information.

[0067] Using the above method, environmental information can be automatically and conveniently collected without the user having to manually input any environmental parameters. This provides a data basis for the adaptive calibration of reverberation parameters and sound field extension parameters, improving the ease of use for the user.

[0068] Figure 2 A flowchart illustrating an embodiment of another audio processing method provided in this application. Figure 2 The process shown is in Figure 1 Based on the illustrated process, the following steps are included: S201. Calculate the actual reverberation time of the room where the target device is located based on the room acoustic characteristics information; S202. Compare the actual reverberation time with the preset reference reverberation time to obtain the comparison result, and generate a reverberation calibration coefficient based on the comparison result. S203. Scale the default value of the reverberation parameter using the reverberation calibration coefficient to obtain the calibrated reverberation parameter.

[0069] For ease of understanding, the following provides a unified explanation of S201-203: Actual reverberation time: refers to the time required for sound to decay by 60dB in the user's current room, reflecting the actual reverberation characteristics of the room.

[0070] Reference reverberation time: refers to the reference reverberation time value corresponding to each sound mode in a standard listening room environment, which is determined in advance by the engineer through subjective listening and adjustment and stored in the device.

[0071] Reverberation calibration factor: refers to the scaling factor used to scale the default values ​​of the reverberation parameters.

[0072] In this embodiment, the specific process of calibrating the reverberation parameters in step S103 is as follows: First, based on the room acoustic characteristic information collected in S102, the actual reverberation time of the user's current room is calculated. The actual reverberation time can be calculated using a reverberation time algorithm in the acoustic field, based on the room's geometric dimensions and the acoustic characteristics (such as sound absorption performance) of each wall surface. After obtaining the actual reverberation time, it is compared with a preset reference reverberation time, and a reverberation calibration coefficient is generated based on the comparison result. Finally, the default value of the reverberation parameters is scaled using this reverberation calibration coefficient (e.g., the default value is multiplied by the reverberation calibration coefficient) to obtain the calibrated reverberation parameters.

[0073] Through the above calibration, the intensity of the reverberation effect can be adaptively adjusted according to the acoustic characteristics of the user's actual room, avoiding the problem of excessive or insufficient reverberation due to room differences, and ensuring that the optimal reverberation listening effect can be obtained in different user environments.

[0074] In one embodiment, to facilitate the calculation of reverberation time, S201 may specifically include the following steps: acquiring image information of the room using the image acquisition device of the target device, extracting room geometric dimension information and material information of each wall from the image information; inputting the material information of each wall into a pre-trained neural network model, and having the neural network model output an estimated value of the sound absorption coefficient of each wall; calculating the actual reverberation time of the room based on the room geometric dimension information and the estimated value of the sound absorption coefficient of each wall.

[0075] Image acquisition device: refers to hardware devices used to acquire image information, such as cameras built into or connected to a television.

[0076] Neural network model: refers to a pre-trained deep learning model used to output an estimated value of the sound absorption coefficient based on the input wall material information.

[0077] Sound absorption coefficient estimate: refers to the parameter used to characterize the ability of a wall to absorb sound energy. Its value is between 0 and 1, where 1 represents total absorption and 0 represents total reflection.

[0078] In this embodiment, the specific process of calculating the actual reverberation time in S201 is as follows: The image acquisition device (such as a camera) of the target device acquires image information of the room where the user is currently located. The geometric dimensions of the room (i.e., the length, width, and height of the room) and the material information of each wall surface (e.g., walls are painted with latex paint, wallpaper, or wood decoration; floors are tiled, wood flooring, or carpet; ceilings are plasterboard or suspended ceilings, etc.) are extracted from the image information. The extracted material information of each wall surface is input into a pre-trained neural network model. The neural network model outputs an estimated value of the sound absorption coefficient of each wall surface based on the correspondence between the material and the sound absorption coefficient. After obtaining the geometric dimensions of the room and the estimated values ​​of the sound absorption coefficient of each wall surface, for a rectangular room, the actual reverberation time can be calculated using the Fitzroy formula:

[0079] Where T is the reverberation time, V is the room volume, and S is the total area of ​​the room's interior walls. These represent the areas of two pairs of parallel walls that are opposite each other along the length, width, and height of the room. These are the estimated sound absorption coefficients for the three pairs of parallel walls mentioned above.

[0080] Using the above method, the actual reverberation time of a user's room can be obtained automatically and conveniently without the need for the user to manually input room parameters or use professional measuring equipment, providing an accurate data basis for subsequent adaptive calibration of reverberation parameters.

[0081] In one embodiment, to determine the adjustment strategy for the reverberation parameters, S202 may specifically include the following steps: when the comparison result indicates that the actual reverberation time is greater than the preset reference reverberation time, a reverberation calibration coefficient less than 1 is generated; when the comparison result indicates that the actual reverberation time is less than the preset reference reverberation time, a reverberation calibration coefficient greater than 1 is generated; when the comparison result indicates that the actual reverberation time is equal to the preset reference reverberation time, a reverberation calibration coefficient equal to 1 is generated.

[0082] In this embodiment, when the comparison result indicates that the actual reverberation time is greater than the preset reference reverberation time, it means that the user's current room is relatively spacious or has strong wall reflections. If the default reverberation parameter obtained from debugging in a standard listening room environment is directly used, the actual reverberation will be too strong and the sound will be muddy. Therefore, it is necessary to reduce the reverberation intensity. At this time, a reverberation calibration coefficient less than 1 is generated to reduce the default value. When the comparison result indicates that the actual reverberation time is less than the preset reference reverberation time, it means that the user's current room has heavy sound absorption and insufficient reflection. If the default value is directly used, it will result in insufficient reverberation and a dry sound. Therefore, it is necessary to increase the reverberation intensity. At this time, a reverberation calibration coefficient greater than 1 is generated to amplify the default value. When the comparison result indicates that the actual reverberation time is equal to the preset reference reverberation time, it means that the reverberation characteristics of the user's room are consistent with those of a standard listening room and no adjustment is needed. At this time, a reverberation calibration coefficient equal to 1 is generated to keep the default value unchanged.

[0083] By using the above comparison and calibration rules, the reverberation parameters are adaptively adjusted according to the acoustic characteristics of the room. This allows the reverberation calibration coefficient to be scaled in a targeted manner according to the deviation direction between the actual reverberation time and the reference value, ensuring that a suitable reverberation effect can be obtained in rooms with different acoustic environments.

[0084] Figure 3 A flowchart illustrating another embodiment of the audio processing method provided in this application. Figure 3 The process shown is in Figure 1 Based on the illustrated process, the following steps are included: S301. Retrieve head-related transfer function data corresponding to the user viewing distance information from the head-related transfer function database; S302. Determine the sound field spread intensity calibration value based on the head-related transfer function data; S303. The default value of the sound field expansion parameter is calibrated using the sound field expansion intensity calibration value to obtain the calibrated sound field expansion parameter.

[0085] For ease of understanding, S301-S303 are explained below: Head-Related Transfer Function (HRTF) database: This refers to a collection of pre-stored head-related transfer function (HRTF) data under different distance conditions. The HRTF describes the acoustic transfer function of sound pressure transmission from a sound source to the eardrum. It reflects the spectral characteristics formed after diffraction, reflection, and scattering of sound waves through the head, auricle, and trunk during their propagation from the sound source to the ear. Under free-field conditions, the normalized head-related transfer function from a point sound source to both ears is defined as:

[0086] in, Indicates the left or right ear. This indicates the sound pressure produced at the eardrum of the listener's left or right ear. This represents the reference sound pressure level measured at the original center of the listener's head after the listener's head has been moved away. Indicates the distance from the sound source to the center of the head. This indicates the azimuth angle of the sound source relative to the center of the head. The frequency is represented. The head-related transfer function data relative to the center position of the target device can be measured in advance using measuring equipment (such as an artificial head) at different distances in front of the target device (such as typical distances of 1 meter, 2 meters, 3 meters, 4 meters, 5 meters, etc.), and the measured data can be saved to the database.

[0087] Sound field extension intensity calibration value: refers to the value determined based on the head-related transfer function data corresponding to the current viewing distance, used to calibrate the default values ​​of the sound field extension parameters.

[0088] In this embodiment, the specific process of calibrating the sound field extension parameters in S103 is as follows: First, based on the user viewing distance information collected in S102, head-related transfer function data corresponding to the current viewing distance is retrieved from the head-related transfer function database. The sound field extension intensity is mainly related to the user's actual viewing distance. The farther the viewing distance, the smaller the angle between the speaker and the user, requiring a stronger sound field extension to create a sense of immersion; the closer the viewing distance, the larger the speaker angle, and a moderate sound field extension is sufficient to achieve a surround effect. Then, the sound field extension intensity calibration value is determined based on the retrieved head-related transfer function data. Finally, the default value of the sound field extension parameters is calibrated using this sound field extension intensity calibration value to obtain the calibrated sound field extension parameters. Through the above calibration, the sound field extension intensity can be adaptively adjusted according to the user's actual viewing distance, ensuring that the optimal virtual surround effect can be obtained at different viewing distances, avoiding problems such as insufficient immersion or over-processing caused by mismatch between the viewing distance and the default sound field extension intensity.

[0089] Figure 4 This is a block diagram illustrating an embodiment of an audio processing apparatus provided in this application. Figure 4 As shown, the device includes: The acquisition module 41 is used to acquire the configuration parameters of the currently selected sound mode of the target device. The configuration parameters include: intelligent equalization parameters, reverberation parameters and sound field expansion parameters. The acquisition module 42 is used to acquire environmental information of the environment in which the target device is located, including room acoustic characteristics information and user viewing distance information; The calibration module 43 is used to calibrate the reverberation parameters according to the room acoustic characteristics information, and to calibrate the sound field extension parameters according to the user viewing distance information; The processing module 44 is used to adjust the frequency response of the input audio of the target device according to the intelligent equalization parameters, add a reverberation effect to the input audio according to the calibrated reverberation parameters, and perform virtual sound field expansion processing on the input audio according to the calibrated sound field expansion parameters to obtain the target audio.

[0090] In one possible implementation, the intelligent equalization parameters include a preset target frequency response curve and filter parameters, and the device further includes a monitoring module for: Monitor the real-time frequency response distribution of the target audio; The real-time frequency response distribution is compared with the preset target frequency response curve to obtain the comparison result; The filter parameters are dynamically adjusted based on the comparison results, and the adjusted filter parameters are used to adjust the frequency response of subsequent input audio.

[0091] In one possible implementation, the calibration module is specifically used for: Based on the room acoustic characteristics information, calculate the actual reverberation time of the room where the target device is located; The actual reverberation time is compared with the preset reference reverberation time to obtain the comparison result, and a reverberation calibration coefficient is generated based on the comparison result. The default values ​​of the reverberation parameters are scaled using the reverberation calibration coefficient to obtain the calibrated reverberation parameters.

[0092] In one possible implementation, the calibration module is further configured to: The target device's image acquisition device is used to acquire image information of the room, and the room's geometric dimensions and the material information of each wall surface are extracted from the image information. The material information of each wall surface is input into a pre-trained neural network model, and the neural network model outputs an estimated value of the sound absorption coefficient of each wall surface. The actual reverberation time of the room is calculated based on the room's geometric dimensions and the estimated sound absorption coefficients of each wall surface.

[0093] In one possible implementation, the calibration module is further configured to: When the comparison result indicates that the actual reverberation time is greater than the preset reference reverberation time, a reverberation calibration coefficient less than 1 is generated; When the comparison result indicates that the actual reverberation time is less than the preset reference reverberation time, a reverberation calibration coefficient greater than 1 is generated; When the comparison result indicates that the actual reverberation time is equal to the preset reference reverberation time, a reverberation calibration coefficient equal to 1 is generated.

[0094] In one possible implementation, the calibration module is further configured to: Retrieve head-related transfer function data corresponding to the user's viewing distance information from the head-related transfer function database; Based on the head-related transfer function data, determine the sound field spread intensity calibration value; The default values ​​of the sound field expansion parameters are calibrated using the sound field expansion intensity calibration value to obtain the calibrated sound field expansion parameters.

[0095] In one possible implementation, the acquisition module is specifically used for: The target device acquires image information of the room using its image acquisition device, obtains acoustic characteristic information of the room based on the image information, and detects the user's viewing distance relative to the target device using the image acquisition device and / or distance sensing device of the target device, thereby obtaining the user's viewing distance information.

[0096] like Figure 5 As shown in the figure, this application provides a device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the audio processing method provided in any of the foregoing method embodiments, including: Obtain the configuration parameters of the currently selected sound mode of the target device, including: intelligent equalization parameters, reverberation parameters, and sound field expansion parameters; Collect environmental information about the environment in which the target device is located, including room acoustic characteristics information and user viewing distance information; The reverberation parameters are calibrated based on the room acoustic characteristics information, and the sound field extension parameters are calibrated based on the user viewing distance information; The frequency response of the input audio to the target device is adjusted according to the intelligent equalization parameters, a reverberation effect is superimposed on the input audio according to the calibrated reverberation parameters, and a virtual sound field expansion is performed on the input audio according to the calibrated sound field expansion parameters to obtain the target audio.

[0097] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the audio processing method provided in any of the foregoing method embodiments.

[0098] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the audio processing methods described in the above embodiments.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0101] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0102] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An audio processing method, characterized in that, The method includes: Obtain the configuration parameters of the currently selected sound mode of the target device, including: intelligent equalization parameters, reverberation parameters, and sound field expansion parameters; Collect environmental information about the environment in which the target device is located, including room acoustic characteristics information and user viewing distance information; The reverberation parameters are calibrated based on the room acoustic characteristics information, and the sound field extension parameters are calibrated based on the user viewing distance information; The frequency response of the input audio to the target device is adjusted according to the intelligent equalization parameters, a reverberation effect is superimposed on the input audio according to the calibrated reverberation parameters, and a virtual sound field expansion is performed on the input audio according to the calibrated sound field expansion parameters to obtain the target audio.

2. The method according to claim 1, characterized in that, The intelligent equalization parameters include a preset target frequency response curve and filter parameters, and the method further includes: Monitor the real-time frequency response distribution of the target audio; The real-time frequency response distribution is compared with the preset target frequency response curve to obtain the comparison result; The filter parameters are dynamically adjusted based on the comparison results, and the adjusted filter parameters are used to adjust the frequency response of subsequent input audio.

3. The method according to claim 1, characterized in that, The calibration of the reverberation parameters based on the room acoustic characteristics information includes: Based on the room acoustic characteristics information, calculate the actual reverberation time of the room where the target device is located; The actual reverberation time is compared with the preset reference reverberation time to obtain the comparison result, and a reverberation calibration coefficient is generated based on the comparison result. The default values ​​of the reverberation parameters are scaled using the reverberation calibration coefficient to obtain the calibrated reverberation parameters.

4. The method according to claim 3, characterized in that, The step of calculating the actual reverberation time of the room where the target device is located based on the room acoustic characteristics information includes: The target device's image acquisition device is used to acquire image information of the room, and the room's geometric dimensions and the material information of each wall surface are extracted from the image information. The material information of each wall surface is input into a pre-trained neural network model, and the neural network model outputs an estimated value of the sound absorption coefficient of each wall surface. The actual reverberation time of the room is calculated based on the room's geometric dimensions and the estimated sound absorption coefficients of each wall surface.

5. The method according to claim 3, characterized in that, The step of generating reverberation calibration coefficients based on the comparison results includes: When the comparison result indicates that the actual reverberation time is greater than the preset reference reverberation time, a reverberation calibration coefficient less than 1 is generated; When the comparison result indicates that the actual reverberation time is less than the preset reference reverberation time, a reverberation calibration coefficient greater than 1 is generated; When the comparison result indicates that the actual reverberation time is equal to the preset reference reverberation time, a reverberation calibration coefficient equal to 1 is generated.

6. The method according to claim 1, characterized in that, The calibration of the sound field extension parameters based on the user viewing distance information includes: Retrieve head-related transfer function data corresponding to the user's viewing distance information from the head-related transfer function database; Based on the head-related transfer function data, determine the sound field spread intensity calibration value; The default values ​​of the sound field expansion parameters are calibrated using the sound field expansion intensity calibration value to obtain the calibrated sound field expansion parameters.

7. The method according to claim 1, characterized in that, The collection of environmental information about the environment in which the target device is located includes: The target device acquires image information of the room using its image acquisition device, obtains acoustic characteristic information of the room based on the image information, and detects the user's viewing distance relative to the target device using the image acquisition device and / or distance sensing device of the target device, thereby obtaining the user's viewing distance information.

8. An audio processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the configuration parameters of the currently selected sound mode of the target device. The configuration parameters include: intelligent equalization parameters, reverberation parameters, and sound field expansion parameters. The acquisition module is used to acquire environmental information of the environment in which the target device is located, including room acoustic characteristics information and user viewing distance information; The calibration module is used to calibrate the reverberation parameters based on the room acoustic characteristics information, and to calibrate the sound field extension parameters based on the user viewing distance information; The processing module is used to adjust the frequency response of the input audio of the target device according to the intelligent equalization parameters, add a reverberation effect to the input audio according to the calibrated reverberation parameters, and perform virtual sound field expansion processing on the input audio according to the calibrated sound field expansion parameters to obtain the target audio.

9. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute an audio processing program stored in the memory to implement the audio processing method according to any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the audio processing method according to any one of claims 1-7.

Images