Audio adjustment method and system, electronic equipment and storage medium

Abstract

The invention relates to an audio adjustment method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring audio playing information; determining a sound effect evaluation reference of the audio playing device in the audio playing environment according to the environmental parameters; inputting the sound effect evaluation reference and an initial audio parameter of the audio playing device into an adjustment model to enable the adjustment model to output an adjustment value of the initial audio parameter, and determining a parameter adjustment strategy of the audio playing device based on the adjustment value of the audio parameter; and updating the initial audio parameters of the audio playing equipment based on a parameter adjustment strategy. For a complex television playing environment, the playing sound effect is determined according to the environment parameters of the audio playing environment, then the audio playing parameters of the television are adjusted and controlled by using a deep learning model, adaptive optimization of the sound effect is realized, the audio parameters are dynamically adjusted according to the change of the environment, and the user experience is improved. And better auditory experience is brought to the user.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an audio adjustment method, system, electronic device and storage medium. Background Technology

[0002] In the field of modern television technology, consumers' expectations for television sound quality are constantly increasing. Traditional television sound adjustment mainly relies on basic equalizer settings, which can only make basic adjustments to high, mid, and low frequencies within the range of frequencies perceptible to humans. The actual listening environment is complex and diverse. Factors such as room layout, placement of furniture and appliances, and materials of walls and floors can significantly affect the propagation and reflection of sound in the space, thus affecting the final listening experience.

[0003] However, current audio adjustment technologies mainly rely on built-in ultrasonic sensors and infrared imaging devices to obtain room geometry data and material property information to adapt to the actual listening environment. Due to the limited accuracy and coverage of the sensors, it is difficult to capture subtle features in complex environments. In addition, current adjustment methods lack specificity for adaptive sound adjustment of television devices.

[0004] Therefore, there is an urgent need to develop an audio adjustment method, system, electronic device, and storage medium to solve one or more of the aforementioned problems. Summary of the Invention

[0005] In view of this, in order to solve the above-mentioned technical problems or some of the technical problems, the present invention provides an audio adjustment method, system, electronic device and storage medium.

[0006] In a first aspect, this application provides an audio adjustment method, the method comprising:

[0007] Acquire audio playback information, which includes the initial audio parameters of the audio playback device and the environmental parameters of the audio playback environment;

[0008] Based on the environmental parameters, a sound effect evaluation reference for the audio playback device in the audio playback environment is determined;

[0009] The sound effect evaluation reference and the initial audio parameters of the audio playback device are input into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and the parameter adjustment strategy of the audio playback device is determined based on the adjustment value of the initial audio parameters;

[0010] Based on the parameter adjustment strategy, the initial audio parameters of the audio playback device are updated.

[0011] In one possible implementation, the environmental parameters include a first environmental parameter and a second environmental parameter;

[0012] Obtain the environmental parameters of the audio playback environment, including:

[0013] The image acquisition system acquires image information of the audio playback environment;

[0014] The image information is analyzed to obtain the first environmental parameters of the audio playback environment;

[0015] The sound information of the audio playback environment is acquired through a sound acquisition system, and the sound information includes the phase of the sound signal in the audio playback environment;

[0016] Based on the image and sound information, a three-dimensional acoustic model of the audio playback environment is constructed.

[0017] The sound pressure of the sound signal at the target point and the first preset point within a preset time period is obtained in the three-dimensional acoustic model to obtain the second environmental parameter of the audio playback environment.

[0018] In one possible implementation, the sound effect evaluation reference includes reverberation evaluation information, clarity evaluation information, and spatial evaluation information;

[0019] The step of determining the sound effect evaluation reference of the audio playback device in the audio playback environment based on the environmental parameters includes:

[0020] Based on the first environmental parameter, determine the reverberation evaluation information of the audio playback device in the audio playback environment;

[0021] Based on the sound pressure level at each moment within a preset time period at the target location, which is included in the second environmental parameter, the clarity assessment information of the audio playback device in the audio playback environment is determined.

[0022] Based on the second environmental parameter, spatial evaluation information of the audio playback device in the audio playback environment is determined.

[0023] In one possible implementation, determining the spatial evaluation information of the audio playback device in the audio playback environment based on the second environmental parameter includes:

[0024] Based on the sound pressure at each moment within a preset time period at the target location and the first preset location, which are included in the second environmental parameters, the audible sound source width and audible envelopment of the audio playback device in the audio playback environment are determined.

[0025] Based on the auditory sound source width and auditory envelopment, spatial evaluation information of the audio playback device in the audio playback environment is obtained.

[0026] In one possible implementation, the first environmental parameters include the playback volume, playback surface area, and sound absorption coefficient of the audio playback environment.

[0027] The step of parsing the image information to obtain the first environmental parameters of the audio playback environment includes:

[0028] The image information is analyzed to obtain the size, material type, and spatial layout of all objects within the audio playback environment; wherein, the spatial layout includes the spatial position, arrangement direction, and arrangement spacing of the objects;

[0029] Based on the size and spatial layout of all objects within the audio playback environment, determine the playback volume and playback surface area of ​​the audio playback environment;

[0030] Find the sound absorption coefficient corresponding to each material type in the material sound absorption coefficient table;

[0031] The average sound absorption coefficient of the audio playback environment is determined based on the size of each material object and its corresponding sound absorption coefficient.

[0032] In one possible implementation, the step of inputting the sound effect evaluation reference and the initial audio parameters of the audio playback device into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and determining the parameter adjustment strategy of the audio playback device based on the adjustment value of the initial audio parameters, includes:

[0033] The three-dimensional acoustic model of the audio playback environment, the sound effect evaluation reference, and the initial audio parameters of the audio playback device are simultaneously input into the adjustment model, so that the adjustment model outputs the adjustment values ​​of each initial audio parameter according to the three-dimensional acoustic model, thereby obtaining the parameter adjustment strategy of the audio playback device.

[0034] In one possible implementation, the method further includes:

[0035] If the environmental parameters of the audio playback environment are detected to be updated, new audio playback information is obtained, and the step of obtaining audio playback information is re-executed.

[0036] Secondly, this application provides an audio adjustment system, the system comprising:

[0037] The acquisition module is used to acquire audio playback information, which includes the initial audio parameters of the audio playback device and the environmental parameters of the audio playback environment.

[0038] An evaluation module is used to determine a sound effect evaluation reference for the audio playback device in the audio playback environment based on the environmental parameters.

[0039] The adjustment module is used to input the sound effect evaluation reference and the initial audio parameters of the audio playback device into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and determines the parameter adjustment strategy of the audio playback device based on the adjustment value of the initial audio parameters;

[0040] An update module is used to update the initial audio parameters of the audio playback device based on the parameter adjustment strategy.

[0041] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the audio adjustment method described in any embodiment of the first aspect.

[0042] Fourthly, this application also provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the audio adjustment method described in any embodiment of the first aspect.

[0043] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application, for the complex environment of TV playback, obtains the environmental parameters of the audio playback environment, and then determines the playback sound effect of the audio playback device in the current playback environment based on the influence law of different furniture materials on sound reflection. Then, a deep learning model is used to adjust and control the audio playback parameters of the TV according to the playback sound effect, so as to realize the adaptive optimization of the sound effect, and make the audio parameters dynamically adjusted according to the changes in the environment, bringing users a better listening experience. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0047] Figure 1A flowchart illustrating an audio adjustment method provided in an embodiment of this application;

[0048] Figure 2 A flowchart illustrating a method for obtaining environmental parameters provided in an embodiment of this application;

[0049] Figure 3 A flowchart illustrating a method for determining a sound effect evaluation reference provided in an embodiment of this application;

[0050] Figure 4 A flowchart illustrating a method for determining spatial assessment information provided in an embodiment of this application;

[0051] Figure 5 A flowchart illustrating another method for obtaining environmental parameters provided in this application embodiment;

[0052] Figure 6 This is a schematic diagram illustrating the steps of an audio adjustment method provided in an embodiment of this application;

[0053] Figure 7 This application provides a schematic diagram of the structure of an audio adjustment system;

[0054] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0057] To address the lack of specificity in existing technologies for adaptive sound effect adjustment of television devices, which rely solely on dynamically adjusting the frequency response and volume distribution of audio signals to optimize sound effects but fail to capture subtle features in complex environments, this application provides an audio adjustment method, system, electronic device, and storage medium. Addressing the complex environment of television playback, this method acquires environmental parameters of the audio playback environment and determines the optimal sound effect for the audio playback device based on the influence of different furniture materials on sound reflection. Then, using a deep learning model, it dynamically adjusts parameters such as vocal enhancement and virtual surround sound based on sound effect parameters including reverberation time, clarity, and spatiality. This allows for dynamic adjustment of audio parameters according to environmental changes, providing users with a superior listening experience.

[0058] Figure 1 This is a flowchart illustrating an audio adjustment method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method specifically includes:

[0059] S101. Obtain audio playback information, which includes the initial audio parameters of the audio playback device and the environmental parameters of the audio playback environment.

[0060] In this embodiment, the audio playback device can be an audio processing unit built into a smart TV, used to process and play audio signals. Initial audio parameters may include, but are not limited to, parameters preset at the factory, such as volume, timbre, pitch, and equalizer settings, or parameters configured when the audio playback device is first used by the user. The environmental parameters of the audio playback environment refer to the physical environment in which the audio playback device is located. Environmental parameters may include, but are not limited to, factors that may affect sound propagation and reflection, such as room size, wall material, floor material, and furniture layout and type. By obtaining the environmental parameters of the audio playback environment, the impact of the environment on the propagation and reflection of audio signals can be more accurately assessed, thereby evaluating the sound performance of the audio playback device in the current environment and providing data for subsequent adaptive adjustments.

[0061] It should be noted that this embodiment takes the audio playback device as a unit built into the smart TV as an example. In specific implementation, the audio playback device can also be other external devices, such as external speakers. The audio playback device has audio processing and playback functions, and can optimize the sound effect in the current audio playback environment by adjusting the audio parameters.

[0062] In specific embodiments, audio playback information can be obtained through various means, such as reading it directly from the control interface of the audio playback device, or by user manual input or preset configuration files; environmental parameters of the audio playback environment can be identified and analyzed by built-in sensors, such as cameras and microphones, combined with deep learning algorithms, or by user manual input or preset configuration files.

[0063] S102. Based on the environmental parameters, determine the sound effect evaluation reference of the audio playback device in the audio playback environment;

[0064] In this embodiment, after obtaining the environmental parameters of the audio playback environment, the playback sound effect of the audio playback device in the environment is predicted and evaluated by combining the environmental parameters with a neural network model. The sound effect evaluation reference may include, but is not limited to, key sound effect parameters such as reverberation time, clarity, and spatial sense, which can comprehensively reflect the sound effect performance of the audio playback device in the current environment and provide a key basis for subsequent adaptive adjustment.

[0065] In practice, the sound effect evaluation model can be trained and optimized using deep learning algorithms combined with the environmental characteristics of the current audio playback environment to improve the accuracy and precision of the evaluation.

[0066] S103. Input the sound effect evaluation reference and the initial audio parameters of the audio playback device into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and determines the parameter adjustment strategy of the audio playback device based on the adjustment value of the initial audio parameters;

[0067] In this embodiment, after obtaining the sound effect evaluation reference of the current audio playback environment, the evaluation data and the initial audio parameters of the audio playback device are input into the pre-trained adjustment model. The adjustment model combines environmental information and audio parameters, refers to the sound reflection coefficient database, and uses deep learning algorithms to formulate a sound effect adjustment strategy, identify the audio parameters that need to be adjusted and their corresponding adjustment values, and finally integrates the adjustment values ​​of all audio parameters to determine the final parameter adjustment strategy of the audio playback device.

[0068] In practice, the adjustment model can be trained and optimized using deep learning algorithms combined with the characteristics of current audio playback devices and user needs to improve the accuracy and personalization of the adjustment. In addition, the adjustment model can also take into account the technical limitations and safety requirements of audio playback devices to ensure that the adjusted audio parameters are within the range that the device can withstand and to avoid damage to the device.

[0069] S104. Based on the parameter adjustment strategy, update the initial audio parameters of the audio playback device.

[0070] In this embodiment, the initial audio parameters of the audio playback device are updated and optimized according to the determined parameter adjustment strategy to achieve adaptive optimization of sound effects, so as to ensure that the quality and effect of the audio output meet the expected standards.

[0071] Furthermore, the initial audio parameters can be updated through the control interface of the audio playback device. Based on the parameter adjustment strategy, the relevant parameters of the audio processing unit, such as volume, timbre, pitch, and equalizer, can be automatically adjusted to achieve dynamic optimization of sound effects.

[0072] Furthermore, the update process can be real-time, meaning the audio playback device can continuously monitor changes in environmental parameters and adjust audio parameters accordingly in real time to ensure that the sound effect is always at its best.

[0073] The audio adjustment method provided in this application obtains environmental parameters of the audio playback environment, combines them with sound effect evaluation references, and uses a deep learning model to dynamically adjust the initial audio parameters of the audio playback device, thereby achieving adaptive optimization of sound effects. This not only improves the accuracy and precision of sound effects but also brings users a better listening experience.

[0074] Figure 2 A flowchart illustrating a method for obtaining environmental parameters provided in an embodiment of this application is shown below. Figure 2 As shown, the environmental parameters include a first environmental parameter and a second environmental parameter;

[0075] Obtain the environmental parameters of the audio playback environment, including:

[0076] S201. Acquire image information of the audio playback environment through an image acquisition system;

[0077] In this embodiment, the environmental monitoring module of the image acquisition system can capture environmental images through the 3D camera built into the TV and transmit the image data to the AI ​​image recognition system. The AI ​​image recognition system is based on an advanced convolutional neural network (CNN) architecture to acquire image information.

[0078] S202. Analyze the image information to obtain the first environmental parameters of the audio playback environment;

[0079] In this embodiment, the acquired image data is first preprocessed by noise reduction and enhancement. Then, AI image recognition algorithms are used to identify objects such as furniture, appliances, walls and floors in the image and determine their materials. At the same time, image depth estimation techniques, such as monocular depth estimation methods based on deep learning, are used to obtain the distance between the TV and the wall and ceiling, as well as the 3D model information of the room, and to obtain the length, width and height information of the room.

[0080] AI image recognition algorithms require training on massive amounts of image data. During training, a vast dataset of images containing various furniture, appliances, walls, floors, and different spatial layouts is used to learn and continuously optimize the algorithm model to improve the accuracy of object recognition.

[0081] S203. Acquire sound information of the audio playback environment through a sound acquisition system, wherein the sound information includes the phase of the sound signal in the audio playback environment;

[0082] In this embodiment, the television can have a built-in microphone to collect sound signals, perform preprocessing operations such as noise reduction and filtering on the audio data, analyze the acoustic characteristics of the room using audio signal processing algorithms, and measure the reflection of the television sound.

[0083] S204. Based on the image information and sound information, construct a three-dimensional acoustic model of the audio playback environment;

[0084] In this embodiment, an AI image recognition algorithm is applied to perform object recognition and material analysis on the preprocessed image. At the same time, an audio signal processing algorithm is used to analyze the acoustic characteristics of the room and measure the reflection of the TV sound. Then, based on the influence of various furniture materials on sound reflection and image depth estimation technology, the distances between the TV and the walls, ceiling, and other objects, as well as the three-dimensional model information of the room, are determined, thereby constructing a three-dimensional acoustic model of the audio playback environment.

[0085] Furthermore, the audio processing module based on the deep learning model in this embodiment can intelligently adjust the audio playback parameters according to the real-time acquired environmental information and acoustic parameters, thereby achieving dynamic optimization of sound effects. Compared with fixed sound effect processing algorithms or simple environmental adaptation methods, it is more flexible and efficient, and can better adapt to different usage scenarios and user needs.

[0086] S205. Obtain the sound pressure of the sound signal at each moment within a preset time period at the target point and the first preset point in the three-dimensional acoustic model, and obtain the second environmental parameter of the audio playback environment.

[0087] In this embodiment, two measurement points are selected in the three-dimensional acoustic model. By obtaining the sound pressure values ​​of these two measurement points at various moments during a period of time in the simulated sound propagation process, a second environmental parameter of the audio playback environment is obtained, which reflects the specific impact of the audio playback environment on sound propagation and reflection, and provides more detailed data support for subsequent sound effect evaluation and adjustment.

[0088] For example, the target point can be the best viewing position for the audio playback device; the first preset point can be the side of the target point, and a representative position can be selected according to the layout of the room and the distribution of furniture, such as near the wall or near the furniture. In this embodiment, the first preset point is the side of the best viewing position in front of the TV, located in the hall. By measuring the sound pressure of the sound signal at the first preset point at various times over a period of time, the impact of the audio playback environment on the sound effect can be understood more comprehensively, thereby formulating a more accurate sound effect adjustment strategy.

[0089] It's important to note that the specific locations of the target and preset points can be determined based on the room's layout, furniture placement, and the user's actual usage habits. For example, the target point is typically chosen as the optimal position for the user to watch television, which is the most crucial location for the sound experience. Preset points may include room corners, near walls, or other key locations, where the sound propagation and reflection characteristics significantly impact the overall sound effect. By accurately measuring the sound signals at these points, the sound effects of audio playback devices can be more accurately evaluated and adjusted, ensuring users receive a high-quality listening experience in various environments.

[0090] The method for obtaining environmental parameters provided in this application constructs a three-dimensional acoustic model of the audio playback environment by combining image information and sound information, thereby accurately obtaining the environmental parameters of the audio playback environment, providing data support for subsequent sound effect evaluation and adjustment, and improving the accuracy of sound effect adjustment.

[0091] Figure 3 A flowchart illustrating a method for determining a sound effect evaluation reference provided in an embodiment of this application is shown below. Figure 3 As shown, the sound effect evaluation reference includes reverberation evaluation information, clarity evaluation information, and spatial evaluation information;

[0092] The step of determining the sound effect evaluation reference of the audio playback device in the audio playback environment based on the environmental parameters includes:

[0093] S301. Determine the reverberation evaluation information of the audio playback device in the audio playback environment based on the first environmental parameters;

[0094] Reverberation time is related to the fullness and clarity of sound quality; excessively long reverberation time makes speech unclear, while excessively short reverberation time indicates weak sound reflections from surfaces in the environment and excessive sound absorption, which affects the fullness of sound quality. The international standard ISO 3382 defines reverberation time as the time required for the average sound energy density to decay by 60 dB after sound reaches a steady state and stops emitting sound.

[0095] In this embodiment, the first environmental parameters include the playback volume, playback surface area, and sound absorption coefficient of the audio playback environment. Based on the first environmental parameters, the formula for determining the reverberation evaluation information of the audio playback device in the audio playback environment is as follows:

[0096]

[0097] Where T60 is the reverberation assessment value, which is the time required for the average sound energy density of the audio playback environment to decrease by 60dB, V is the playback volume, S is the playback surface area, and α is the average sound absorption coefficient.

[0098] S302. Based on the sound pressure at each moment within a preset time period at the target location, which is included in the second environmental parameters, determine the clarity assessment information of the audio playback device in the audio playback environment.

[0099] Audio clarity is related to the size of the playback environment and the building materials used in its construction. Studies have found that delayed sound within 30ms (with an intensity no greater than 10dB of the direct sound) can enhance the perceived loudness of the direct sound, thus improving speech clarity. Delayed sound of a certain intensity with an interval of more than 50ms will create an echo effect and degrade sound quality. Early sound energy is beneficial to speech clarity, with the time boundary between early and late sound energy being 50ms. The international standard ISO 3382 defines clarity as the percentage of early sound energy in the total sound energy.

[0100] In this embodiment, the clarity assessment information of the audio playback device in the current audio playback environment, i.e., audio clarity, is calculated and determined based on the sound pressure at various times within a certain period of time at the target point in the three-dimensional acoustic model.

[0101] S303. Based on the second environmental parameter, determine the spatial evaluation information of the audio playback device in the audio playback environment.

[0102] Spatial perception of transient signals (such as music) is considered from two attributes: auditory source width (ASW) and auditory enclosure. ASW represents the listener's perception of the width of the sound source's position on the horizontal plane, while auditory enclosure represents the listener's perception of the sound coming from all directions.

[0103] In this embodiment, by analyzing the propagation path and reflection of sound signals at the target point and the first preset point in the three-dimensional acoustic model, the spatial evaluation information of the audio playback device in the current audio playback environment, namely the audio spatial sense, is determined.

[0104] The method for determining sound effect evaluation references provided in this application embodiment obtains the influence of the environment on the continuous reflection of sound by determining reverberation evaluation information, determines the clarity and intelligibility of sound propagation by determining clarity evaluation information, and obtains the distribution and perception of sound in three-dimensional space by determining spatial evaluation information. The three types of sound effect evaluation references complement each other and together constitute a comprehensive evaluation of the sound effect performance of the audio playback device, which can comprehensively and accurately evaluate the sound effect performance of the audio playback device in the current environment.

[0105] Figure 4 A flowchart illustrating a method for determining spatial assessment information provided in an embodiment of this application is shown below. Figure 4 As shown, determining the spatial evaluation information of the audio playback device in the audio playback environment based on the second environmental parameter includes:

[0106] S401. Based on the sound pressure at each moment within a preset time period at the target location and the first preset location, which are included in the second environmental parameters, determine the auditory source width and auditory envelopment of the audio playback device in the audio playback environment.

[0107] The parameter of the width of the auditory sound source is the early lateral sound energy ratio, which refers to the proportion of lateral sound energy arriving in the first 80ms. In this embodiment, after the room model is obtained, the sound pressure value of the target point at each moment within a certain period of time can be obtained by means of simulation, etc., using an omnidirectional microphone, and the sound pressure value of the first preset point at each moment within a certain period of time can be obtained by using a figure-eight microphone.

[0108] In this embodiment, taking the first preset point as the target point and the measuring point in the side hall as an example, the sound pressure at each moment of a certain period of time of the pulse response of the first preset point measured by the figure-eight microphone and the sound pressure at each moment of a certain period of time of the pulse response of the first preset point measured by the omnidirectional microphone are simulated.

[0109] The formula for determining the width of an auditory sound source is as follows:

[0110]

[0111] Among them, P L P(t) represents the sound pressure at each moment of the impulse response at the first preset point measured by the figure-eight microphone over a period of time; P(t) represents the sound pressure at each moment of the impulse response at the target point over a period of time.

[0112] The evaluation parameter for the listener's sense of immersion is the late lateral sound energy level, which is used to assess the evaluation parameter for the listener's sense of immersion.

[0113] In this embodiment, the formula for calculating auditory envelopment is as follows:

[0114]

[0115] Among them, P L (t) represents the sound pressure at various moments within a certain time period of the pulse response at the first preset point, measured by the figure-eight microphone; P 10 (t) represents the sound pressure at various moments within a certain time period of the impulse response at the first preset point, measured by an omnidirectional microphone.

[0116] S402. Based on the auditory sound source width and auditory envelopment, obtain the spatial evaluation information of the audio playback device in the audio playback environment.

[0117] In this embodiment, the auditory sound source width and auditory envelopment are used as spatial evaluation information of the audio playback device in the audio playback environment.

[0118] It should be noted that, in practice, the above sound effect evaluation references can be adjusted according to the user's actual auditory needs and preferences. For example, for users who prefer immersive sound effects, the weight of spatial evaluation information can be appropriately increased to enhance the spatial and stereoscopic feel of audio playback; while for users who value speech intelligibility, the weight of clarity evaluation information can be increased to ensure the clarity of spoken words.

[0119] The spatial evaluation information determination method provided in this application embodiment determines the audible sound source width and audible envelopment of the audio playback device in the current audio playback environment by analyzing the sound pressure at various times within a certain period of time at the target point and the first preset point, thereby obtaining the spatial evaluation information of the audio playback device. It not only considers the width of the sound on the horizontal plane, but also the perception of the sound coming from all directions, so as to comprehensively and accurately evaluate the spatial sound effect performance of the audio playback device in the current environment.

[0120] Figure 5 A flowchart illustrating another method for obtaining environmental parameters provided in this application embodiment is shown below. Figure 5 As shown, the first environmental parameters include the playback volume, playback surface area, and sound absorption coefficient of the audio playback environment.

[0121] The step of parsing the image information to obtain the first environmental parameters of the audio playback environment includes:

[0122] S501. Analyze the image information to obtain the size, material type, and spatial layout of all objects in the audio playback environment; wherein, the spatial layout includes the spatial position, arrangement direction, and arrangement spacing of the objects;

[0123] In this embodiment, by deeply analyzing image information, the specific size, material type, and spatial layout of all objects in the audio playback environment are obtained, including the size and texture of the objects, as well as the relative position and arrangement of each object in space.

[0124] S502. Based on the size and spatial layout of all objects within the audio playback environment, determine the playback volume and playback surface area of ​​the audio playback environment.

[0125] In this embodiment, based on the image recognition results, the specific dimensions of each object within the audio playback environment are determined. Combining the material types of the objects, such as the materials of walls, floors, and furniture, and the spatial layout of the objects, the total volume of the audio playback environment, i.e., the playback volume, is calculated. Simultaneously, the total surface area of ​​each object is calculated as the playback surface area.

[0126] S503. Find the sound absorption coefficient corresponding to each material type in the material sound absorption coefficient table;

[0127] AI audio processing has established a database of sound reflection coefficients for different materials. This database contains information on the reflection coefficients of various common materials (such as wood, glass, metal, and fabric) for different frequencies of sound. The data in this database was obtained through a large number of experimental measurements and theoretical calculations, and is constantly being updated and optimized.

[0128] In this embodiment, the sound absorption coefficient of the surface of each material object is determined by referring to a sound reflection coefficient database based on the material type of the object.

[0129] S504. Determine the average sound absorption coefficient of the audio playback environment based on the size of each material object and its corresponding sound absorption coefficient.

[0130] In this embodiment, when performing acoustic analysis on the audio playback environment, the average sound absorption coefficient of the entire audio playback environment is calculated based on the size of each object of different materials in the environment and the sound absorption coefficient of each object.

[0131] The method for obtaining environmental parameters provided in this application first obtains the size, material type, and spatial layout of all objects in the audio playback environment through image recognition technology. Then, based on the size and spatial layout of the objects, the playback volume and playback surface area of ​​the audio playback environment are determined. Finally, referring to the sound reflection coefficient database, the sound absorption coefficient of each material object surface is determined according to the material type of the object. This provides accurate data support for subsequent sound effect evaluation and adjustment, ensuring the accuracy and effectiveness of sound effect adjustment.

[0132] In an optional embodiment of the present invention, the step of inputting the sound effect evaluation reference and the initial audio parameters of the audio playback device into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and determining the parameter adjustment strategy of the audio playback device based on the adjustment value of the initial audio parameters, includes:

[0133] The three-dimensional acoustic model of the audio playback environment, the sound effect evaluation reference, and the initial audio parameters of the audio playback device are simultaneously input into the adjustment model, so that the adjustment model outputs the adjustment values ​​of each initial audio parameter according to the three-dimensional acoustic model, thereby obtaining the parameter adjustment strategy of the audio playback device.

[0134] Because sound waves emitted from a sound source reach both ears with a certain time difference, intensity difference, and phase difference, the direction and distance of the sound source can be determined, allowing for sound image localization. This sound image localization ability, obtained through binaural hearing, depends primarily on the intensity difference of the sound reaching both ears at frequencies above 1400Hz; below 1400Hz, it depends primarily on the time difference of sound arrival. Sound reflections within a room can affect the intensity and phase differences perceived by both ears.

[0135] For example, the ratio of sound energy from the sides to sound from above can affect the sense of sound localization. In a room where the ceiling is completely reflective but the side walls are completely sound-absorbing, the viewer will feel too much reflected sound from above, interfering with the viewer's ability to locate sounds from left to right.

[0136] In this embodiment, in order to more accurately adjust the human ear's auditory perception in a real environment, the three-dimensional acoustic model of the audio playback environment and the audio parameters are simultaneously input into the adjustment model after outputting the adjustment value of the initial audio parameters. This allows the adjustment model to comprehensively consider the sound wave propagation path and reflection characteristics, as well as the time difference, intensity difference, and phase difference between the two ears of the sound source and the receiver, to accurately simulate the human ear's auditory experience in the environment, thereby optimizing the sound effect adjustment strategy and ensuring the authenticity and immersion of the sound quality reproduction.

[0137] For example, based on the room acoustic parameters (reverberation time, clarity, spatial sense, sound image localization, etc.) obtained by the environmental perception module, the three-dimensional acoustic model of the current audio playback environment, and the material information of objects, the AI ​​audio processing module uses a deep learning model, such as a convolutional neural network (CNN) based audio processing model, to adjust the audio playback parameters of the TV.

[0138] Specifically, the TV's sound effects are adjusted based on the acquired reverberation time, sharpness, spatial awareness, and sound image localization parameters. For example, Dolby Atmos typically includes many sub-modules, with commonly used ones including voice enhancement and virtual surround sound. If the reverberation time T60 is greater than 1 second, or the sharpness D50 is less than 0.5, the voice enhancement is set to a strong setting; otherwise, it's set to a weak setting. If the early lateral sound energy ratio is less than 0.3, or the later lateral sound energy level is less than -10dB, the virtual surround sound is set to a strong setting; otherwise, it's set to a weak setting. The preset sound image correction algorithm adjusts the amplitude and phase of each speaker to correct the sound image to be centered.

[0139] In an optional embodiment of the present invention, the method further includes:

[0140] If the environmental parameters of the audio playback environment are detected to be updated, new audio playback information is obtained, and the step of obtaining audio playback information is re-executed.

[0141] In this embodiment, when changes in audio playback environment parameters are detected, new data is automatically acquired, and the three-dimensional acoustic model and audio parameters are recalculated and adjusted to achieve adaptive matching between audio playback and the playback environment. This ensures that the sound effects always conform to the characteristics of the current environment. Through real-time dynamic adjustment, sound effect distortion caused by environmental changes is avoided, thereby improving the user's audio experience.

[0142] Figure 6 This is a schematic diagram illustrating the steps of an audio adjustment method provided in an embodiment of this application, as shown below. Figure 6 As shown, the process of adjusting the audio of a television mainly includes the following steps:

[0143] Step 1: System Initialization: After the TV starts up, initialize the environmental perception module, AI audio processing module, and related hardware devices (3D camera, microphone array, TV AI chip, etc.). Load the pre-trained AI model and sound reflection coefficient database.

[0144] Step 2: Environmental Data Acquisition: The 3D camera in the environmental perception module captures images of the environment surrounding the TV, and the microphone array collects sound signals.

[0145] Step 3: Data preprocessing: Perform preprocessing operations such as noise reduction and enhancement on the acquired image data, and perform preprocessing operations such as noise reduction and filtering on the audio data.

[0146] Step 4: Environmental Information Recognition and Analysis: AI image recognition algorithms are used to identify objects and analyze materials in the pre-processed images. Simultaneously, image depth estimation technology is used to obtain the distances between the television and the walls and ceiling, as well as 3D model information of the room. Audio signal processing algorithms are used to analyze the acoustic characteristics of the pre-processed audio data. The measured dimensions (length, width, height), building materials, sound pressure levels at various times within a given period (measured by a figure-eight microphone in the hall), sound pressure levels at specific measurement points, and sound pressure levels at a distance of 10m from the sound source in a free field (measured by an omnidirectional microphone) are substituted into the formulas for calculating reverberation time, intelligibility, and spatial perception, thus obtaining the parameters for reverberation time, intelligibility, and spatial perception.

[0147] Step 5: Sound Effect Adjustment Decision: Based on the environmental information and acoustic parameters obtained by the environmental perception module, combined with the sound reflection coefficient database, the AI ​​audio processing module uses a deep learning model to formulate a sound effect adjustment strategy and determine the audio parameters that need to be adjusted.

[0148] Step 6: Sound Effect Adjustment: Adjust audio parameters such as human voice enhancement and virtual surround sound on the TV to achieve adaptive sound effect optimization.

[0149] Step 7: Real-time Monitoring and Updates: The system continuously monitors environmental changes by collecting data in real time through a 3D camera and microphone. When environmental changes are detected, such as furniture moving or family members entering or leaving the room, steps 2 through 6 are repeated to update the sound effects in real time, ensuring that users always have the best audiovisual experience.

[0150] The aforementioned application and TV audio adjustment process are designed specifically for television devices, with targeted optimizations in areas such as sound image positioning and sound effect mode adjustment. It can automatically sense complex environmental changes in which the TV is located and adaptively optimize and adjust the TV sound effects based on environmental information, restoring a more realistic and natural sound effect and bringing users an immersive audio-visual experience. Compared with related technologies of traditional TV sound effects, it greatly improves the adaptability and flexibility of TV sound effects, eliminating the need for users to manually perform complex sound effect settings, and improving the convenience and satisfaction of users using the TV.

[0151] Figure 7 This application provides a schematic diagram of the structure of an audio adjustment system, as shown in the embodiments below. Figure 7 As shown, the system specifically includes:

[0152] The acquisition module 701 is used to acquire audio playback information, which includes the initial audio parameters of the audio playback device and the environmental parameters of the audio playback environment.

[0153] Evaluation module 702 is used to determine the sound effect evaluation reference of the audio playback device in the audio playback environment based on the environmental parameters;

[0154] The adjustment module 703 is used to input the sound effect evaluation reference and the initial audio parameters of the audio playback device into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and determines the parameter adjustment strategy of the audio playback device based on the adjustment value of the initial audio parameters;

[0155] The update module 704 is used to update the initial audio parameters of the audio playback device based on the parameter adjustment strategy.

[0156] In one possible implementation, the acquisition module 701 is further configured to acquire image information of the audio playback environment through an image acquisition system; parse the image information to obtain a first environmental parameter of the audio playback environment; acquire sound information of the audio playback environment through a sound acquisition system, the sound information including the phase of the sound signal in the audio playback environment; construct a three-dimensional acoustic model of the audio playback environment based on the image information and the sound information; acquire the sound pressure of the sound signal at each moment within a preset time period at the target point and the first preset point in the three-dimensional acoustic model to obtain a second environmental parameter of the audio playback environment.

[0157] In one possible implementation, the evaluation module 702 is further configured to determine the reverberation evaluation information of the audio playback device in the audio playback environment based on the first environmental parameter; determine the clarity evaluation information of the audio playback device in the audio playback environment based on the sound pressure at each moment within a preset time period at the target location included in the second environmental parameter; and determine the spatial evaluation information of the audio playback device in the audio playback environment based on the second environmental parameter.

[0158] In one possible implementation, the evaluation module 702 is further configured to determine the audible sound source width and audible envelopment of the audio playback device in the audio playback environment based on the sound pressure at each moment within a preset time period at the target location and the first preset location, which are included in the second environmental parameters; and to obtain spatial evaluation information of the audio playback device in the audio playback environment based on the audible sound source width and audible envelopment.

[0159] In one possible implementation, the acquisition module 701 is further configured to parse the image information to obtain the size, material type, and spatial layout of all objects within the audio playback environment; wherein, the spatial layout includes the spatial position, arrangement direction, and arrangement spacing of the objects; determine the playback volume and playback surface area of ​​the audio playback environment based on the size and spatial layout of all objects within the audio playback environment; look up the sound absorption coefficient corresponding to each material type in the material sound absorption coefficient table; and determine the average sound absorption coefficient of the audio playback environment based on the size of each material object within the audio playback environment and its corresponding sound absorption coefficient.

[0160] In one possible implementation, the adjustment module 703 is further configured to simultaneously input the three-dimensional acoustic model of the audio playback environment, the sound effect evaluation reference, and the initial audio parameters of the audio playback device into the adjustment model, so that the adjustment model outputs the adjustment values ​​of each initial audio parameter according to the three-dimensional acoustic model, thereby obtaining the parameter adjustment strategy of the audio playback device.

[0161] In one possible implementation, the update module 704 is further configured to obtain new audio playback information and re-execute the step of obtaining audio playback information when it is detected that the environmental parameters of the audio playback environment have been updated.

[0162] The audio adjustment system provided in this embodiment can be as follows: Figure 7 The audio adjustment system shown can perform, for example... Figure 1-6 All steps of mid-frequency adjustment, thereby achieving Figure 1-6 For details on the technical effects of the audio adjustments shown, please refer to [link / reference]. Figure 1-6 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0163] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 8As shown, this application provides an electronic device including a processor 801, a communication interface 802, a memory 803, and a communication bus 804. The processor 801, communication interface 802, and memory 803 communicate with each other via the communication bus 804. The memory 803 stores computer programs. When the processor 801 executes the program stored in the memory 803, it implements the audio adjustment steps provided in any of the aforementioned method embodiments.

[0165] The system acquires audio playback information, including initial audio parameters of the audio playback device and environmental parameters of the audio playback environment. Based on the environmental parameters, it determines a sound effect evaluation reference for the audio playback device in the audio playback environment. The sound effect evaluation reference and the initial audio parameters of the audio playback device are input into an adjustment model, causing the adjustment model to output an adjustment value for the initial audio parameters. Based on the adjustment value of the initial audio parameters, a parameter adjustment strategy for the audio playback device is determined. Based on the parameter adjustment strategy, the initial audio parameters of the audio playback device are updated.

[0166] In one possible implementation, image information of the audio playback environment is acquired through an image acquisition system; the image information is parsed to obtain a first environmental parameter of the audio playback environment; sound information of the audio playback environment is acquired through a sound acquisition system, the sound information including the phase of the sound signal in the audio playback environment; a three-dimensional acoustic model of the audio playback environment is constructed based on the image information and the sound information; the sound pressure of the sound signal at each moment within a preset time period at the target point and a first preset point in the three-dimensional acoustic model is acquired to obtain a second environmental parameter of the audio playback environment.

[0167] In one possible implementation, reverberation evaluation information of the audio playback device in the audio playback environment is determined based on the first environmental parameter; clarity evaluation information of the audio playback device in the audio playback environment is determined based on the sound pressure level at each moment within a preset time period at the target location, which is included in the second environmental parameter; and spatial evaluation information of the audio playback device in the audio playback environment is determined based on the second environmental parameter.

[0168] In one possible implementation, the audible sound source width and audible enclosure of the audio playback device in the audio playback environment are determined based on the sound pressure at each moment within a preset time period at the target location and the first preset location, which are included in the second environmental parameters; and the spatial evaluation information of the audio playback device in the audio playback environment is obtained based on the audible sound source width and audible enclosure.

[0169] In one possible implementation, the image information is parsed to obtain the size, material type, and spatial layout of all objects within the audio playback environment; wherein, the spatial layout includes the spatial position, arrangement direction, and spacing of the objects; based on the size and spatial layout of all objects within the audio playback environment, the playback volume and playback surface area of ​​the audio playback environment are determined; the sound absorption coefficient corresponding to each material type is looked up in the material sound absorption coefficient table; based on the size of each material object within the audio playback environment and its corresponding sound absorption coefficient, the average sound absorption coefficient of the audio playback environment is determined.

[0170] In one possible implementation, the three-dimensional acoustic model of the audio playback environment, the sound effect evaluation reference, and the initial audio parameters of the audio playback device are simultaneously input into the adjustment model, so that the adjustment model outputs the adjustment values ​​of each initial audio parameter according to the three-dimensional acoustic model, thereby obtaining the parameter adjustment strategy of the audio playback device.

[0171] In one possible implementation, if an update to the environmental parameters of the audio playback environment is detected, new audio playback information is obtained, and the step of obtaining the audio playback information is re-executed.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0173] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0174] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An audio adjustment method, characterized in that, include: Acquire audio playback information, which includes the initial audio parameters of the audio playback device and the environmental parameters of the audio playback environment; Based on the environmental parameters, a sound effect evaluation reference for the audio playback device in the audio playback environment is determined; The sound effect evaluation reference and the initial audio parameters of the audio playback device are input into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and the parameter adjustment strategy of the audio playback device is determined based on the adjustment value of the initial audio parameters; Based on the parameter adjustment strategy, the initial audio parameters of the audio playback device are updated.

2. The method according to claim 1, characterized in that, The environmental parameters include a first environmental parameter and a second environmental parameter; Obtain the environmental parameters of the audio playback environment, including: The image acquisition system acquires image information of the audio playback environment; The image information is analyzed to obtain the first environmental parameters of the audio playback environment; The sound information of the audio playback environment is acquired through a sound acquisition system, and the sound information includes the phase of the sound signal in the audio playback environment; Based on the image and sound information, a three-dimensional acoustic model of the audio playback environment is constructed. The sound pressure of the sound signal at the target point and the first preset point within a preset time period is obtained in the three-dimensional acoustic model to obtain the second environmental parameter of the audio playback environment.

3. The method according to claim 2, characterized in that, The sound effect evaluation reference includes reverberation evaluation information, clarity evaluation information, and spatial evaluation information; The step of determining the sound effect evaluation reference of the audio playback device in the audio playback environment based on the environmental parameters includes: Based on the first environmental parameter, determine the reverberation evaluation information of the audio playback device in the audio playback environment; Based on the sound pressure level at each moment within a preset time period at the target location, which is included in the second environmental parameter, the clarity assessment information of the audio playback device in the audio playback environment is determined. Based on the second environmental parameter, spatial evaluation information of the audio playback device in the audio playback environment is determined.

4. The method according to claim 3, characterized in that, The step of determining the spatial evaluation information of the audio playback device in the audio playback environment based on the second environmental parameter includes: Based on the sound pressure at each moment within a preset time period at the target location and the first preset location, which are included in the second environmental parameters, the audible sound source width and audible envelopment of the audio playback device in the audio playback environment are determined. Based on the auditory sound source width and auditory envelopment, spatial evaluation information of the audio playback device in the audio playback environment is obtained.

5. The method according to claim 2, characterized in that, The first environmental parameters include the playback volume, playback surface area, and sound absorption coefficient of the audio playback environment; The step of parsing the image information to obtain the first environmental parameters of the audio playback environment includes: The image information is analyzed to obtain the size, material type, and spatial layout of all objects within the audio playback environment; wherein, the spatial layout includes the spatial position, arrangement direction, and arrangement spacing of the objects; Based on the size and spatial layout of all objects within the audio playback environment, determine the playback volume and playback surface area of ​​the audio playback environment; Find the sound absorption coefficient corresponding to each material type in the material sound absorption coefficient table; The average sound absorption coefficient of the audio playback environment is determined based on the size of each material object and its corresponding sound absorption coefficient.

6. The method according to claim 2, characterized in that, The step of inputting the sound effect evaluation reference and the initial audio parameters of the audio playback device into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and determining the parameter adjustment strategy of the audio playback device based on the adjustment value of the initial audio parameters, includes: The three-dimensional acoustic model of the audio playback environment, the sound effect evaluation reference, and the initial audio parameters of the audio playback device are simultaneously input into the adjustment model, so that the adjustment model outputs the adjustment values ​​of each initial audio parameter according to the three-dimensional acoustic model, thereby obtaining the parameter adjustment strategy of the audio playback device.

7. The method according to claim 1, characterized in that, The method further includes: If the environmental parameters of the audio playback environment are detected to be updated, new audio playback information is obtained, and the step of obtaining audio playback information is re-executed.

8. An audio adjustment system, characterized in that, include: The acquisition module is used to acquire audio playback information, which includes the initial audio parameters of the audio playback device and the environmental parameters of the audio playback environment. An evaluation module is used to determine a sound effect evaluation reference for the audio playback device in the audio playback environment based on the environmental parameters. The adjustment module is used to input the sound effect evaluation reference and the initial audio parameters of the audio playback device into the adjustment model, so that the adjustment model outputs the adjustment value of the initial audio parameters, and determines the parameter adjustment strategy of the audio playback device based on the adjustment value of the initial audio parameters; An update module is used to update the initial audio parameters of the audio playback device based on the parameter adjustment strategy.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the audio adjustment method according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the audio adjustment method according to any one of claims 1 to 7.

Images