An audio processing method and display device

Abstract

This application relates to the field of computer technology, and in particular to an audio processing method and display device for improving audio playback quality. The method comprises: acquiring re-sampled audio based on test audio, and determining compensation coefficients for each frequency band according to the re-sampled energy values ​​of each frequency band in the re-sampled audio; acquiring audio to be played, and performing energy compensation on each frequency band in the audio to be played according to the determined compensation coefficients to obtain the target audio. In this way, by using the compensation coefficients corresponding to each frequency band, the audio to be played is compensated, thereby improving the audio playback quality.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an audio processing method and a display device. Background Technology

[0002] With the continuous development of technology, audio playback devices are increasingly used indoors. However, during indoor transmission, audio is affected by physical objects, resulting in phenomena such as reflection, absorption, refraction, diffraction, and scattering. This leads to a certain degree of audio loss, causing inconsistent acoustic frequency balance and poor audio playback quality. Summary of the Invention

[0003] This application provides an audio processing method and a display device to solve the problem in the prior art of inconsistent audio acoustic frequency balance and poor audio playback effect caused by audio loss.

[0004] The specific technical solutions provided in this application are as follows:

[0005] In a first aspect, a display device includes:

[0006] A monitor is used to display the user interface.

[0007] The controller is configured as follows:

[0008] Acquire the re-collected audio based on the test audio, and determine the compensation coefficient corresponding to each frequency band based on the re-collected energy value of each frequency band in the re-collected audio.

[0009] The audio to be played is acquired, and energy compensation is performed on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio.

[0010] In this embodiment, the audio playback is compensated using the compensation coefficients corresponding to each frequency band. This improves the audio playback effect while ensuring consistent acoustic frequency balance. Furthermore, adaptive compensation can be performed for different playback devices, improving the usability and compensation efficiency of the audio playback devices.

[0011] Secondly, an audio processing method includes:

[0012] Acquire the re-collected audio based on the test audio, and determine the compensation coefficient corresponding to each frequency band based on the re-collected energy value of each frequency band in the re-collected audio.

[0013] The audio to be played is acquired, and energy compensation is performed on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio.

[0014] In some embodiments, determining target frame audio data from the frame data based on the zero-crossing rate of each frame of audio data in the re-sampled audio includes:

[0015] If the difference between the zero-crossing rates of M consecutive frames of audio data in each frame and the zero-crossing rates of M consecutive frames of audio data in the test audio all conform to a preset range of zero-crossing rate differences, then the Mth frame of audio data in the M consecutive frames of audio data is taken as the target frame audio data; or...

[0016] If the difference between the zero-crossing rates of the M consecutive frames of audio data in each frame and the zero-crossing rates of the M consecutive frames of audio data in the test audio both conform to a preset range of zero-crossing rate differences, and the zero-crossing rate changes of the M consecutive frames of audio data in each frame and the M consecutive frames of audio data in the test audio have the same trend, then the Mth frame of audio data in the M consecutive frames of audio data in each frame is taken as the target frame audio data.

[0017] In some embodiments, obtaining the compensation coefficient corresponding to each frequency band based on the sampled energy value of each frequency band in the target frame audio data and the actual energy value of each frequency band in the source frame audio data includes:

[0018] Based on the re-sampling energy values ​​of each frequency band in the target frame audio data, the average re-sampling energy value is determined; based on the actual energy values ​​of each frequency band in the source frame audio data, the actual average energy value is determined; based on the average re-sampling energy value and the actual average energy value, the audio scaling factor is obtained, and based on the audio scaling factor, the compensation factor corresponding to each frequency band is obtained.

[0019] In some embodiments, obtaining the compensation coefficients corresponding to each frequency band based on the audio scaling factor includes:

[0020] Based on the audio scaling factor, the re-sampling energy value of each frequency band is scaled to obtain each scaled energy value; each scaled energy value is compared with the re-sampling energy value of each frequency band to obtain the compensation coefficient corresponding to each frequency band.

[0021] In some embodiments, after determining the compensation coefficient corresponding to each frequency band based on the sampled energy value of each frequency band in the sampled audio, the method further includes:

[0022] Starting from the playback volume of the test audio, the playback volume is sequentially decreased according to a preset volume decrease step size to obtain the compensation coefficients corresponding to each frequency band at each playback volume, until the data acquisition stop condition is met. Each time the playback volume is decreased, the following operations are performed on the target frame audio data: the target frame audio data is played at the current playback volume to obtain the retrieval energy value of each frequency band at the current playback volume; based on the retrieval energy value of each frequency band at the current playback volume, the compensation coefficient corresponding to each frequency band at the current playback volume is determined.

[0023] In some embodiments, the step of performing energy compensation on each frequency band of the audio to be played according to determined compensation coefficients to obtain the target audio includes:

[0024] Obtain the target playback volume of the audio to be played; determine the compensation coefficients corresponding to each frequency band under the target playback volume based on the target playback volume and the compensation coefficients corresponding to each frequency band under each playback volume; perform energy compensation on each frequency band in the audio to be played based on the compensation coefficients corresponding to each frequency band under the target playback volume to obtain the target audio.

[0025] In some embodiments, it also includes:

[0026] If the compensation coefficients corresponding to each frequency band at each playback volume do not include the compensation coefficients corresponding to each frequency band at the target playback volume, then based on the volume value, determine the two playback volumes adjacent to the target playback volume from the playback volumes; and based on the compensation coefficients corresponding to each frequency band at the two playback volumes, determine the compensation coefficients corresponding to each frequency band at the target playback volume.

[0027] Thirdly, an image processing apparatus includes:

[0028] The test unit is used to acquire the re-collected audio based on the test audio, and determine the compensation coefficient corresponding to each frequency band based on the re-collected energy value of each frequency band in the re-collected audio.

[0029] The compensation unit is used to acquire the audio to be played and perform energy compensation on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio.

[0030] Fourthly, a storage medium, when instructions in the storage medium are executed by a processor, enables the processor to perform the audio processing method as described in any of the preceding claims. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram illustrating the operation scenario between the display device and the control device according to the embodiments of this application;

[0033] Figure 2 This is a hardware configuration block diagram of the display device 200 provided in the embodiments of this application;

[0034] Figure 3 This is a hardware configuration block diagram of the control device 100 provided in the embodiments of this application;

[0035] Figure 4 This is a schematic diagram of the software configuration of the display device 200 provided in the embodiments of this application;

[0036] Figure 5 This is a schematic diagram showing the icon control interface of an application in a display device 200 according to an embodiment of this application;

[0037] Figure 6 This is a flowchart illustrating the audio processing method provided in the embodiments of this application;

[0038] Figure 7 This is a schematic diagram illustrating the acquisition of data for each frequency band through filtering, as provided in the embodiments of this application.

[0039] Figure 8 This is a schematic diagram illustrating the acquisition of various recovered energy values ​​provided in the embodiments of this application;

[0040] Figure 9 This is a schematic diagram illustrating the process of obtaining the compensation coefficients for each frequency band at different volume levels provided in the embodiments of this application;

[0041] Figure 10 This is a schematic diagram showing the relationship between the actual energy value and the compensation coefficient of the target frequency band provided in the embodiments of this application;

[0042] Figure 11 This is a schematic diagram of the audio processing device provided in the embodiments of this application. Detailed Implementation

[0043] To address the problem of inconsistent acoustic frequency balance in existing audio technologies due to audio loss, this application provides an audio processing solution. This solution comprises two stages: a sampling and analysis stage and an audio compensation stage. In the sampling and analysis stage, re-sampled audio is acquired based on test audio data, and compensation coefficients are determined for each frequency band according to its re-sampled energy value. In the audio compensation stage, the audio to be played is acquired, and energy compensation is performed on each frequency band of the audio to be played based on the determined compensation coefficients to obtain the target audio.

[0044] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0045] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0046] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete implementation on its own. It should be noted that the brief descriptions of terminology in this application are merely for the convenience of understanding the embodiments described below, and are not intended to limit the implementation of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0047] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device according to an embodiment of this application. Figure 1 As shown, a user can operate the display device 200 via a mobile terminal 300 and a control device 100. The control device 100 can be a remote control, and communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods to control the display device 200. The user can input user commands through buttons on the remote control, voice input, control panel input, etc., to control the display device 200. In some embodiments, a mobile terminal, tablet computer, computer, laptop computer, and other smart devices can also be used to control the display device 200.

[0048] In some embodiments, the mobile terminal 300 can install software applications with the display device 200 to achieve connection and communication via network communication protocols, enabling one-to-one control operations and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 for synchronous display. The display device 200 also communicates with the server 400 via various communication methods. The display device 200 can communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 can provide various content and interactive features to the display device 200. The display device 200 can be a liquid crystal display, an OLED display, or a projection display device. In addition to providing broadcast television reception functions, the display device 200 can also be equipped with a smart network television function that provides computer support.

[0049] Figure 2 An exemplary configuration block diagram of a control device 100 is shown. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input commands and convert them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200. The communication interface 130 is used for external communication and includes at least one of a Wi-Fi chip, a Bluetooth module, NFC, or a replacement module. The user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, buttons, or a replacement module.

[0050] Figure 3 An exemplary hardware configuration block diagram of a display device 200 is shown. Figure 3The display device 200 shown includes at least one of the following: a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. The controller includes a central processing unit, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first to nth interface for input / output. The display 260 may be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and may also be a projection device and a projection screen. The audio output interface 270 is used to implement audio playback functionality. The tuner / demodulator 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals. The detector 230 is used to collect signals from the external environment or signals interacting with the external environment. The controller 250 and the tuner 210 can be located in different separate devices, that is, the tuner 210 can also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0051] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200. The user can input user commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0052] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, enabling the conversion between the internal form of information and a form acceptable to the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0053] Figure 4 This is a schematic diagram of the software configuration of a display device 200 according to an embodiment of this application, such as... Figure 4As shown, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer. The kernel layer contains at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, Wi-Fi driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0054] Figure 5 This is a schematic diagram showing the icon control interface of an application in a display device 200 according to one or more embodiments of this application, such as... Figure 5 As shown, the application layer contains at least one application whose corresponding icon control can be displayed on the screen, such as: live TV application icon control, video-on-demand application icon control, media center application icon control, application center icon control, game application icon control, etc. Live TV applications can provide live television from different signal sources. Video-on-demand applications can provide video from different storage sources. Unlike live TV applications, video-on-demand provides video display from certain storage sources. Media center applications can provide applications for playing various multimedia content. The application center can provide storage for various applications.

[0055] See Figure 6 The diagram shown is a flowchart of an audio processing method provided in an embodiment of this application. This method can be applied to an audio playback device, which can be configured independently or integrated into a display device. The specific process is as follows:

[0056] S601. The audio playback device acquires the re-sampled audio collected based on the test audio, and determines the corresponding compensation coefficient for each frequency band based on the re-sampled energy value of each frequency band in the re-sampled audio.

[0057] Specifically, when executing S601, the following steps may be used, but are not limited to:

[0058] S6011. The audio playback device determines the target frame audio data from each frame of audio data based on the zero-crossing rate of each frame of audio data in the re-acquired audio.

[0059] Zero-crossing rate refers to the rate at which the sign of a signal changes; for example, a signal changes from a positive number to a negative number, or from a negative number to an integer. Energy value can also be called loudness.

[0060] To improve processing efficiency, in this embodiment of the application, the audio acquisition device can perform frame-by-frame processing on the test audio before playing the test audio to obtain audio data of each frame in the test audio. Then, during the playback of the test audio, each frame of audio data in the test audio is played separately. Accordingly, the audio acquisition device can collect each frame of audio data in the test audio separately as the re-acquired audio.

[0061] For ease of description, in the following text, each frame of audio data in the test audio will be referred to as audio data y1, audio data y2, ..., audio data yg, and each frame of audio data in the test audio will be referred to as audio data x1, audio data x2, ..., audio data xj, where g and j represent the frame number. Since all frames of audio data in the test audio can be collected during re-encoding, or only a portion of the frames can be collected, the values ​​of g and j can be the same or different.

[0062] When executing S6011, the following two methods can be used, but are not limited to:

[0063] Method 1: If the zero-crossing rate difference between the zero-crossing rate of M consecutive frames of audio data in audio data x1, audio data x2, ..., audio data xj and the zero-crossing rate difference of M consecutive frames of audio data in audio data y1, audio data y2, ..., audio data yg all meet the preset zero-crossing rate difference range, then the audio playback device will use the Mth frame of audio data from the M consecutive frames of audio data in the re-sampled audio as the target frame audio data.

[0064] The value of M can be a positive integer between [1, 100000], for example, the value of M can be 10.

[0065] For example, assuming M is 3, and the preset zero-crossing rate difference range is 0 to 2, as shown in Table 1, the zero-crossing rates of three consecutive audio frames in the test audio—audio data y1, audio data y2, and audio data y3—are 5, 6, and 7, respectively. As shown in Table 2, the zero-crossing rates of three consecutive audio frames in the re-collected audio—audio data x1, audio data x2, and audio data x3—are 6, 7, and 8, respectively. Obviously, the zero-crossing rate differences between audio data x1 and audio data y1, between audio data x2 and audio data y2, and between audio data x3 and audio data y3 all conform to the preset zero-crossing rate difference range of 0 to 2. Therefore, the audio playback device can use audio data x3 as the target frame audio data.

[0066] Table 1 shows the zero-crossing rate of M consecutive frames of audio data in the test audio.

[0067] Test audio Audio data y1 Audio data y2 Audio data y3 Zero crossing rate 5 6 7

[0068] Table 2 Zero-crossing rate of consecutive M frames of audio data in the re-acquired audio.

[0069] Audio re-collection Audio data x1 Audio data x2 Audio data x3 Zero crossing rate 6 7 8

[0070] Method 2: In order to improve the accuracy of the compensation coefficient, in this embodiment of the application, if the zero-crossing rate difference between the zero-crossing rate of M consecutive frames of audio data in audio data x1, audio data x2, ..., audio data xj and the zero-crossing rate difference of M consecutive frames of audio data in audio data y1, audio data y2, ..., audio data yg all meet the preset zero-crossing rate difference range, and the zero-crossing rate change trend of the M consecutive frames of audio data in audio data y1, audio data y2, ..., audio data yg is the same as that of the M consecutive frames of audio data in audio data y1, audio data y2, ..., audio data yg, then the audio playback device will take the Mth frame of audio data in each frame of audio data as the target frame audio data.

[0071] For example, assuming M is 3, and the preset zero-crossing rate difference range is 0 to 1, as shown in Table 1, the zero-crossing rates of three consecutive audio frames in the test audio—audio data y1, audio data y2, and audio data y3—are 5, 6, and 7, respectively. As shown in Table 2, the zero-crossing rates of three consecutive audio frames in the re-collected audio—audio data x1, audio data x2, and audio data x3—are 6, 7, and 8, respectively. Obviously, the zero-crossing rates between audio data x1 and audio data y1, between audio data x2 and audio data y2, and between audio data x3 and audio data y3 all conform to the preset zero-crossing rate difference range of 0 to 1. Furthermore, the zero-crossing rate changes of audio data y1, audio data y2, and audio data y3, and between audio data x1, audio data x2, and audio data x3, all show an increasing trend. Therefore, the audio playback device can use audio data x3 as the target frame audio data.

[0072] It should be noted that, in this embodiment of the application, in order to improve the determination efficiency, the zero-crossing rate variation pattern of each frame of audio data in the test audio can be set to change according to a certain pattern. For example, the zero-crossing rate variation trend of each frame of audio data in the test audio is an increasing trend. In this way, when the audio playback device obtains a frame of re-sampled audio data, it can quickly locate the corresponding audio data in the test file.

[0073] S6012. The audio playback device obtains the compensation coefficient corresponding to each frequency band based on the sampled energy value of each frequency band in the target frame audio data and the real energy value of each frequency band in the source frame audio data. The source frame audio data is used to characterize the audio data corresponding to the target frame audio data in the test audio.

[0074] Specifically, when executing S6012, the audio playback device may perform, but is not limited to, the following steps:

[0075] A1. The audio playback device determines the average energy value of the re-sampled data based on the re-sampled energy values ​​of each frequency band in the target frame audio data.

[0076] In the embodiments of this application, see the following: Figure 7 As shown, an audio playback device can filter the target frame audio data using a comb filter to obtain frequency band data for each frequency band. Based on the frequency band data, the resampled energy value for each frequency band can be obtained. The following explanation uses the resampled energy values ​​for N frequency bands as an example.

[0077] Let sn(i) represent the frequency band data of the i-th sampling point in the n-th frequency band, where n ranges from (0, N], i represents the sample index in each frame, and i ranges from (0, FrameLen). FrameLen represents the number of sampling points in each frame, and FrameLen ranges from (0, 100000). In practical applications, FrameLen can be 512.

[0078] For details, please refer to Figure 8 As shown, the audio device can use the following formula to calculate the sampling energy value for each frequency band:

[0079]

[0080] Where En represents the energy value collected in the nth frequency band.

[0081] For example, referring to Table 3, taking the first test as an example, when the test number is 1, the energy recovery value of frequency band 1 is -2, the energy recovery value of frequency band 1 is -3, and the energy recovery value of frequency band N is -2.

[0082] Table 3. Retrieved energy values ​​and actual energy values ​​for N frequency bands

[0083]

[0084] After the audio playback device determines the re-sampling energy value for each frequency band, the average re-sampling energy value can be calculated according to formula (2):

[0085]

[0086] Where M represents the average energy value obtained from the retrieval, and BandCount represents the number of frequency bands of the comb filter. The value of BandCount ranges from [1, 2048], and in practical applications, the value of BandCount is usually 8. In the embodiments of this application, the audio playback device directly performs subsequent calculations based on the frequency band data of each frequency band obtained from the filtering. Therefore, BandCount can also be understood as N.

[0087] Taking the first test in Table 3 as an example, assuming that the value of BandCount is 3, that is, N=3, then after the audio playback device determines that the sampling energy value of band 1 is -2, the sampling energy value of band 1 is -3, and the sampling energy value of band N is -2, the above formula (2) is used to calculate that the value of M is -7 / 3, which is approximately -2.33.

[0088] A2. The audio playback device determines the true average energy value based on the true energy values ​​of each frequency band in the source frame audio data.

[0089] Taking audio data x3 as the target frame audio data as an example, audio data x3 corresponds to audio data y3. Therefore, audio data y3 is the source frame audio data.

[0090] Since the method of obtaining the true energy value of each frequency band in the source frame audio data is the same as the method of obtaining the sampled energy value in step A1, and the method of determining the true average energy value is the same as the method of determining the sampled average energy value in step A1, it will not be repeated here.

[0091] In this embodiment of the application, ESn can be used to represent the true energy value in the nth frequency band, and ES can be used to represent the true average energy value.

[0092] A3. The audio playback device obtains the audio scaling factor based on the average energy value of the back sample and the actual average energy value, and obtains the corresponding compensation factor for each frequency band based on the audio scaling factor.

[0093] Specifically, in this embodiment of the application, the audio playback device can use the ratio between the actual average energy value and the resampled average energy value as the audio scaling factor.

[0094] For example, assuming the true average energy value is 1 and the resampled average energy value is 2, the audio playback device uses the ratio 1 / 2 between the true average energy value and the resampled average energy value as the audio scaling factor.

[0095] After obtaining the audio scaling factor, the audio playback device can scale the resampled energy value of each frequency band based on the audio scaling factor to obtain each scaled energy value. Then, each scaled energy value is compared with the resampled energy value of each frequency band to obtain the corresponding compensation factor for each frequency band.

[0096] Specifically, the audio playback device can obtain the various scaling energy values ​​according to formula (3):

[0097] WEn=En*W Formula (3)

[0098] Where WEn represents the scaling energy value of the nth frequency band, En represents the re-sampling energy value of the nth frequency band, and W represents the audio scaling factor.

[0099] The audio playback device can obtain the compensation coefficients for each frequency band according to formula (4):

[0100] Wbn = ESN - WEn Formula (4)

[0101] Where Wbn represents the compensation coefficient of the nth frequency band, and ESn represents the actual energy value of the nth frequency band.

[0102] For example, suppose that the sampled energy value of frequency band 1 is -2, the actual energy value of frequency band 1 is -1.5, and the audio scaling factor is 0.5. The audio playback device scales the sampled energy value of frequency band 1 based on the audio scaling factor to obtain a scaled energy value of -1. Then, the scaled energy value of -1 is compared with the sampled energy value of -1.5 of frequency band 1 to obtain the compensation factor of 0.5 corresponding to frequency band 1.

[0103] S602. The audio playback device acquires the audio to be played and performs energy compensation on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio.

[0104] It should be noted that in some implementations, the audio playback device can use a comb filter to filter each frame of the audio data to be played, obtain the frequency band data of each frequency band in each frame of the audio data to be played, and then perform energy compensation on the frequency band data of each frequency band in each frame of the audio data to be played according to the determined compensation coefficients, so as to obtain the target audio.

[0105] Taking frequency band 1 as an example, the compensation coefficient corresponding to frequency band 1 is 0.5. After the audio playback device acquires the audio to be played, it filters each frame of audio data in the audio to be played through a comb filter to obtain the frequency band data of frequency band 1 in each frame of audio data to be played. Then, according to the compensation coefficient of 0.5 corresponding to frequency band 1, energy compensation is performed on the frequency band data of frequency band 1 in each frame of audio data to be played. Similarly, the audio playback device performs energy compensation on the frequency band data of each frequency band in the frame of audio data to be played to obtain the target audio.

[0106] Considering that the compensation coefficients for each frequency band vary at different volumes, audio playback devices can play test audio at different volumes to obtain the compensation coefficients for each frequency band at different volumes.

[0107] Starting with the playback volume of the test audio, the audio playback device gradually decreases the playback volume according to a preset volume decrease step size until the data acquisition stop condition is met, obtaining the compensation coefficients corresponding to each frequency band at each playback volume. Specifically, for each decrease in playback volume, the following operations are performed on the target frame audio data:

[0108] The audio playback device plays the target frame audio data according to the current playback volume, and obtains the retrieval energy value of each frequency band at the current playback volume; based on the retrieval energy value of each frequency band at the current playback volume, the audio playback device determines the corresponding compensation coefficient for each frequency band at the current playback volume.

[0109] For details, please refer to Figure 9 As shown, the audio playback device performs the following steps:

[0110] S901. The audio playback device plays the target frame audio data at the current playback volume. When playing the target frame audio data for the first time, the current playback volume is the same as the playback volume of the test audio.

[0111] S902, the audio playback device obtains the retrieval energy values ​​of each frequency band at the current playback volume.

[0112] S903. The audio playback device determines the compensation coefficient corresponding to each frequency band at the current playback volume based on the sampled energy value of each frequency band at the current playback volume.

[0113] S904. The audio playback device reduces the playback volume according to a preset volume decrease step.

[0114] S905. The audio playback device determines whether the data acquisition stop condition is met. If yes, execute S906; otherwise, return to execute S901.

[0115] It should be noted that in this embodiment of the application, test audio can also be played in S901 to redetermine the target audio data, and the compensation coefficients corresponding to each frequency band at the previous playback volume can be determined based on the new target audio data, which will not be elaborated here.

[0116] For example, referring to Table 3, test numbers 2, 3, 4, 5, and 6 correspond to the sampled energy values ​​and actual energy values ​​of each frequency band at playback volumes of -6dB, -12dB, -18dB, -24dB, and -30dB, respectively. Based on the sampled energy values ​​of each frequency band at playback volumes of -6dB, -12dB, -18dB, -24dB, and -30dB, the audio playback device determines the corresponding compensation coefficients for each frequency band at playback volumes of -6dB, -12dB, -18dB, -24dB, and -30dB. Since the process of determining the compensation coefficients for each frequency band at each playback volume is the same as the process described above, it will not be repeated here.

[0117] Specifically, the data collection can stop under any of the following conditions:

[0118] Condition 1: The current playback volume is lower than the preset volume threshold;

[0119] Condition 2: The current volume has decreased a preset number of times.

[0120] The preset volume threshold value ranges from [-100, 0). For example, the preset volume threshold value can be -80 dB.

[0121] Assuming a volume decrease step size of 6dB and a preset volume threshold of -80dB, starting from the playback volume of the test audio, the audio playback device gradually decreases the playback volume in 6dB steps until the current playback volume is less than -80dB. Taking the first volume decrease as an example, if the current playback volume is -12dB after the first decrease, then for the target frame audio data, the audio playback device plays the target frame audio data at the current playback volume of -12dB, obtaining the retrieval energy values ​​of each frequency band at -12dB. Based on these retrieval energy values, the audio playback device determines the corresponding compensation coefficients for each frequency band at the current playback volume.

[0122] After obtaining the compensation coefficients corresponding to each frequency band at each playback volume, the audio playback device can perform energy compensation on each frequency band of the audio to be played at the target playback volume.

[0123] Specifically, the audio playback device acquires the target playback volume of the audio to be played; based on the target playback volume and the compensation coefficients corresponding to each frequency band at each playback volume, it determines the compensation coefficients corresponding to each frequency band at the target playback volume; based on the compensation coefficients corresponding to each frequency band at the target playback volume, it performs energy compensation on each frequency band in the audio to be played to obtain the target audio.

[0124] For example, assuming the target playback volume is -6dB, after the audio playback device obtains the target playback volume of the audio to be played, it determines the compensation coefficients corresponding to each frequency band when the playback volume is -6dB, based on the target playback volume and the compensation coefficients corresponding to each frequency band when the playback volume is -6dB, -12dB, -18dB, -24dB, and -30dB. Then, based on the compensation coefficients corresponding to each frequency band when the playback volume is -6dB, it performs energy compensation on each frequency band in the audio to be played to obtain the target audio.

[0125] Since the playback volume decreases in increments, the compensation coefficients for each frequency band at each playback volume determined by the audio playback device do not include the compensation coefficients for each frequency band at the target playback volume. Therefore, in this embodiment, if the compensation coefficients for each frequency band at each playback volume do not include the compensation coefficients for each frequency band at the target playback volume, the audio playback device determines the two playback volumes adjacent to the target playback volume from the playback volumes based on the volume values; and determines the compensation coefficients for each frequency band at the target playback volume based on the compensation coefficients for each frequency band at the two playback volumes.

[0126] The two playback volumes adjacent to the target playback volume can be either the playback volumes whose volume values ​​are closest to the target playback volume, or any two playback volumes from among all available playback volumes; there are no restrictions on this. To improve compensation accuracy, the two playback volumes adjacent to the target playback volume can be the playback volumes whose volume values ​​are closest to the target playback volume.

[0127] See Figure 10 As shown, the compensation coefficients for each frequency band can be determined according to formula (5):

[0128]

[0129] Wherein, Wnew is used to characterize the compensation coefficient corresponding to the target frequency band at the target playback volume, ESnew is used to characterize the true energy value corresponding to the target frequency band at the target playback volume, the target frequency band is any one of the frequency bands, WbA and WbC are used to characterize the true energy value of the target frequency band at the two playback volumes, and ESA and ESC are used to characterize the compensation coefficient of the target frequency band at the two playback volumes.

[0130] For example, see Figure 7As shown, the values ​​of WbA and WbC are -8 and -11, respectively; the values ​​of ESA and ESC are -6 and -18, respectively; and the value of ESnew is -14. Therefore, the audio playback device determines the compensation coefficient corresponding to the target frequency band at the target playback volume to be -10 based on the compensation coefficients corresponding to the target frequency band at the two playback volumes.

[0131] Based on the same inventive concept, this disclosure provides an audio processing apparatus, see reference. Figure 11 As shown, it includes at least: a test unit 1101 and a compensation unit 1102.

[0132] The test unit 1101 is used to acquire the back-collected audio based on the test audio, and determine the compensation coefficient corresponding to each frequency band based on the back-collected energy value of each frequency band in the back-collected audio.

[0133] The compensation unit 1102 is used to acquire the audio to be played and perform energy compensation on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio.

[0134] In some embodiments, when determining the compensation coefficient corresponding to each frequency band based on the sampled energy value of each frequency band in the sampled audio, the test unit 1101 is used to:

[0135] Based on the zero-crossing rate of each frame of audio data in the re-sampled audio, the target frame audio data is determined from each frame of data; based on the re-sampled energy value of each frequency band in the target frame audio data and the real energy value of each frequency band in the source frame audio data, the compensation coefficient corresponding to each frequency band is obtained, wherein the source frame audio data is used to characterize the audio data corresponding to the target frame audio data in the test audio.

[0136] In some embodiments, based on the zero-crossing rate of each frame of audio data in the re-sampled audio, the target frame audio data is determined from the frame data, and the test unit 1101 is used for:

[0137] If the difference between the zero-crossing rates of M consecutive frames of audio data in each frame and the zero-crossing rates of M consecutive frames of audio data in the test audio all conform to a preset range of zero-crossing rate differences, then the Mth frame of audio data in the M consecutive frames of audio data is taken as the target frame audio data; or...

[0138] If the difference between the zero-crossing rates of the M consecutive frames of audio data in each frame and the zero-crossing rates of the M consecutive frames of audio data in the test audio both conform to a preset range of zero-crossing rate differences, and the zero-crossing rate changes of the M consecutive frames of audio data in each frame and the M consecutive frames of audio data in the test audio have the same trend, then the Mth frame of audio data in the M consecutive frames of audio data in each frame is taken as the target frame audio data.

[0139] In some embodiments, the testing unit 1101 is used to obtain the compensation coefficient corresponding to each frequency band based on the sampled energy value of each frequency band in the target frame audio data and the actual energy value of each frequency band in the source frame audio data.

[0140] Based on the re-sampling energy values ​​of each frequency band in the target frame audio data, the average re-sampling energy value is determined; based on the actual energy values ​​of each frequency band in the source frame audio data, the actual average energy value is determined; based on the average re-sampling energy value and the actual average energy value, the audio scaling factor is obtained, and based on the audio scaling factor, the compensation factor corresponding to each frequency band is obtained.

[0141] In some embodiments, the test unit 1101 is used to obtain the compensation coefficients corresponding to each frequency band based on the audio scaling factor, and the test unit 1101 is used to:

[0142] Based on the audio scaling factor, the re-sampling energy value of each frequency band is scaled to obtain each scaled energy value; each scaled energy value is compared with the re-sampling energy value of each frequency band to obtain the compensation coefficient corresponding to each frequency band.

[0143] In some embodiments, after determining the compensation coefficient corresponding to each frequency band based on the retrieval energy value of each frequency band in the retrieval audio, the test unit 1101 is further configured to:

[0144] Starting from the playback volume of the test audio, the playback volume is sequentially decreased according to a preset volume decrease step size to obtain the compensation coefficients corresponding to each frequency band at each playback volume, until the data acquisition stop condition is met. Each time the playback volume is decreased, the following operations are performed on the target frame audio data:

[0145] Play the target frame audio data according to the current playback volume to obtain the retrieval energy value of each frequency band at the current playback volume; based on the retrieval energy value of each frequency band at the current playback volume, determine the compensation coefficient corresponding to each frequency band at the current playback volume.

[0146] In some embodiments, when energy compensation is performed on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio, the compensation unit 1102 is used for:

[0147] Obtain the target playback volume of the audio to be played; determine the compensation coefficients corresponding to each frequency band under the target playback volume based on the target playback volume and the compensation coefficients corresponding to each frequency band under each playback volume; perform energy compensation on each frequency band in the audio to be played based on the compensation coefficients corresponding to each frequency band under the target playback volume to obtain the target audio.

[0148] In some embodiments, the compensation unit 1102 is further configured to:

[0149] If the compensation coefficients corresponding to each frequency band at each playback volume do not include the compensation coefficients corresponding to each frequency band at the target playback volume, then based on the volume value, determine the two playback volumes adjacent to the target playback volume from the playback volumes; and based on the compensation coefficients corresponding to each frequency band at the two playback volumes, determine the compensation coefficients corresponding to each frequency band at the target playback volume.

[0150] Based on the same inventive concept, embodiments of this application provide a storage medium that, when instructions in the storage medium are executed by a processor, enables the processor to execute any of the methods implemented by the audio processing device in the above process.

[0151] In summary, in this embodiment, the audio playback is compensated using the compensation coefficients corresponding to each frequency band. This improves the audio playback effect while ensuring consistent acoustic frequency balance. Furthermore, adaptive compensation can be performed for different playback devices, improving the usability and compensation efficiency of the audio playback devices.

[0152] For system / device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments.

[0153] It should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0154] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] Although preferred embodiments of this disclosure have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0159] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A display device, characterized in that, include: A monitor is used to display the user interface. The controller is configured as follows: Acquire the re-collected audio based on the test audio, and determine the target frame audio data from each frame of audio data based on the zero-crossing rate of each frame of audio data in the re-collected audio. Based on the re-sampling energy values ​​of each frequency band in the target frame audio data, the average re-sampling energy value is determined; The true average energy value is determined based on the true energy values ​​of each frequency band in the source frame audio data. The source frame audio data is used to characterize the audio data corresponding to the target frame audio data in the test audio; Based on the average energy value of the back sample and the actual average energy value, an audio scaling factor is obtained, and based on the audio scaling factor, the back sample energy value of each frequency band is scaled to obtain each scaled energy value. Each scaled energy value is compared with the sampled energy value of each frequency band to obtain the compensation coefficient corresponding to each frequency band. The audio to be played is acquired, and energy compensation is performed on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio.

2. The device as described in claim 1, characterized in that, When determining the target frame audio data from the acquired audio data based on the zero-crossing rate of each frame, the controller is configured to: If the difference between the zero-crossing rates of M consecutive frames of audio data in each frame and the zero-crossing rates of M consecutive frames of audio data in the test audio both meet the preset range of zero-crossing rate difference, then the Mth frame of audio data in the M consecutive frames of audio data in each frame will be taken as the target frame audio data. or, If the difference between the zero-crossing rates of the M consecutive frames of audio data in each frame and the zero-crossing rates of the M consecutive frames of audio data in the test audio both conform to a preset range of zero-crossing rate differences, and the zero-crossing rate changes of the M consecutive frames of audio data in each frame and the M consecutive frames of audio data in the test audio have the same trend, then the Mth frame of audio data in the M consecutive frames of audio data in each frame is taken as the target frame audio data.

3. The device as described in claim 1 or 2, characterized in that, After comparing each scaled energy value with the sampled energy value of each frequency band to obtain the corresponding compensation coefficient for each frequency band, the controller is further configured to: Starting from the playback volume of the test audio, the playback volume is sequentially decreased according to a preset volume decrease step size to obtain the compensation coefficients corresponding to each frequency band at each playback volume, until the data acquisition stop condition is met. Each time the playback volume is decreased, the following operations are performed on the target frame audio data: Play the target frame audio data according to the current playback volume, and obtain the retrieval energy value of each frequency band at the current playback volume; Based on the sampled energy values ​​of each frequency band at the current playback volume, the compensation coefficients corresponding to each frequency band at the current playback volume are determined.

4. The device as described in claim 3, characterized in that, When performing energy compensation on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio, the controller is configured to: Obtain the target playback volume of the audio to be played; Based on the target playback volume and the compensation coefficients corresponding to each frequency band at each playback volume, determine the compensation coefficients corresponding to each frequency band at the target playback volume; Based on the compensation coefficients corresponding to each frequency band at the target playback volume, energy compensation is performed on each frequency band in the audio to be played to obtain the target audio.

5. The device as described in claim 4, characterized in that, The controller is also configured to: If the compensation coefficients corresponding to each frequency band at each playback volume do not include the compensation coefficients corresponding to each frequency band at the target playback volume, then based on the volume value, determine the two playback volumes adjacent to the target playback volume from among the playback volumes. Based on the compensation coefficients corresponding to each frequency band at the two playback volumes, the compensation coefficients corresponding to each frequency band at the target playback volume are determined.

6. An audio processing method, characterized in that, include: Acquire the re-collected audio based on the test audio, and determine the target frame audio data from each frame of audio data based on the zero-crossing rate of each frame of audio data in the re-collected audio. Based on the re-sampling energy values ​​of each frequency band in the target frame audio data, the average re-sampling energy value is determined; The true average energy value is determined based on the true energy values ​​of each frequency band in the source frame audio data. The source frame audio data is used to characterize the audio data corresponding to the target frame audio data in the test audio; Based on the average energy value of the back sample and the actual average energy value, an audio scaling factor is obtained, and based on the audio scaling factor, the back sample energy value of each frequency band is scaled to obtain each scaled energy value. Each scaled energy value is compared with the sampled energy value of each frequency band to obtain the compensation coefficient corresponding to each frequency band. The audio to be played is acquired, and energy compensation is performed on each frequency band of the audio to be played according to the determined compensation coefficients to obtain the target audio.

Images