Volume adjustment method, device and apparatus

Abstract

The application discloses a volume adjusting method, device and equipment, and belongs to the technical field of audio processing. The volume adjusting method comprises the following steps: in the case that it is detected that volume adjustment is performed on an object, acquiring a first audio signal of an audio and video at a first time, first environmental noise data of each first time point in a first sliding time window, and a first volume of the object at each second time point in a second sliding time window; in the case that a first voiceprint feature of the first audio signal matches a reference voiceprint feature, determining an environmental noise feature at the first time according to the first environmental noise data; determining a volume preference coefficient of a user at the first time according to the first volume of the object at each second time point; and adjusting a second volume of the first audio signal at the first time to a third volume according to the environmental noise feature and the volume preference coefficient.

Description

Technical Field

[0001] This application belongs to the field of audio and video processing technology, specifically relating to a volume adjustment method, device, and equipment. Background Technology

[0002] Due to the varying skill levels and production methods of audio and video producers, users often encounter uneven volume issues when playing audio and video on electronic devices. This means that the volume of some characters in the audio or video is too high, while the volume of others is too low.

[0003] In different viewing environments, due to the uneven volume of characters in the audio and video, users need to frequently adjust the volume manually in order to ensure that they can clearly hear the dialogue of each character in every frame of the audio and video, which is cumbersome. Summary of the Invention

[0004] The purpose of this application is to provide a volume adjustment method, apparatus, and device that can solve the problem in related technologies where users need to frequently and manually adjust the volume due to different viewing environments and uneven volume of people in audio and video, which is cumbersome.

[0005] In a first aspect, embodiments of this application provide a volume adjustment method, including:

[0006] When volume adjustment is detected for an object, the first audio signal of the object's audio and video at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the object at each second time point within the second sliding time window are obtained. Both the first and second sliding time windows are time windows before the first moment.

[0007] When the first voiceprint feature of the first audio signal matches the reference voiceprint feature, the environmental noise feature of the viewing environment of the audio and video at the first moment is determined based on each first environmental noise data, and the reference voiceprint feature is the voiceprint feature of the object.

[0008] Based on the first volume of the object at each second time point, determine the user's volume preference coefficient at the first moment;

[0009] Based on the environmental noise characteristics and volume preference coefficient, the second volume of the first audio signal at the first moment is adjusted to the third volume.

[0010] Secondly, embodiments of this application provide a volume adjustment device, including: an acquisition module, a determination module, and an adjustment module;

[0011] The acquisition module is used to acquire the first audio signal of the object corresponding to the audio and video at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the object at each second time point within the second sliding time window when the volume adjustment of the object is detected. The first sliding time window and the second sliding time window are both time windows before the first moment.

[0012] The determination module is used to determine the environmental noise characteristics of the viewing environment of the audio and video at a first moment based on each first environmental noise data when the first voiceprint feature of the first audio signal matches the reference voiceprint feature. The reference voiceprint feature is the voiceprint feature of the object.

[0013] The determination module is also used to determine the user's volume preference coefficient at the first moment based on the first volume of the target object at each second time point;

[0014] The adjustment module is used to adjust the second volume of the first audio signal to the third volume at the first moment based on the characteristics of the ambient noise and the volume preference coefficient.

[0015] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0017] Fifthly, embodiments of this application provide a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the steps of the method described in the first aspect.

[0018] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the steps of the method described in the first aspect.

[0019] In this embodiment, when volume adjustment is detected for an object and the first voiceprint feature of the first audio signal matches the reference voiceprint feature of the object, the environmental noise features at a first moment can be determined based on each first environmental noise data. Furthermore, based on the first volume of the object at each second time point, the user's volume preference coefficient at the first moment can be determined. Then, based on the environmental noise features and the volume preference coefficient, the second volume of the first audio signal at the first moment can be automatically adjusted to a third volume. In other words, this embodiment can adaptively adjust the volume of the object under different viewing environments based on environmental noise and user preferences, providing the user with the best auditory experience in various viewing environments. This eliminates the need for frequent manual volume adjustments due to different viewing environments and uneven volume among characters, thereby reducing user operations and improving the smoothness of the viewing experience. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a volume adjustment method provided in this application embodiment;

[0021] Figure 2 A schematic diagram of a window adjustment factor provided in an embodiment of this application;

[0022] Figure 3 A flowchart illustrating another volume adjustment method provided in this application embodiment;

[0023] Figure 4 A flowchart illustrating another volume adjustment method provided in this application embodiment;

[0024] Figure 5 A schematic diagram of a spectral feature weighting function provided in an embodiment of this application;

[0025] Figure 6 A schematic diagram of a noise intensity weighting function provided in an embodiment of this application;

[0026] Figure 7 A schematic diagram of a first weighting coefficient provided in an embodiment of this application;

[0027] Figure 8 A schematic diagram of a video playback interface provided in an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the structure of a volume adjustment device provided in an embodiment of this application;

[0029] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0030] Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0032] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0033] Due to the varying skill levels and production methods of audio and video producers, users often encounter uneven volume issues when playing audio and video on electronic devices. This means that the volume of some characters in the audio or video is too high, while the volume of others is too low.

[0034] In different viewing environments, due to the uneven volume of characters in the audio and video, users need to frequently adjust the volume manually in order to ensure that they can clearly hear the dialogue of each character in every frame of the audio and video, which is cumbersome.

[0035] Therefore, this application provides a volume adjustment method, apparatus, and device that can solve the problem in related technologies where users need to frequently and manually adjust the volume due to different viewing environments and uneven volume of people in audio and video, which is cumbersome.

[0036] The volume adjustment method, apparatus, and device provided in this application will be described in detail below with reference to specific embodiments.

[0037] Figure 1 A flowchart of a volume adjustment method provided in this application embodiment is shown below. Figure 1 As shown, the volume adjustment method may include the following steps:

[0038] S110. When volume adjustment is detected for the object, the first audio signal of the object's audio and video at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the object at each second time point within the second sliding time window are obtained.

[0039] Both the first sliding time window and the second sliding time window are time windows prior to the first moment.

[0040] S120, when the first voiceprint feature of the first audio signal matches the reference voiceprint feature, the environmental noise feature of the viewing environment of the audio and video at the first moment is determined based on each first environmental noise data.

[0041] Among them, the reference voiceprint feature is the voiceprint feature of the object.

[0042] S130. Determine the user's volume preference coefficient at the first moment based on the object's first volume at each second time point.

[0043] S140. Based on the environmental noise characteristics and volume preference coefficient, adjust the second volume of the first audio signal to the third volume at the first moment.

[0044] In this embodiment, when volume adjustment is detected for an object and the first voiceprint feature of the first audio signal matches the reference voiceprint feature of the object, the environmental noise features at a first moment can be determined based on each first environmental noise data. Furthermore, based on the first volume of the target object at each second time point, the user's volume preference coefficient at the first moment can be determined. Then, based on the environmental noise features and the volume preference coefficient, the second volume of the first audio signal at the first moment can be automatically adjusted to a third volume. In other words, this embodiment can adaptively adjust the volume of the object in different viewing environments based on environmental noise and user preferences, providing the user with the best auditory experience in various viewing environments. This eliminates the need for frequent manual volume adjustments due to different viewing environments and uneven volume among characters, thereby reducing user operations and improving the smoothness of the viewing experience.

[0045] The above steps are explained in detail below:

[0046] In S110, the audio and video here can be the audio and video that the user is playing. The object can be an object in the audio and video that can emit audio signals. In this embodiment, the object is a person, that is, this embodiment can adaptively adjust the volume of the person.

[0047] In this embodiment of the application, users can set the volume for each character appearing in the audio and video separately, thereby achieving a personalized viewing experience.

[0048] The first moment can be the current moment or a moment after the current moment. For example, if the user enables the function of adjusting the volume of an object at the current moment, the embodiment of this application can adaptively adjust the volume of the object at subsequent moments, that is, the first moment. If the user enables the function of adjusting the volume of an object at a moment before the current moment, the embodiment of this application can adaptively adjust the volume of the object at the current moment, that is, the first moment.

[0049] In other words, after the function of detecting and adjusting the volume of the target object is enabled in the embodiments of this application, the volume of the target object appearing in the audio and video at subsequent times can be automatically adjusted, eliminating the need for the user to manually adjust it frequently, reducing the user's operation, and without disrupting the user's continuous viewing experience.

[0050] For example, the function of adjusting volume for specific objects can be enabled either before or during the playback of the audio / video. For instance, if the user enables this function before playback, the electronic device can adaptively adjust the volume of each object appearing in the audio / video during playback. If the user enables this function during playback, it means that the volume of only that specific object will be adaptively adjusted. For example, if the user wants to adaptively adjust the volume of object A and object B, they can enable the volume adjustment function for object A separately, and the volume adjustment function for object B separately. This achieves personalized volume adjustment.

[0051] The first audio signal is the audio signal of the audio and video at the first moment. Both the first and second sliding time windows are time windows containing multiple time points prior to the first moment; the first and second sliding time windows can be the same or different. The first time point is a time point within the first sliding time window, and the second time point is a time point within the second sliding time window.

[0052] The first environmental noise data is the environmental noise data of the viewing environment where the audio and video are located. For example, the environmental noise data of the viewing environment at each first time point within the first sliding time window can be collected.

[0053] The first volume refers to the first volume of the object at each of the second time points within the second sliding time window.

[0054] For example, when volume adjustment is detected for an object, the audio signal of the audio-visual video at a first moment, the ambient noise data of the viewing environment at each first time point, and the volume of the object at each second time point can be collected to provide a basis for subsequent volume adjustment.

[0055] In S120, in order to achieve precise volume adjustment, after acquiring the first audio signal at the first moment of the audio-visual interaction, it is necessary to determine whether the object to which the first audio signal belongs is the object corresponding to the activation of the volume adjustment function. For the convenience of subsequent description, this embodiment of the application refers to the object corresponding to the activation of the volume adjustment function as the target object.

[0056] For example, it can be determined whether the first audio signal belongs to the target object by judging whether the voiceprint features of the first audio signal match the voiceprint features of the target object. Here, the voiceprint features of the first audio signal are also known as the first voiceprint features, and the voiceprint features of the target object are also known as the reference voiceprint features.

[0057] For example, the voiceprint features of the first audio signal can be extracted in the following manner:

[0058] a) Perform a Fast Fourier Transform (FFT) on the first audio signal to convert the first audio signal from the time domain to the frequency domain: S(f) = FFT{s(t)}, where s(t) is the first audio signal in the time domain, S(f) is the first audio signal in the frequency domain, and f is the frequency.

[0059] b) Calculate the energy (or power) spectrum P(f) of the FFT result: P(f) = |S(f)| 2 .

[0060] c) Design a set of filters m i (f) Assuming the frequency range of the audio signal is 300Hz to 8000Hz, the frequency f of the audio signal can be converted to the Mel frequency using the following formula:

[0061]

[0062] The above conversion can convert the frequency range (300Hz to 8000Hz) to 401.25Mels to 2834.99Mels.

[0063] On the Mel scale, the range from the lowest to the highest frequency (401.25 Mels to 2834.99 Mels) can be divided into N+1 equally spaced points, where N is the number of filters. For example, for 26 filters, this will generate 27 equally spaced points. These Mel-scale frequency points can then be converted back to actual frequencies using the following formula:

[0064]

[0065] For each filter m i (f), whose center frequency is determined by two adjacent Mel-scale frequency points, the i-th filter mi The frequency response of (f) is defined as follows:

[0066] When f i-1 ≤f <f i hour, When f i ≤f <f i+1 hour, When f <f i-1 or f≥f i+1 At that time, m i (f) = 0. Therefore, a set of filters m can be obtained. i (f).

[0067] d) Process the energy spectrum P(f) obtained above through a Mel filter bank to obtain the Mel spectrum P. mel (f): P mel (f)=∑ f m i (f)·P(f), that is, for each Mel filter, a Mel spectrum P is obtained. mel (f). The Mel filter is the filter obtained in step c) above.

[0068] e) For each Mel spectrum P mel (f) Taking the logarithm, we get logP mel (f). For the Mel spectrum P mel (f) Taking the logarithm can make the final extracted voiceprint features closer to human ear perception.

[0069] f) For each log-Mel spectrum logP mel (f) Using the Discrete Cosine Transform (DCT), the Mel Frequency Cepstrum Coefficient (MFCC) is obtained: MFCC = DCT{logP mel (f)}. Each Mel filter corresponds to one MFCC, resulting in 26 MFCCs. In practical applications, only the 2nd to 13th MFCCs can be selected, as these MFCCs are generally considered to contain the most important information in the audio signal.

[0070] g) Combine the selected MFCCs from the 2nd to the 13th with their increment and acceleration coefficients to obtain the eigenvector F. stored eigenvector F storedThis refers to the voiceprint features extracted from the first audio signal. The increment coefficient is the first-order derivative of each selected MFCC over time, used to capture the rate of change of the spectrum. The acceleration coefficient is the first-order derivative of the increment coefficient, enhancing higher-order dynamic feature information.

[0071] Of course, other methods can also be used to extract the voiceprint features of the first audio signal, and this application does not limit the specific methods used in the embodiments.

[0072] After the first voiceprint feature of the first audio signal is extracted, it can be matched with the reference voiceprint feature of the target object. For example, the cosine similarity between the first voiceprint feature and the reference voiceprint feature can be calculated according to the following formula:

[0073]

[0074] Among them, F live F is the feature vector of the first voiceprint feature. stored For the feature vector of the reference voiceprint feature, Similarity(F) live ,F stored ) represents the cosine similarity between the first voiceprint feature and the reference voiceprint feature.

[0075] For example, if Similarity(F) live ,F stored If the similarity(F) is greater than or equal to a set threshold, it can be determined that the first voiceprint feature matches the reference voiceprint feature, that is, the first audio signal is the audio signal of the target object. live ,F stored If the first voiceprint feature is less than a set threshold, it can be determined that the first voiceprint feature does not match the reference voiceprint feature, meaning that the first audio signal is not the audio signal of the target object. This ensures the accuracy of audio recognition.

[0076] In some embodiments, environmental noise features can be used to characterize the noise level of the viewing environment within a time window; for example, environmental noise features may include the noise intensity of the viewing environment within a time window.

[0077] In some embodiments, environmental noise characteristics can also characterize the type of viewing environment. In this case, environmental noise characteristics may include spectral characteristics. For example, spectral characteristics may include spectral entropy. That is, the viewing environment can be classified into silent, low-noise, medium-noise, high-noise, and other environmental types by spectral entropy.

[0078] In some embodiments, environmental noise characteristics may also include both noise intensity and spectral characteristics.

[0079] The specific process for determining the characteristics of environmental noise can be found in the following examples.

[0080] In S130, the volume preference coefficient represents a user's preference for volume. Different users can have different volume preferences, that is, different volume preference coefficients. The same user can also have different volume preference coefficients at different times or in different environments.

[0081] Based on the target object's first volume at each historical time point (second time point), the user's volume preference coefficient at the first moment can be determined.

[0082] For example, based on the target object's first volume at each second time point, the average volume of the target object in the second sliding time window can be calculated, and the user's volume preference coefficient at the first time point can be determined based on the difference between the average volume and the reference volume.

[0083] In S140, the second volume is the second volume of the first audio signal at the first moment.

[0084] Once the environmental noise characteristics and the user's volume preference coefficient are determined, the volume of the first audio signal can be automatically adjusted to a third volume that is suitable for the viewing environment and the user's preferences at the first moment, based on the environmental noise characteristics and the user's volume preference coefficient. This eliminates the need for manual operation by the user, thereby simplifying the user's operation and improving the smoothness of the viewing experience.

[0085] Understandably, the size of the first sliding time window will affect the analysis results of the noise data, and thus the accuracy of subsequent volume adjustment. Therefore, it is necessary to accurately determine the size of the first sliding time window.

[0086] For example, prior to S110, the volume adjustment method may further include the following steps:

[0087] Acquire the second environmental noise data at each third time point within the initial sliding time window, where the initial sliding time window is the time window before the first time point;

[0088] Determine the mean and standard deviation of each second environmental noise data point within the initial sliding time window;

[0089] The window adjustment factor is determined based on the ratio of the standard deviation to the mean.

[0090] The size of the initial sliding time window is adjusted according to the window adjustment factor to obtain the first sliding time window.

[0091] The initial sliding time window is the size of the initially set sliding time window. The initial sliding time window may include multiple time points. For ease of distinction, each time point in the initial sliding time window is recorded as the third time point in this embodiment.

[0092] The second environmental noise data consists of environmental noise data collected at each of the third time points. The environmental noise level at the first time point is estimated by analyzing the second environmental noise data at the third time points.

[0093] For each second environmental noise data point within the initial sliding time window, its mean μ and standard deviation σ can be calculated.

[0094]

[0095]

[0096] Where T is the size of the initial sliding time window, that is, the initial sliding time window contains T time points, t is the first time point, and x(i) represents the second environmental noise data at the i-th time point.

[0097] The window adjustment factor is used to adjust the size of the initial sliding time window. For example, the window adjustment factor can be determined based on the ratio of the standard deviation to the mean.

[0098] For example, the rate of change R of the second environmental noise data can be calculated based on the ratio of the standard deviation to the mean. The rate of change R is used to measure the volatility of the second environmental noise data. For example, Here, ε is a small constant used to avoid μ = 0.

[0099] The window adjustment factor can be determined based on the rate of change R. For example, if the fluctuation of the second environmental noise data is large, that is, R is large, it means that a larger time window is needed to capture and analyze these fluctuations. Conversely, if the fluctuation of the second environmental noise data is large, that is, R is small, it means that the second environmental noise data within the time window is relatively stable. In this case, the time window can be narrowed to focus on analyzing the second environmental noise data within a shorter period of time.

[0100] For example, f(R) = 2*sigmoid(R), where f(R) is the window adjustment factor. When R is 0 (i.e., unchanged), f(R) returns 1, meaning the window size remains unchanged. The curve for the window adjustment factor f(R) can be found in [reference needed]. Figure 2 .

[0101] The size of the initial sliding time window can be adjusted based on the window adjustment factor f(R) to obtain the first sliding time window.

[0102] In some embodiments, the first sliding time window T new =T·f(R).

[0103] In some embodiments, to ensure that the size of the first sliding time window is reasonable, exemplarily, T final =max(T) min,min(T new ,T max Therefore, T can be made final It's neither too small nor too large. At this point, T can be... final As the first sliding time window. Where T min and T max These represent the minimum and maximum values ​​of the sliding time window, respectively.

[0104] This application embodiment calculates the rate of change of the second environmental noise data within the initial sliding time window to obtain a window adjustment factor, and then adjusts the size of the initial sliding time window based on the window adjustment factor, making the noise analysis more accurate and thus improving the accuracy of volume adjustment.

[0105] In some embodiments, such as Figure 3 As shown, the volume adjustment method may include the following steps:

[0106] S310. When volume adjustment is detected for an object, the first audio signal of the object's audio and video at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the object at each second time point within the second sliding time window are obtained.

[0107] S320, when the first voiceprint feature of the first audio signal matches the reference voiceprint feature, the environmental noise feature at the first moment is determined based on each first environmental noise data.

[0108] S330. Determine the weighted average volume of the object at the first moment based on the first volume at each second time point.

[0109] S340. Determine the difference between the weighted average volume and the reference volume.

[0110] S350. The product of the difference and the user preference compensation coefficient is determined as the user's volume preference coefficient at the first moment.

[0111] S360: Based on the environmental noise characteristics and volume preference coefficient, adjust the second volume of the first audio signal to the third volume at the first moment.

[0112] The processes of S310, S320, and S360 can be found in the above embodiments, and will not be repeated here for the sake of brevity.

[0113] The other steps described above are explained in detail below:

[0114] In S330, for example, the first volume at each second time point can be weighted and summed based on the Exponentially Weighted Moving Averages Model (EWMA) method to serve as the weighted average volume of the target object at the first time point.

[0115] EWMA assigns different weights to data in a time series. For example, it can assign different weights to the first volume at each second time point based on the distance between each second time point and the first time point. For example, the closer the second time point is to the first time point, the greater the weight of the corresponding first volume.

[0116] For example, V ewma (t)=α·V(t)+(1-α)·V ewma (t-1), where V ewma V(t) represents the weighted average volume at the first moment, and V(t) represents the second volume of the target object at the first moment. ewma (t-1) represents the weighted average volume at the previous time point, and α is a weighting coefficient between 0 and 1.

[0117] In practice, the sliding time window increases over time; therefore, V can be obtained using a recursive formula. ewma (t), where an initial value V can be set. ewma (t0) is used to initiate the above recursive process.

[0118] The user preference compensation coefficient is used to adjust the difference between the weighted average volume and the reference volume, reflecting the user's sensitivity or preference to volume changes.

[0119] For example, G user =β·(V ewma (t)-V0), where G user β is the user's volume preference coefficient at the first moment, β is the user preference compensation coefficient, the value of β can be preset, and V0 is the reference volume.

[0120] This application embodiment combines a sliding time window and a weighted average volume to calculate the user's volume preference coefficient. When adjusting the volume based on this volume preference coefficient, the adjusted volume can be closer to the user's ideal or expected level.

[0121] Understandably, different environments produce noise with different spectral characteristics. For example, a quiet library and a busy traffic intersection will produce drastically different noise spectra. In simple, monotonous noise environments (such as quiet environments), the spectral entropy is usually low because the noise is concentrated at a few frequencies. In complex, diverse noise environments (such as busy city centers), the spectral entropy is usually high because the noise is distributed over a wider frequency range.

[0122] By calculating the spectral entropy of ambient noise, the environment can be classified into categories such as silent, low noise, medium noise, and high noise. This allows us to determine the complexity and distribution characteristics of the viewing environment, providing a basis for subsequent volume adjustment.

[0123] Taking environmental noise characteristics, including spectral characteristics, as an example, the above-described S120 may include the following steps:

[0124] Decompose each first environmental noise data according to frequency to obtain at least one third environmental noise data corresponding to a frequency.

[0125] For each frequency, perform a Fourier transform on each third ambient noise data to obtain the Fourier transform result, and determine the first power of the third ambient noise data at that frequency based on the Fourier transform result.

[0126] Based on the first power, the noise spectrum entropy of the viewing environment at the first moment is determined, and the noise spectrum entropy is defined as the noise spectrum characteristic of the viewing environment at the first moment.

[0127] It is understood that the first environmental noise data usually contains noise data of at least one frequency. In order to ensure that users can have a better auditory experience in different viewing environments, for example, each first environmental noise data can be decomposed based on frequency to obtain environmental noise data corresponding to at least one frequency. For ease of description, the environmental noise data of different frequencies in this application embodiment are collectively referred to as third environmental noise data.

[0128] For example, an FFT can be performed on the third ambient noise data for each frequency to obtain the FFT result corresponding to that frequency.

[0129] For example, Where X(f) is the FFT result of the third ambient noise data x(i) at frequency f, and T final This represents the size of the first sliding time window.

[0130] The first power of the third ambient noise data at frequency f can be obtained based on the square of the FFT result, for example, P(f) = |X(f)| 2 One frequency corresponds to one primary power.

[0131] Based on the first power, the noise spectrum entropy of the viewing environment at the first moment can be determined, for example, H = -∑ f P(f)·log2P(f). Where H is the spectral entropy of the viewing environment at the first moment. For example, this noise spectral entropy can be determined as the spectral characteristic of the viewing environment at the first moment.

[0132] Determining the noise spectrum characteristics of the viewing environment helps to understand its features, thus ensuring a better auditory experience for users in different viewing environments.

[0133] Taking an environmental noise characteristic including a first noise intensity as an example, the above-described S120 may include the following steps:

[0134] The average noise power of the viewing environment within the first sliding time window is obtained by integrating each first environmental noise data.

[0135] Logarithmic processing of the average noise power yields the first noise intensity of the viewing environment at the first moment.

[0136] The first noise intensity is used to characterize the noise level of the viewing environment at a first moment. For example, the first noise intensity can be obtained by processing the first ambient noise data within a first sliding time window.

[0137] For example, the first ambient noise data within the first sliding time window can be integrated to obtain the average noise power of the viewing environment within the first sliding time window.

[0138] For example, Where P1 is the average noise power of the viewing environment within the first sliding time window, and x(q) is the first environmental noise data at the qth first time point.

[0139] By taking the logarithm of the average noise power, the initial noise intensity of the viewing environment at the first moment can be obtained. For example, N(t) = 10·log 10 (P1). Where N(t) is the first noise intensity.

[0140] By integrating and taking the logarithm of the first ambient noise data within the first sliding time window, the noise intensity of the viewing environment at the first moment is obtained, thus realizing the quantification of noise intensity. This helps to accurately adjust the volume based on the quantized first noise intensity.

[0141] Taking environmental noise characteristics, including noise spectrum characteristics and first noise intensity, as an example, such as... Figure 4 As shown, the volume adjustment method may include the following steps:

[0142] S410. When volume adjustment is detected for an object, the first audio signal of the object's audio and video at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the object at each second time point within the second sliding time window are obtained.

[0143] S420, when the first voiceprint feature of the first audio signal matches the reference voiceprint feature, the environmental noise feature of the viewing environment of the audio and video at the first moment is determined based on each first environmental noise data.

[0144] S430. Determine the user's volume preference coefficient at the first moment based on the object's first volume at each second time point.

[0145] S440. Generate the first weighting coefficient based on the noise spectrum characteristics and the first noise intensity.

[0146] S450. The first noise intensity is compensated using the first weighting coefficient to obtain the second noise intensity.

[0147] S460. Determine the noise compensation gain based on the difference between the second noise intensity and the reference noise intensity.

[0148] S470. Adjust the second volume to the third volume based on the noise compensation gain and volume preference coefficient.

[0149] The processes S410-S430 can be found in the above embodiments, and will not be repeated here for the sake of brevity.

[0150] The other steps described above are explained in detail below:

[0151] In S440, the first weighting coefficient is used to compensate for the first noise intensity. For example, the first weighting coefficient can be determined based on spectral characteristics and the first noise intensity.

[0152] For example, a spectral weighting coefficient can be generated based on the noise spectral characteristics, and an intensity weighting coefficient can be generated based on the first noise intensity;

[0153] The product of the spectral weighting coefficient and the intensity weighting coefficient is determined as the first weighting coefficient.

[0154] For example, spectral weighting coefficients can be generated based on noise spectral characteristics and combined with a spectral feature weighting function, which can be set as needed.

[0155] In some embodiments, the spectral feature weighting function can take the following form:

[0156]

[0157] Where H0 is the threshold of the noise spectral entropy, used to adjust the sensitivity of the spectral feature weighting function, and α(H) is the spectral weighting coefficient. For example, when H0 = 1.2, the curve of the spectral feature weighting function can be found in [reference needed]. Figure 5 .Depend on Figure 5 It can be seen that the spectral weighting coefficient increases with the increase of spectral entropy.

[0158] For example, the intensity weighting coefficients can be determined based on a first noise intensity and a noise intensity weighting function, which can be set as needed.

[0159] In some embodiments, the noise intensity weighting function may take the following form:

[0160]

[0161] Where β(N(t)) is the intensity weighting coefficient, N max This is the maximum value of the noise intensity, used to normalize the effect of the first noise intensity. For example, when N... max When the value is 100, the curve of the noise intensity weighting function can be found in [reference needed]. Figure 6 .Depend on Figure 6 It can be seen that the intensity weighting coefficient decreases as the intensity of the first frequency noise increases.

[0162] For example, the first weighting coefficient ω(H,N(t)) = α(H)·β(N(t)). The first weighting coefficient ω(H,N(t)) can change with the environment. The curve of the first weighting coefficient ω(H,N(t)) can be found in [reference needed]. Figure 7 ,Depend on Figure 7 It can be seen that when H and N(t) are low, ω(H,N(t)) is high, reflecting the sensitivity to noise in a quiet environment. As N(t) increases, ω(H,N(t)) gradually decreases, indicating that stronger noise compensation is needed in noisier environments.

[0163] The embodiments of this application can calculate a first weighting coefficient by combining the spectral entropy and the first noise intensity under various viewing environments to compensate for environmental noise, so that appropriate noise compensation can be performed under different viewing environments, thereby adaptively and dynamically adjusting the volume to meet the user's needs.

[0164] In S450, the first noise intensity can be compensated based on the first weighting coefficient to obtain the second noise intensity. For example, the first noise intensity can be compensated using the following formula:

[0165]

[0166] Where, N updated(t,H) represents the first noise intensity after compensation, which is also the second noise intensity.

[0167] In S460, noise compensation gain is used to adjust volume; for example, G env (t,H)=N updated (t,H)-N0, where G env (t,H) represents the noise compensation gain, and N0 represents the reference noise intensity.

[0168] In the S470, noise compensation gain G is used. env (t,H) and volume preference coefficient G user This allows you to adjust the second volume of the first audio signal to the third volume.

[0169] For example, it can be based on the noise compensation gain G env (t,H) and volume preference coefficient G user Calculate the target adjustment gain for the second volume, and use the target adjustment gain to adjust the second volume to the third volume.

[0170] For example, S final (t)=G video (t)*S original (t), where S final (t) represents the target volume of the first audio signal at the first moment, G video (t) is the target to adjust the gain, S original (t) represents the second volume of the first audio signal at the first moment.

[0171] Among them, the target adjustment gain can be achieved by adjusting the noise compensation gain G. env (t,H) and volume preference coefficient G user The weighted summation can be obtained, or it can be obtained through other calculation methods. This application does not limit the specific methods used in the embodiments.

[0172] This application embodiment utilizes noise spectrum characteristics and a first noise intensity to appropriately compensate for environmental noise under different noise conditions. Based on the compensated noise intensity, it determines the noise compensation gain and then adjusts the volume according to the noise compensation gain and the user's volume preference coefficient. This fully considers environmental factors and user preferences, and improves the accuracy of volume adjustment.

[0173] In some embodiments, S470 may include the following steps:

[0174] Determine the first difference between the noise compensation gain and the reference gain, and the second difference between the volume preference coefficient and the reference gain;

[0175] Determine the first weighted result of the first difference and the first weight, and the second weighted result of the second difference and the second weight. The first weight is the influence weight of the viewing environment, and the second weight is the influence weight of user preferences.

[0176] The target adjustment gain is determined by summing the first weighted result, the second weighted result, and the reference gain.

[0177] Adjust the second volume to the third volume using the target adjustment gain.

[0178] The reference gain, the first weight, and the second weight can all be preset by the user. For example, the first weight can be determined based on the degree of influence of the viewing environment on the volume, and the second weight can be determined based on the degree of influence of user preferences on the volume.

[0179] For example, G video (t)=G base +k env *(G env (t,H)-G base )+k user *(G user -G base ), where G base For reference gain, k env As the first weight, k user As the second weight. Through G video (t) can adjust the second volume to the third volume.

[0180] To avoid G video (t) is too large or too small, for example, G video (t)=min(max(G video (t),G min ),G max ), of which G min and G max These are the minimum and maximum values ​​for volume adjustment gain, respectively.

[0181] The embodiments of this application determine the target adjustment gain based on the noise compensation gain and the volume preference coefficient, thereby realizing the dynamic increase of the target adjustment gain and maintaining the audio quality under different noise levels.

[0182] To avoid drastic fluctuations in the reference gain in the short term that could affect the volume adjustment effect, in some embodiments, the reference gain can be determined as follows before "determining the first difference between the noise compensation gain and the reference gain, and the second difference between the volume preference coefficient and the reference gain":

[0183] During audio and video playback, the average volume adjustment of the object within a preset time period is determined based on the object's historical volume adjustment data. The historical volume adjustment data includes the number of volume adjustments and the volume adjustment amount for each adjustment. The preset time period is the time period before the first moment.

[0184] The initial gain is updated based on the product of the average volume adjustment and the smoothing factor to obtain the first gain;

[0185] The maximum value is determined from the lower limits of the first gain and the reference gain;

[0186] The minimum of the maximum value and the upper limit of the reference gain is determined as the reference gain.

[0187] For example, during audio and video playback, data on user volume adjustments made to a target object can be collected. In this embodiment, such data is recorded as historical volume adjustment data for the target object. Historical volume adjustment data may include when the user increased or decreased the volume, and the amount of adjustment.

[0188] Based on historical volume adjustment data, the average volume adjustment of the target object within a preset time period can be determined. The preset time period can be the time period from the start of audio / video playback to the first moment.

[0189] For example, the average volume adjustment amount Where N represents the number of times the user adjusts the volume for the target object within a preset time period. i Let be the volume adjustment amount for the i-th time.

[0190] Based on G adjust The initial gain can be adjusted; for example, G base_new =G base0 +λ*G adjust Among them, G base_new For the first gain, G base0 Let λ be the initial gain, and λ be a smoothing factor used to control the adjustment speed of the initial gain and avoid drastic fluctuations. For example, λ < 1.

[0191] To avoid unreasonable gain adjustments, the first gain can be limited by combining the minimum and maximum values ​​of the reference gain.

[0192] For example, G base =min(max(G) base_new G min ),G max That is, it can be obtained from G. base_new and G min Determine the maximum value from G, and then use that maximum value and G maxThe minimum value is determined and used as the reference gain for subsequent volume adjustment.

[0193] For example, when G base_new , <G min At that time, max(G) base_new G min ) = G min This can avoid G base_new Too small. When G base_new ,>G min At that time, max(G) base_new G min ) = G base_new When G base_new >G max At that time, G base =G max This can avoid G base_new Too large.

[0194] In this embodiment, a smoothing factor is introduced based on the initial gain to control the speed of gain adjustment. At the same time, by combining the minimum and maximum values ​​of the reference gain, the reference gain can be kept at a reasonable gain level, avoiding the effect of volume adjustment due to being too small or too large.

[0195] In some embodiments, to avoid abrupt volume changes and provide a more comfortable user experience, volume changes can be smoothed out. For example, the above-described "adjusting the second volume to the third volume using a target gain adjustment" may include the following steps:

[0196] The target adjustment gain is smoothed by linear interpolation to obtain a smoothed gain.

[0197] Use smooth gain to adjust the second volume to the third volume.

[0198] For example, at each time point t, the current gain G can be determined. current (t) and target adjustment gain G video The difference between (t) indicates the volume level that needs adjustment. Based on this difference, the target adjustment gain can be smoothed using linear interpolation.

[0199] G smooth (t)=G current (t-1)+α*(G video (t)-G current (t-1))

[0200] Among them, G smooth (t) is the smoothing gain, G current(0) = 1, meaning the initial value of the current gain is 1, which keeps the current volume constant. α is a constant between 0 and 1, used to control the rate of volume change.

[0201] For example, S smooth (t)=G smooth (t)*S original (t).

[0202] By smoothing the target gain, the discomfort caused by sudden changes in volume can be avoided, thus improving the user's listening experience.

[0203] Taking video as an example, in some embodiments, when the first voiceprint feature of the first audio signal does not match the reference voiceprint feature, it indicates that the first audio signal is the audio signal of a new object, and the display is then executed. Figure 8 The interface shown displays the volume adjustment options for the new character. Users can enable the volume adjustment function for the new character, allowing the volume to be adjusted adaptively based on the viewing environment and user preferences.

[0204] Of course, during audio and video playback, electronic devices can also display a settings button. Through this settings button, users can maintain and modify the reference voiceprint features and related volume settings in the cache.

[0205] This application embodiment allows users to adjust the volume individually for each object in real time, achieving a personalized viewing experience. Moreover, after the user adjusts the volume, the subsequent electronic device can adaptively adjust the volume based on the noise level of the viewing environment and the user's volume preference, so as to provide the best auditory experience for the user in different environments without the need for manual operation, simplifying the user's operation and improving the user's viewing experience.

[0206] It should be noted that the volume adjustment method provided in this application embodiment can be executed by a volume adjustment device or a processing module within the volume adjustment device for executing the volume adjustment method. This application embodiment uses a volume adjustment device executing the volume adjustment method as an example to illustrate the volume adjustment device provided in this application embodiment.

[0207] Figure 9 This is a schematic diagram of a volume adjustment device provided in an embodiment of this application.

[0208] like Figure 9 As shown, the volume adjustment device 900 may include: an acquisition module 901, a determination module 902, and an adjustment module 903;

[0209] The acquisition module 901 is used to acquire, when the volume adjustment is detected for the object, the first audio signal of the object corresponding to the audio and video at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the target object at each second time point within the second sliding time window. The first sliding time window and the second sliding time window are both time windows before the first moment.

[0210] The determination module 902 is used to determine the environmental noise characteristics of the viewing environment of the audio and video at a first moment based on each first environmental noise data when the first voiceprint feature of the first audio signal matches the reference voiceprint feature. The reference voiceprint feature is the voiceprint feature of the object.

[0211] The determination module 902 is also used to determine the user's volume preference coefficient at the first moment based on the first volume of the object at each second time point;

[0212] The adjustment module 903 is used to adjust the second volume of the first audio signal to the third volume at the first moment according to the characteristics of the ambient noise and the volume preference coefficient.

[0213] In this embodiment, when volume adjustment is detected for an object and the first voiceprint feature of the first audio signal matches the reference voiceprint feature of the object, the environmental noise features at a first moment can be determined based on each first environmental noise data. Furthermore, based on the first volume of the object at each second time point, the user's volume preference coefficient at the first moment can be determined. Then, based on the environmental noise features and the volume preference coefficient, the second volume of the first audio signal at the first moment can be automatically adjusted to a third volume. In other words, this embodiment can adaptively adjust the volume of the object under different viewing environments based on environmental noise and user preferences, providing the user with the best auditory experience in various viewing environments. This eliminates the need for frequent manual volume adjustments due to different viewing environments and uneven volume among characters, thereby reducing user operations and improving the smoothness of the viewing experience.

[0214] In some possible implementations of the embodiments of this application, the acquisition module 901 is further configured to acquire the second environmental noise data of each third time point in the initial sliding time window before acquiring the first audio signal of the audio and video corresponding to the object at the first moment, the first environmental noise data of each first time point in the first sliding time window, and the first volume of the object at each second time point in the second sliding time window. The initial sliding time window is the time window before the first moment.

[0215] The determination module 902 is also used to determine the mean and standard deviation of each second environmental noise data within the initial sliding time window;

[0216] The window adjustment factor is determined based on the ratio of the standard deviation to the mean.

[0217] The adjustment module 903 is also used to adjust the size of the initial sliding time window according to the window adjustment factor to obtain the first sliding time window.

[0218] In some possible implementations of the embodiments of this application, the determining module 902 is specifically used for:

[0219] Based on the first volume at each second time point, determine the weighted average volume of the object at the first time point;

[0220] Determine the difference between the weighted average volume and the reference volume;

[0221] The product of the difference and the user preference compensation coefficient is determined as the user's volume preference coefficient at the first moment.

[0222] In some possible implementations of the embodiments of this application, the environmental noise characteristics include noise spectrum characteristics;

[0223] Module 902 is specifically used for:

[0224] Decompose each first environmental noise data according to frequency to obtain at least one third environmental noise data corresponding to a frequency.

[0225] For each frequency, perform a Fourier transform on each third ambient noise data to obtain the Fourier transform result, and determine the first power of the third ambient noise data at that frequency based on the Fourier transform result.

[0226] Based on the first power, the noise spectrum entropy of the viewing environment at the first moment is determined, and the noise spectrum entropy is defined as the noise spectrum characteristic of the viewing environment at the first moment.

[0227] In some possible implementations of the embodiments of this application, the environmental noise feature further includes a first noise intensity;

[0228] Module 902 is specifically used for:

[0229] The average noise power of the viewing environment within the first sliding time window is obtained by integrating each first environmental noise data.

[0230] Logarithmic processing of the average noise power yields the first noise intensity of the viewing environment at the first moment.

[0231] In some possible implementations of the embodiments of this application, the environmental noise characteristics include noise spectrum characteristics and a first noise intensity;

[0232] The volume control device 900 may also include:

[0233] A generation module is used to generate a first weighting coefficient based on the noise spectrum characteristics and a first noise intensity;

[0234] The compensation module is used to compensate for the first noise intensity using a first weighting coefficient to obtain the second noise intensity;

[0235] The determination module 902 is also used to determine the noise compensation gain based on the difference between the second noise intensity and the reference noise intensity;

[0236] Adjustment module 903 is specifically used for:

[0237] Adjust the second volume to the third volume based on the noise compensation gain and volume preference coefficient.

[0238] In some possible implementations of the embodiments of this application, the generation module is specifically used for:

[0239] Spectral weighting coefficients are generated based on the noise spectrum characteristics, and intensity weighting coefficients are generated based on the first noise intensity.

[0240] The product of the spectral weighting coefficient and the intensity weighting coefficient is determined as the first weighting coefficient.

[0241] In some possible implementations of the embodiments of this application, the determining module 902 is further configured to determine a first difference between the noise compensation gain and the reference gain, and a second difference between the volume preference coefficient and the reference gain;

[0242] Determine the first weighted result of the first difference and the first weight, and the second weighted result of the second difference and the second weight. The first weight is the influence weight of the viewing environment, and the second weight is the influence weight of user preferences.

[0243] The target adjustment gain is determined by summing the first weighted result, the second weighted result, and the reference gain.

[0244] Adjustment module 903 is specifically used for:

[0245] Adjust the second volume to the third volume using the target adjustment gain.

[0246] In some possible implementations of the embodiments of this application, the determining module 902 is further configured to determine the average volume adjustment of the object within a preset time period based on the object's historical volume adjustment data during the audio and video playback process, before determining the first difference between the noise compensation gain and the reference gain, and the second difference between the volume preference coefficient and the reference gain. The historical volume adjustment data includes the number of volume adjustments and the volume adjustment amount of each adjustment, and the preset time period is the time period before the first moment.

[0247] The initial gain is updated based on the product of the average volume adjustment and the smoothing factor to obtain the first gain;

[0248] The maximum value is determined from the lower limits of the first gain and the reference gain;

[0249] The minimum of the maximum value and the upper limit of the reference gain is determined as the reference gain.

[0250] In some possible implementations of the embodiments of this application, the adjustment module 903 is specifically used for:

[0251] The target adjustment gain is smoothed by linear interpolation to obtain a smoothed gain.

[0252] Use smooth gain to adjust the second volume to the third volume.

[0253] This application embodiment allows users to adjust the volume individually for each object in real time, achieving a personalized viewing experience. Moreover, after the user adjusts the volume, the subsequent electronic device can adaptively adjust the volume based on the noise level of the viewing environment and the user's volume preference, so as to provide the best auditory experience for the user in different environments without the need for manual operation, simplifying the user's operation and improving the user's viewing experience.

[0254] The volume adjustment device in this application embodiment can be a device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. The embodiments of this application do not specifically limit the scope.

[0255] The electronic device in this application embodiment can be an electronic device with an operating system. The operating system can be Android, iOS, or other possible operating systems, and this application embodiment does not specifically limit it.

[0256] The volume adjustment device provided in this application embodiment can achieve... Figures 1 to 4 The various processes in the volume adjustment method embodiment can achieve the same technical effect, and will not be described again here to avoid repetition.

[0257] like Figure 10 As shown, this application embodiment also provides an electronic device 1000, including a processor 1001 and a memory 1002. The memory 1002 stores programs or instructions that can run on the processor 1001. When the program or instructions are executed by the processor 1001, they implement the various steps of the volume adjustment method embodiment described above and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0258] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0259] Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0260] The electronic device 1100 includes, but is not limited to, components such as: radio frequency unit 1101, network module 1102, audio output unit 1103, input unit 1104, sensor 1105, display unit 1106, user input unit 1107, interface unit 1108, memory 1109, and processor 1110.

[0261] Those skilled in the art will understand that the electronic device 1100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0262] The processor 1110 is used to acquire, when it detects that the volume is adjusted for the object, the first audio signal of the audio and video at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the object at each second time point within the second sliding time window. The first sliding time window and the second sliding time window are both time windows before the first moment.

[0263] When the first voiceprint feature of the first audio signal matches the reference voiceprint feature, the environmental noise feature of the viewing environment of the audio and video at the first moment is determined based on each first environmental noise data, and the reference voiceprint feature is the voiceprint feature of the object.

[0264] Based on the first volume of the object at each second time point, determine the user's volume preference coefficient at the first moment;

[0265] Based on the environmental noise characteristics and volume preference coefficient, the second volume of the first audio signal at the first moment is adjusted to the third volume.

[0266] In this embodiment, when volume adjustment is detected for an object and the first voiceprint feature of the first audio signal matches the reference voiceprint feature of the object, the environmental noise features at a first moment can be determined based on each first environmental noise data. Furthermore, based on the first volume of the object at each second time point, the user's volume preference coefficient at the first moment can be determined. Then, based on the environmental noise features and the volume preference coefficient, the second volume of the first audio signal at the first moment can be automatically adjusted to a third volume. In other words, this embodiment can adaptively adjust the volume of the object under different viewing environments based on environmental noise and user preferences, providing the user with the best auditory experience in various viewing environments. This eliminates the need for frequent manual volume adjustments due to different viewing environments and uneven volume among characters, thereby reducing user operations and improving the smoothness of the viewing experience.

[0267] In some possible implementations of the embodiments of this application, before the processor 1110 acquires the first audio signal of the audio and video corresponding to the object at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the object at each second time point within the second sliding time window, it is also used to acquire the second environmental noise data at each third time point within the initial sliding time window, wherein the initial sliding time window is the time window before the first moment.

[0268] Determine the mean and standard deviation of each second environmental noise data point within the initial sliding time window;

[0269] The window adjustment factor is determined based on the ratio of the standard deviation to the mean.

[0270] The size of the initial sliding time window is adjusted according to the window adjustment factor to obtain the first sliding time window.

[0271] In some possible implementations of embodiments of this application, the processor 1110 is specifically used for:

[0272] Based on the first volume at each second time point, determine the weighted average volume of the object at the first time point;

[0273] Determine the difference between the weighted average volume and the reference volume;

[0274] The product of the difference and the user preference compensation coefficient is determined as the user's volume preference coefficient at the first moment.

[0275] In some possible implementations of the embodiments of this application, the environmental noise characteristics include noise spectrum characteristics;

[0276] Processor 1110, specifically used for:

[0277] Decompose each first environmental noise data according to frequency to obtain at least one third environmental noise data corresponding to a frequency.

[0278] For each frequency, perform a Fourier transform on each third ambient noise data to obtain the Fourier transform result, and determine the first power of the third ambient noise data at that frequency based on the Fourier transform result.

[0279] Based on the first power, the noise spectrum entropy of the viewing environment at the first moment is determined, and the noise spectrum entropy is defined as the noise spectrum characteristic of the viewing environment at the first moment.

[0280] In some possible implementations of the embodiments of this application, the environmental noise feature further includes a first noise intensity;

[0281] Processor 1110, specifically used for:

[0282] The average noise power of the viewing environment within the first sliding time window is obtained by integrating each first environmental noise data.

[0283] Logarithmic processing of the average noise power yields the first noise intensity of the viewing environment at the first moment.

[0284] In some possible implementations of the embodiments of this application, the environmental noise characteristics include noise spectrum characteristics and a first noise intensity;

[0285] Processor 1110, specifically used for:

[0286] Based on the noise spectrum characteristics and the first noise intensity, a first weighting coefficient is generated;

[0287] The first noise intensity is compensated using the first weighting coefficient to obtain the second noise intensity;

[0288] The noise compensation gain is determined based on the difference between the second noise intensity and the reference noise intensity.

[0289] Adjust the second volume to the third volume based on the noise compensation gain and volume preference coefficient.

[0290] In some possible implementations of embodiments of this application, the processor 1110 is specifically used for:

[0291] Spectral weighting coefficients are generated based on the noise spectrum characteristics, and intensity weighting coefficients are generated based on the first noise intensity.

[0292] The product of the spectral weighting coefficient and the intensity weighting coefficient is determined as the first weighting coefficient.

[0293] In some possible implementations of embodiments of this application, the processor 1110 is specifically used for:

[0294] Determine the first difference between the noise compensation gain and the reference gain, and the second difference between the volume preference coefficient and the reference gain;

[0295] Determine the first weighted result of the first difference and the first weight, and the second weighted result of the second difference and the second weight. The first weight is the influence weight of the viewing environment, and the second weight is the influence weight of user preferences.

[0296] The target adjustment gain is determined by summing the first weighted result, the second weighted result, and the reference gain.

[0297] Adjust the second volume to the third volume using the target adjustment gain.

[0298] In some possible implementations of the embodiments of this application, the processor 1110 is further configured to determine the average volume adjustment of the object within a preset time period based on the object's historical volume adjustment data during the audio and video playback process, before determining the first difference between the noise compensation gain and the reference gain, and the second difference between the volume preference coefficient and the reference gain. The historical volume adjustment data includes the number of volume adjustments and the volume adjustment amount of each adjustment, and the preset time period is the time period before the first moment.

[0299] The initial gain is updated based on the product of the average volume adjustment and the smoothing factor to obtain the first gain;

[0300] The maximum value is determined from the lower limits of the first gain and the reference gain;

[0301] The minimum of the maximum value and the upper limit of the reference gain is determined as the reference gain.

[0302] In some possible implementations of embodiments of this application, the processor 1110 is specifically used for:

[0303] The target adjustment gain is smoothed by linear interpolation to obtain a smoothed gain.

[0304] Use smooth gain to adjust the second volume to the third volume.

[0305] This application embodiment allows users to adjust the volume individually for each object in real time, achieving a personalized viewing experience. Moreover, after the user adjusts the volume, the subsequent electronic device can adaptively adjust the volume based on the noise level of the viewing environment and the user's volume preference, so as to provide the best auditory experience for the user in different environments without the need for manual operation, simplifying the user's operation and improving the user's viewing experience.

[0306] It should be understood that, in this embodiment, the input unit 1104 may include a graphics processing unit (GPU) 11041 and a microphone 11042. The GPU 11041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1106 may include a display panel 11061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0307] The memory 1109 can be used to store software programs and various data. The memory 1109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1109 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0308] Processor 1110 may include one or more processing units; optionally, processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1110.

[0309] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the volume adjustment method embodiments described above and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0310] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0311] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the volume adjustment method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0312] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0313] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the volume adjustment method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0314] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0315] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0316] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A volume adjustment method, characterized in that, include: When volume adjustment is detected for an object, the system acquires the first audio signal of the audio / video corresponding to the object at a first moment, the first environmental noise data at each first time point within a first sliding time window, and the first volume of the object at each second time point within a second sliding time window. Both the first and second sliding time windows are time windows prior to the first moment. The object is the object that emits the audio signal in the audio / video. When the first voiceprint feature of the first audio signal matches the reference voiceprint feature, the environmental noise feature of the viewing environment of the audio and video at the first moment is determined based on each of the first environmental noise data, and the reference voiceprint feature is the voiceprint feature of the object. Based on the first volume of the object at each of the second time points, determine the user's volume preference coefficient at the first time point; Based on the environmental noise characteristics and the volume preference coefficient, the second volume of the first audio signal at the first moment is adjusted to the third volume; The step of determining the user's volume preference coefficient at the first moment based on the first volume of the object at each of the second time points includes: Based on the first volume at each of the second time points, determine the weighted average volume of the object at the first time point; Determine the difference between the weighted average volume and the reference volume; The product of the difference and the user preference compensation coefficient is determined as the user's volume preference coefficient at the first moment; the user preference compensation coefficient is used to adjust the difference to reflect the user's sensitivity or preference for volume changes.

2. The method according to claim 1, characterized in that, Before acquiring the first audio signal of the audio / video corresponding to the object at the first moment, the first environmental noise data at each first time point within the first sliding time window, and the first volume of the object at each second time point within the second sliding time window, the method further includes: Acquire the second environmental noise data at each third time point within the initial sliding time window, wherein the initial sliding time window is the time window prior to the first time point; Determine the mean and standard deviation of each of the second environmental noise data within the initial sliding time window; The window adjustment factor is determined based on the ratio of the standard deviation to the mean. The size of the initial sliding time window is adjusted according to the window adjustment factor to obtain the first sliding time window.

3. The method according to claim 1 or 2, characterized in that, The environmental noise characteristics include noise spectrum characteristics; The step of determining the environmental noise characteristics of the viewing environment of the audio and video at the first moment based on each of the first environmental noise data includes: Decompose each of the first environmental noise data according to the frequency to obtain at least one third environmental noise data corresponding to the frequency. For each frequency, a Fourier transform is performed on each of the third environmental noise data to obtain the Fourier transform result, and the first power of the third environmental noise data at the frequency is determined based on the Fourier transform result. Based on the first power, the noise spectrum entropy of the viewing environment at the first moment is determined, and the noise spectrum entropy is determined as the noise spectrum feature of the viewing environment at the first moment.

4. The method according to claim 3, characterized in that, The environmental noise characteristics also include a first noise intensity; The step of determining the environmental noise characteristics of the viewing environment of the audio and video at the first moment based on each of the first environmental noise data includes: The first environmental noise data are integrated to obtain the average noise power of the viewing environment within the first sliding time window. Logarithmic processing is performed on the average noise power to obtain the first noise intensity of the viewing environment at the first moment.

5. The method according to claim 1 or 2, characterized in that, The environmental noise characteristics include noise spectrum characteristics and a first noise intensity; Adjusting the second volume of the first audio signal at the first moment to the third volume based on the environmental noise characteristics and the volume preference coefficient includes: Based on the noise spectrum characteristics and the first noise intensity, a first weighting coefficient is generated; The first noise intensity is compensated using the first weighting coefficient to obtain the second noise intensity; The noise compensation gain is determined based on the difference between the second noise intensity and the reference noise intensity. The second volume is adjusted to the third volume based on the noise compensation gain and the volume preference coefficient.

6. The method according to claim 5, characterized in that, The step of generating the first weighting coefficient based on the noise spectrum characteristics and the first noise intensity includes: A spectral weighting coefficient is generated based on the noise spectral characteristics, and an intensity weighting coefficient is generated based on the first noise intensity; The product of the spectral weighting coefficient and the intensity weighting coefficient is determined as the first weighting coefficient.

7. The method according to claim 5, characterized in that, Adjusting the second volume to the third volume based on the noise compensation gain and the volume preference coefficient includes: Determine a first difference between the noise compensation gain and the reference gain, and a second difference between the volume preference coefficient and the reference gain; Determine the first weighted result of the first difference and the first weight, and the second weighted result of the second difference and the second weight, wherein the first weight is the influence weight of the viewing environment, and the second weight is the influence weight of user preference; The target adjustment gain is determined by summing the first weighted result, the second weighted result, and the reference gain. The second volume is adjusted to the third volume using the target adjustment gain.

8. The method according to claim 7, characterized in that, Before determining the first difference between the noise compensation gain and the reference gain, and the second difference between the volume preference coefficient and the reference gain, the method further includes: During the audio and video playback, the average volume adjustment of the object within a preset time period is determined based on the object's historical volume adjustment data. The historical volume adjustment data includes the number of volume adjustments and the volume adjustment amount for each adjustment. The preset time period is the time period before the first moment. The initial gain is updated based on the product of the average volume adjustment and the smoothing factor to obtain the first gain; The maximum value is determined from the lower limit of the first gain and the reference gain; The minimum value between the maximum value and the upper limit value of the reference gain is determined as the reference gain.

9. The method according to claim 7, characterized in that, Adjusting the second volume to the third volume using the target adjustment gain includes: The target adjustment gain is smoothed by linear interpolation to obtain a smoothed gain. The second volume is adjusted to the third volume using the smoothing gain.

10. A volume control device, characterized in that, include: Acquisition module, determination module, and adjustment module; The acquisition module is used to acquire, when volume adjustment is detected for an object, the first audio signal of the audio-visual content corresponding to the object at a first moment, the first environmental noise data at each first time point within a first sliding time window, and the first volume of the object at each second time point within a second sliding time window, wherein the first sliding time window and the second sliding time window are both time windows prior to the first moment; the object is the object that emits the audio signal in the audio-visual content. The determining module is used to determine the environmental noise characteristics of the viewing environment of the audio and video at the first moment based on each of the first environmental noise data when the first voiceprint feature of the first audio signal matches the reference voiceprint feature. The reference voiceprint feature is the voiceprint feature of the object. The determining module is further configured to determine the user's volume preference coefficient at the first moment based on the first volume of the object at each of the second time points; The adjustment module is used to adjust the second volume of the first audio signal at the first moment to the third volume according to the environmental noise characteristics and the volume preference coefficient; The determining module is specifically used for: Based on the first volume at each of the second time points, determine the weighted average volume of the object at the first time point; Determine the difference between the weighted average volume and the reference volume; The product of the difference and the user preference compensation coefficient is determined as the user's volume preference coefficient at the first moment; The user preference compensation coefficient is used to adjust the difference, reflecting the user's sensitivity or preference for volume changes.

11. The apparatus according to claim 10, characterized in that, The environmental noise characteristics include a first noise intensity; The determining module is specifically used for: The first environmental noise data are integrated to obtain the average noise power of the viewing environment within the first sliding time window. Logarithmic processing is performed on the average noise power to obtain the first noise intensity of the viewing environment at the first moment.

12. The apparatus according to claim 10, characterized in that, The environmental noise characteristics include noise spectrum characteristics and a first noise intensity; The device further includes: A generation module is configured to generate a first weighting coefficient based on the spectral characteristics and the first noise intensity; The compensation module is used to compensate the first noise intensity using the first weighting coefficient to obtain the second noise intensity; The determining module is further configured to determine the noise compensation gain based on the difference between the second noise intensity and the reference noise intensity; The adjustment module is specifically used for: The second volume is adjusted to the third volume based on the noise compensation gain and the volume preference coefficient.

13. The apparatus according to claim 12, characterized in that, The generation module is specifically used for: Spectral weighting coefficients are generated based on the spectral characteristics, and intensity weighting coefficients are generated based on the first noise intensity. The product of the spectral weighting coefficient and the intensity weighting coefficient is determined as the first weighting coefficient.

14. An electronic device, characterized in that, include: A processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as claimed in any one of claims 1 to 9.

Images