Multi-source audio quality unified optimization method and system

Abstract

The invention relates to the technical field of audio processing, in particular to a multi-source audio quality unified optimization method and system. According to the multi-source audio quality unified optimization method provided by the invention, unified optimization can be carried out on multi-source audios with relatively large parameter differences; the method comprises the following steps: firstly marking a plurality of audio segments needing to be optimized as audio segments to be optimized, then sequentially carrying out format optimization, sampling rate optimization and loudness optimization on the audio segments to be optimized, then splicing the optimized audio segments to obtain an audio segment to be played, and subsequently sending the audio segment to be played to a user for listening. And the hearing feeling of the optimized to-be-played audio is better, and the unified feeling of the audio is better, so that the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, specifically to a method and system for unified optimization of multi-source audio quality. Background Technology

[0002] Currently, audio content is produced and distributed by different entities; and the parameters of audio content produced by different entities vary significantly (e.g., differences in audio format, sampling rate, volume, etc.). This results in a poor listening experience for listeners when multi-source audio content from different entities is spliced ​​and distributed on the same distribution platform. Therefore, there is an urgent need for a technical solution that can uniformly optimize multi-source audio with significantly different parameters. Summary of the Invention

[0003] The main objective of this invention is to provide a method and system for unified optimization of multi-source audio quality, aiming to solve the current urgent need for a technical solution that can uniformly optimize multi-source audio with significant differences in parameters.

[0004] The technical solution proposed in this invention is as follows: A method for unified optimization of multi-source audio quality, applied to a multi-source audio quality unified optimization system; the system includes an audio optimization terminal; the method includes: The audio optimization terminal acquires multiple audio segments that need to be spliced ​​and mixed, and marks them as audio segments to be optimized; The audio optimization terminal determines whether format optimization needs to be performed; If format optimization is required, the audio optimization terminal determines the target format, marks the audio segments to be optimized that do not match the target format as audio segments to be converted, and converts the audio format of the audio segments to be converted to the target format; After format optimization is completed, or when format optimization is not required, the audio optimization terminal determines whether sampling rate optimization needs to be performed. If sampling rate optimization is required, the audio optimization terminal determines the target sampling rate based on each audio segment to be optimized, and performs resampling processing on the audio segments to be optimized based on the target sampling rate, so that the sampling rate of each audio segment to be optimized is consistent. When sampling rate optimization is completed, or when sampling rate optimization is not required, the audio optimization terminal determines whether loudness optimization needs to be performed. If loudness optimization is required, the audio optimization terminal obtains the average loudness value corresponding to each audio segment to be optimized, and performs loudness optimization on the audio segment to be optimized based on the average loudness value; When loudness optimization is completed, or when loudness optimization is not required, the audio optimization terminal splices the audio segments to be optimized end to end in a preset order to obtain the audio to be played.

[0005] Preferably, if sampling rate optimization is required, the audio optimization terminal determines a target sampling rate based on each audio segment to be optimized, and performs resampling processing on the audio segments to be optimized based on the target sampling rate to ensure that the sampling rates of each audio segment to be optimized are consistent, including: The audio optimization terminal acquires the sampling rate of each audio segment to be optimized and marks it as the original sampling rate; The audio optimization terminal uses the most frequent original sampling rate among all the original sampling rates as the target sampling rate. The audio optimization terminal marks audio segments to be optimized with an original sampling rate higher than the target sampling rate as first audio segments, and audio segments to be optimized with an original sampling rate lower than the target sampling rate as second audio segments; The audio optimization terminal performs downsampling processing on the first audio segment to adjust the sampling rate of the first audio segment to the target sampling rate; The audio optimization terminal performs upsampling processing on the second audio segment to adjust the sampling rate of the second audio segment to the target sampling rate.

[0006] Preferably, if loudness optimization needs to be performed, the audio optimization terminal obtains the average loudness value corresponding to each audio segment to be optimized, and performs loudness optimization on the audio segment to be optimized based on the average loudness value, including: The audio optimization terminal obtains the average loudness value corresponding to each audio segment to be optimized, and obtains the preset standard loudness value; The audio optimization terminal marks the audio segment to be optimized as the third audio segment, which is greater than the standard loudness value and whose difference from the standard loudness value is greater than a first preset value. The audio optimization terminal marks the audio segment to be optimized as the fourth audio segment, which is less than the standard loudness value and whose absolute value of the difference from the standard loudness value is greater than the first preset value. The audio optimization terminal obtains the difference between the standard loudness value and the average loudness value of the third audio segment, and uses it as the first loudness difference to attenuate the loudness of the third audio segment, with the attenuation amount being the first loudness difference. The audio optimization terminal obtains the difference between the standard loudness value and the average loudness value of the fourth audio segment, and uses it as the second loudness difference to perform loudness gain on the fourth audio segment, with the gain amount being the second loudness difference.

[0007] Preferably, the system further includes an audio distribution terminal communicatively connected to the audio optimization terminal, and a user terminal communicatively connected to the audio distribution terminal; when loudness optimization is completed, or when loudness optimization is not required, the audio optimization terminal splices the audio segments to be optimized end-to-end in a preset order to obtain the audio to be played, and then further includes: The audio optimization terminal sends the audio to be played to the audio distribution terminal; The audio distribution terminal sends the audio to be played to the user terminal; The user terminal obtains the historical operation log of the current user's operation on the user terminal, and determines the target volume based on the historical operation log; The user terminal plays the audio to be played, and the volume of the playback is the target volume.

[0008] Preferably, the historical operation information includes the average playback volume corresponding to the user's use of the user terminal in different time periods; the user terminal obtains the historical operation log of the current user's operation of the user terminal, and determines the target volume based on the historical operation log, including: The user terminal marks the time period that the current moment falls into as the target time period; The user terminal obtains the average playback volume corresponding to the user's use of the user terminal during the target time period based on historical operation logs, and uses it as the target volume.

[0009] Preferably, the user terminal plays the audio to be played, and the volume of the playback is the target volume, and then the method further includes: The user terminal determines whether it has received a volume adjustment command from the user within a first preset duration from the start of the audio to be played. If so, the user terminal marks the audio to be played this time as adjusted audio, obtains the volume of the adjusted audio after the user adjusts the volume minus the volume before the volume adjustment, and marks it as the volume adjustment value corresponding to the adjusted audio; The user terminal determines whether the number of adjusted audios in the first preset number of audios to be played in the past is greater than or equal to a second preset number, wherein the second preset number is less than the first preset number; If the quantity is greater than or equal to the second preset quantity, the user terminal determines whether the first condition or the second condition is met. The first condition is that the volume adjustment values ​​corresponding to the adjusted audio in the first preset quantity of audio to be played in the past are all positive values ​​and are all greater than the preset volume value. The second condition is that the volume adjustment values ​​corresponding to the adjusted audio in the first preset quantity of audio to be played in the past are all negative values ​​and are all less than the opposite value of the preset volume value. If the first condition is met, the user terminal increases the target volume by the amount of the increase, which is equal to the preset volume value. If the second condition is met, the user terminal will reduce the target volume by the amount of reduction, which is the preset volume value.

[0010] Preferably, the user terminal further includes a microphone capable of acquiring sound signals; the user terminal plays the audio to be played, and the volume of the playback is the target volume, further including: Before the user terminal starts playing the audio to be played, the user terminal divides the audio to be played into multiple sequentially adjacent sub-audio segments and marks them as the first sub-audio segments, obtains the average loudness value of each first sub-audio segment, and marks it as the first loudness value; The user terminal marks the first sub-audio segment whose first loudness value is less than the second preset value as the target audio segment; After the user terminal starts playing the audio to be played, the user terminal uses the microphone to collect the sound signal of the surrounding environment during each playback of the target audio segment, and obtains the average loudness value of the sound signal of the surrounding environment; When the average loudness value of the sound signals of the surrounding environment within the past second preset time period is less than the third preset value, and the current time is within the preset time period, the user terminal presumes that the user is in a resting state. When the user is in a resting state, the user terminal reduces the playback volume of the currently playing audio once every third preset time interval.

[0011] Preferably, the user terminal further includes a microphone capable of acquiring sound signals; the user terminal plays the audio to be played, and the volume of the playback is the target volume, and then further includes: When the user terminal plays the audio to be played through the headphones, and the current time is within a preset time period, the user terminal generates a monitoring command; The user terminal collects ambient sound signals through the microphone based on the monitoring command and marks them as ambient sound signals; The user terminal divides the ambient sound signal within the past fourth preset time period into multiple sequentially adjacent sub-audio segments and marks them as second sub-audio segments, obtains the average loudness value of each second sub-audio segment, and marks it as a second loudness value; The user terminal marks the second loudness value that meets the third condition within the past fourth preset time period as the first target value, wherein the third condition is: the two second loudness values ​​adjacent to the first target value are both less than the first target value; When the number of the first target values ​​is at least two, and the interval between the collection times of two adjacent first target values ​​is less than a fifth preset time, the user terminal determines the second target value and the third target value corresponding to each first target value, wherein the second target value corresponding to the first target value is a second loudness value that is earlier than the first target value and equal to a fourth preset value, and the third target value corresponding to the first target value is a second loudness value that is later than the first target value and equal to a fourth preset value. The user terminal marks the second loudness value that is between the first target value and the corresponding second target value at the time of collection as the fourth target value corresponding to the first target value, and marks the second loudness value that is between the first target value and the corresponding third target value at the time of collection as the fifth target value corresponding to the first target value; The user terminal calculates the first slope and the second slope corresponding to each first target value: , , In the formula, The first slope corresponding to the j-th first target value; Let be the second slope corresponding to the j-th first objective value; j is a positive integer, and 1≤j≤J, where J is the number of first objective values; Let $\mathbf(j)$ be the $\mathbf(o+1)$ third objective value corresponding to the $\mathbf(j)$ first objective value, where $o$ is a positive integer and $1 \leq \mathbf(j)$. , This represents the number of third objective values ​​corresponding to the j-th first objective value. The duration of the second sub-audio segment; Let be the (q+1)th fifth objective value corresponding to the j-th first objective value, where q is a positive integer and 1 ≤ q ≤ 1. , The number of fifth objective values ​​corresponding to the j-th first objective value; When the standard value between the first slopes corresponding to each first target value and the standard deviation between the second slopes corresponding to each first target value are both less than a preset threshold, the user terminal presumes that the user is in a resting state. When the user is in a resting state, the user terminal reduces the playback volume of the currently playing audio once every third preset time interval.

[0012] This invention also proposes a multi-source audio quality unified optimization system, which applies a multi-source audio quality unified optimization method; the system includes an audio optimization terminal.

[0013] The above technical solution can achieve the following beneficial effects: The multi-source audio quality unification optimization method proposed in this invention can uniformly optimize multi-source audio with significant differences in parameters. First, multiple audio segments that need optimization are marked as audio segments to be optimized. Then, the audio segments to be optimized are optimized in sequence in terms of format, sampling rate, and loudness. Then, the audio segments that have been optimized are spliced ​​together to obtain the audio to be played. The audio to be played can then be sent to the user for listening. The optimized audio to be played will have a better listening experience and a better sense of audio uniformity, thereby improving the user experience. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating the first embodiment of a multi-source audio quality unified optimization method proposed in this invention. Detailed Implementation

[0016] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0017] This invention proposes a method and system for unified optimization of multi-source audio quality.

[0018] As attached Figure 1 As shown, in the first embodiment of the multi-source audio quality unified optimization method proposed in this invention, this embodiment is applied to a multi-source audio quality unified optimization system; the system includes an audio optimization terminal (e.g., a computer terminal); this embodiment includes the following steps: Step S110: The audio optimization terminal acquires multiple audio segments that need to be spliced ​​and mixed, and marks them as audio segments to be optimized.

[0019] Specifically, the audio segments to be optimized here are audio segments from multiple sources, and also audio segments that need to be optimized uniformly.

[0020] Step S120: The audio optimization terminal determines whether format optimization needs to be performed.

[0021] Specifically, there are currently many audio formats (such as MP3, WMA, WAV, etc.). When the formats of multiple audio files to be optimized are not unified, format optimization is required.

[0022] Step S130: If format optimization is required, the audio optimization terminal determines the target format (e.g., WAV), marks the audio segments to be optimized that are inconsistent with the target format as audio segments to be converted, and converts the audio format of the audio segments to be converted to the target format.

[0023] Step S140: When format optimization is completed, or when format optimization is not required, the audio optimization terminal determines whether sampling rate optimization needs to be performed.

[0024] Specifically, the mainstream audio sampling rates currently include 11025Hz, 22050Hz, 24000Hz, 44100Hz, and 48000Hz. When the sampling rates of the audio to be optimized are not the same, sampling rate optimization needs to be performed.

[0025] Step S150: If sampling rate optimization is required, the audio optimization terminal determines a target sampling rate (e.g., 24000Hz) based on each audio segment to be optimized, and performs resampling processing on the audio segments to be optimized based on the target sampling rate so that the sampling rate of each audio segment to be optimized is consistent.

[0026] Specifically, when sampling rate optimization is required, the audio segments to be optimized are resampled (i.e., the sampling rate of the audio segments to be optimized is adjusted) to make the sampling rate of each audio segment to be optimized consistent.

[0027] Step S160: When sampling rate optimization is completed, or when sampling rate optimization is not required, the audio optimization terminal determines whether loudness optimization needs to be performed.

[0028] Specifically, loudness is the intensity of sound; it is a subjective measure of how loud a sound is perceived. The magnitude of loudness depends on the amplitude of the sound signal's vibration. When the loudness difference between audio samples to be optimized is significant, loudness optimization is necessary.

[0029] Step S170: If loudness optimization is required, the audio optimization terminal obtains the average loudness value corresponding to each audio segment to be optimized, and performs loudness optimization on the audio segment to be optimized based on the average loudness value so that the loudness of each audio segment to be optimized tends to be consistent.

[0030] Step S180: When loudness optimization is completed, or when loudness optimization is not required, the audio optimization terminal splices the audio segments to be optimized end to end in a preset order to obtain the audio to be played.

[0031] Specifically, the preset order here is the order determined by the audio program director to obtain the complete audio to be played; the audio to be played here is the audio after unified optimization and splicing, so that users will have a better listening experience.

[0032] The multi-source audio quality unification optimization method proposed in this invention can uniformly optimize multi-source audio with significant differences in parameters. First, multiple audio segments that need optimization are marked as audio segments to be optimized. Then, the audio segments to be optimized are optimized in sequence in terms of format, sampling rate, and loudness. Then, the audio segments that have been optimized are spliced ​​together to obtain the audio to be played. The audio to be played can then be sent to the user for listening. The optimized audio to be played will have a better listening experience and a better sense of audio uniformity, thereby improving the user experience.

[0033] In the second embodiment of the multi-source audio quality unified optimization method proposed in this invention, based on the first embodiment, step S140 includes the following steps: Step S210: The audio optimization terminal obtains the sampling rate of each audio segment to be optimized and marks it as the original sampling rate.

[0034] Step S220: The audio optimization terminal uses the original sampling rate that appears most frequently among all the original sampling rates as the target sampling rate.

[0035] Specifically, for example, if 24000Hz appears most frequently among the original sampling rates of multiple audio segments to be optimized, then 24000Hz will be used as the target sampling rate.

[0036] Step S230: The audio optimization terminal marks the audio segments to be optimized with an original sampling rate higher than the target sampling rate as the first audio segment, and marks the audio segments to be optimized with an original sampling rate lower than the target sampling rate as the second audio segment.

[0037] Specifically, the first audio segment is the one where the sampling rate needs to be reduced, and the second audio segment is the one where the sampling rate needs to be increased.

[0038] Step S240: The audio optimization terminal performs downsampling processing on the first audio segment to adjust the sampling rate of the first audio segment to the target sampling rate, specifically including the following steps: Step S241: The audio optimization terminal sets the audio signal of the first audio segment to be... The original sampling rate of the first audio segment is (Unit: Hz) The original sampling period of the first audio segment (Unit: seconds), that is Corresponding time , (n=0,1,2,...), where n is the index of the original sampling point.

[0039] Step S242: The audio optimization terminal sets the target sampling rate to... And satisfy < The target sampling period is The audio signal after downsampling the first audio segment is set as follows: , (p=0,1,2,...), where p is the index of the sampling point after downsampling.

[0040] Step S243: The audio optimization terminal obtains the downsampling factor D, where, .

[0041] Step S244: The audio optimization terminal filters out audio signals with frequencies higher than a certain value from the first audio segment based on a filter. The audio signal is used to obtain the intermediate audio segment.

[0042] Step S245: The audio optimization terminal retains one signal every D sampling points in the relay audio segment to complete the downsampling process. The specific processing formula is as follows: , , In the formula, For relay audio segments, For convolution; For filters.

[0043] Step S250: The audio optimization terminal performs upsampling processing on the second audio segment to adjust the sampling rate of the second audio segment to the target sampling rate, specifically including the following steps: Step S251: The audio optimization terminal sets the original audio signal of the second audio segment as... The original sampling rate of the second audio segment is (Unit: Hz) Original sampling period of the second audio band (Unit: seconds), that is Corresponding time , (n=0,1,2,...), where n is the index of the original sampling point.

[0044] Step S252: The audio optimization terminal sets the target sampling rate to... And satisfy > The target sampling period is The audio signal after upsampling the second audio segment is set as follows: (k=0,1,2,...), where k is the index of the sampling point after upsampling, and the time of the audio signal after upsampling the second audio segment. .

[0045] Step S253: The audio optimization terminal obtains the upsampling factor M, wherein, .

[0046] Step S254: The audio optimization terminal inserts (M-1) zero values ​​between adjacent original sampling points of the second audio segment to obtain an intermediate signal. Where z represents the inserted zero value, and the original sampling point correspond To preserve the original audio information, the position where zero values ​​are inserted is... .

[0047] Step S255: The audio optimization terminal optimizes the intermediate signal based on a filter (LPF). Filtering is performed to obtain the final audio signal. : , In the formula, Let be the unit impulse response of the filter, and satisfy: cutoff frequency , where m is the sequence of sampling points.

[0048] Specifically, this embodiment provides a specific scheme for optimizing the sampling rate of the audio segment to be optimized.

[0049] In the third embodiment of the multi-source audio quality unified optimization method proposed in this invention, based on the first embodiment, step S160 includes the following steps: Step S310: The audio optimization terminal obtains the average loudness value corresponding to each audio segment to be optimized, and obtains the preset standard loudness value.

[0050] Specifically, the formula for calculating the loudness value of a certain sampling point of the audio to be optimized is as follows: , In the formula, Here, abs(data) is the loudness value, and abs(data) is the absolute value of the audio signal at a certain sampling point of the audio to be optimized; the average loudness value is obtained by averaging the loudness values ​​of all sampling points of the audio to be optimized.

[0051] The standard loudness value here is the loudness value that is relatively acceptable to the listening experience of listeners in the industry, such as 30dB.

[0052] Step S320: The audio optimization terminal marks the audio segment to be optimized as the third audio segment if it is greater than the standard loudness value and the difference between the audio segment and the standard loudness value is greater than a first preset value (e.g., 10dB).

[0053] Specifically, the third audio segment here is the audio segment with high loudness that needs to be optimized, and its loudness needs to be reduced.

[0054] Step S330: The audio optimization terminal marks the audio segment to be optimized that is less than the standard loudness value and whose absolute value of the difference from the standard loudness value is greater than the first preset value as the fourth audio segment.

[0055] Specifically, the fourth audio segment here is the audio segment with low loudness that needs to be optimized, and its loudness needs to be increased.

[0056] Step S340: The audio optimization terminal obtains the difference between the standard loudness value and the average loudness value of the third audio segment, and uses it as the first loudness difference (which is negative), to attenuate the loudness of the third audio segment, and the attenuation amount is the first loudness difference. Specifically, it includes the following steps: Step S341: The audio optimization terminal acquires the original amplitude of each sampling point of the third audio segment. 'n' is the sampling point index of the third audio segment, which is the quantization value in digital audio. The amplitude of each sampling point in the third audio segment is attenuated to... : , , In the formula, This is the first loudness difference.

[0057] Step S350: The audio optimization terminal obtains the difference between the standard loudness value and the average loudness value of the fourth audio segment, and uses it as the second loudness difference (which is a positive value), and applies loudness gain to the fourth audio segment, with the gain amount being the second loudness difference, including the following steps: Step S351: The audio optimization terminal acquires the original amplitude of each sampling point of the fourth audio segment. , where n is the sampling point index of the fourth audio segment, and in digital audio, it is the quantization value. The amplitude gain of each sampling point in the fourth audio segment is... : , , In the formula, This is the second loudness difference.

[0058] Specifically, this embodiment provides a specific scheme for loudness optimization of the audio segment to be optimized.

[0059] In the fourth embodiment of the multi-source audio quality unified optimization method proposed in this invention, based on the first embodiment, the system further includes an audio distribution terminal (e.g., a computer terminal with network communication capabilities) communicatively connected to the audio optimization terminal, and a user terminal (e.g., a smartphone terminal) communicatively connected to the audio distribution terminal; step S180, followed by the following steps: Step S410: The audio optimization terminal sends the audio to be played to the audio distribution terminal.

[0060] Step S420: The audio distribution terminal sends the audio to be played to the user terminal.

[0061] Step S430: The user terminal obtains the historical operation log of the current user operation on the user terminal, and determines the target volume based on the historical operation log.

[0062] Specifically, the historical operation logs of user terminals can reflect the user's volume requirements for audio playback, so the target volume can be determined based on the historical operation logs.

[0063] Specifically, the volume unit here is the gain percentage when the user terminal plays audio. For example, when the volume is 100%, the user terminal plays audio at its maximum volume, and when the volume is 0%, the user terminal is muted.

[0064] Step S440: The user terminal plays the audio to be played, and the volume of the playback is the target volume.

[0065] In the fifth embodiment of the multi-source audio quality unified optimization method proposed in this invention, based on the fourth embodiment, the historical operation information includes the average playback volume corresponding to the user's use of the user terminal in different time periods; for example, in this embodiment, each day is divided into 12 time periods, each time period is 2 hours long, from 0:00-2:00, 2:00-4:00, and so on; since the external environment faced in different time periods is different, different playback volumes will be used, so when the user plays the audio to be played, the appropriate playback volume (i.e., the target volume) can be determined based on the time period in which the user is currently located; the user terminal obtains the historical operation log of the current user operation of the user terminal, and determines the target volume based on the historical operation log, including; Step S510: The user terminal marks the time period that the current time falls into as the target time period.

[0066] Step S520: The user terminal obtains the average playback volume corresponding to the user's use of the user terminal during the target time period based on the historical operation log, and uses it as the target volume.

[0067] Specifically, for example, if the current time when the user is ready to play the audio to be played is 7:00 AM, then the target time period that 7:00 AM falls into is 6:00 AM to 8:00 AM; therefore, the average playback volume corresponding to the terminal between 6:00 AM and 8:00 AM is used as the target volume.

[0068] In the sixth embodiment of the multi-source audio quality unified optimization method proposed in this invention, based on the fourth embodiment, after step S440, the following steps are further included: Step S610: The user terminal determines whether it has received a volume adjustment command from the user within a first preset duration (e.g., 3 seconds) since the audio to be played started.

[0069] Specifically, the operation command here can be triggered by the user pressing the volume control button on the user terminal.

[0070] If so, proceed to step S620: The user terminal marks the audio to be played this time as adjusted audio, obtains the volume of the adjusted audio after the user adjusts the volume minus the volume before the volume adjustment, and marks it as the volume adjustment value corresponding to the adjusted audio.

[0071] Specifically, if so, it means that the user adjusted the volume of the audio at the beginning of playing it through the user terminal, indicating that the volume of the audio is not suitable for the user's listening experience; the volume adjustment value here is the amount by which the user adjusts the volume (e.g., 20%).

[0072] Step S630: The user terminal determines whether the number of adjusted audios in the first preset number (e.g., 10) of the audios to be played in the past is greater than or equal to the second preset number (e.g., 5), wherein the second preset number is less than the first preset number.

[0073] Specifically, if five or more of the ten previously played audio files have been adjusted, it indicates that the user has adjusted the volume of the audio files multiple times. Therefore, it can be inferred that the initial volume of the audio files played on the user terminal (i.e., the aforementioned target volume) is not suitable for the current listening experience; thus, the target volume needs to be adjusted.

[0074] Step S640: If the quantity is greater than or equal to the second preset quantity, the user terminal determines whether the first condition or the second condition is met. The first condition is that the volume adjustment values ​​corresponding to the adjusted audio in the first preset quantity of audio to be played in the past are all positive values ​​(i.e., each adjustment increases the volume) and are all greater than the preset volume value (e.g., 30%). The second condition is that the volume adjustment values ​​corresponding to the adjusted audio in the first preset quantity of audio to be played in the past are all negative values ​​(i.e., each adjustment decreases the volume) and are all less than the opposite value of the preset volume value (e.g., -30%).

[0075] Step S650: If the first condition is met, the user terminal increases the target volume by the amount of the increase, which is equal to the preset volume value.

[0076] Specifically, if the first condition is met, it means that the user has manually increased the playback volume of the audio to be played multiple times in the past. Therefore, it can be inferred that the user prefers to listen to audio at a louder volume. Thus, the target volume is increased by the aforementioned preset volume value (i.e., 30%).

[0077] Step S660: If the second condition is met, the user terminal reduces the target volume by the amount of reduction, which is the preset volume value.

[0078] Specifically, if the second condition is met, it means that the user has manually lowered the playback volume of the audio to be played multiple times in the past. Therefore, it can be inferred that the user prefers to listen to audio at a lower volume. Thus, the target volume is reduced by the aforementioned preset volume value (i.e., 30%).

[0079] In the seventh embodiment of the multi-source audio quality unified optimization method proposed in this invention, based on the fourth embodiment, the user terminal further includes a microphone capable of acquiring sound signals; step S440 is preceded by the following steps: Step S710: Before the user terminal starts playing the audio to be played, the user terminal divides the audio to be played into multiple sequentially adjacent sub-audio segments and marks them as the first sub-audio segments, obtains the average loudness value of each first sub-audio segment, and marks it as the first loudness value.

[0080] Specifically, each first sub-audio segment here has the same duration, for example, 1 second.

[0081] Step S720: The user terminal marks the first sub-audio segment whose first loudness value is less than the second preset value (e.g., 20dB) as the target audio segment.

[0082] Specifically, the target audio segment here is the first sub-audio segment with lower loudness, such as the audio segment corresponding to the interlude of human voices in the audio to be played.

[0083] Step S730: After the user terminal starts playing the audio to be played, the user terminal uses the microphone to collect the ambient sound signal during each playback of the target audio segment, and obtains the average loudness value of the ambient sound signal.

[0084] Specifically, the ambient sound signals collected by the microphone during each playback of the target audio segment can reflect the overall sound conditions of the user terminal's current environment. Therefore, the average loudness value of the ambient sound signals can reflect the strength of the ambient sound during the playback of the target audio segment. The reason for collecting the ambient sound signals only during each playback of the target audio segment is that the loudness of the target audio segment itself is not high, so the interference with the strength of the external ambient sound signals will be smaller, that is, the average loudness value of the ambient sound signals will be more realistic.

[0085] Step S740: When the average loudness value of the sound signals of the surrounding environment in the past second preset time period (e.g., 10 minutes) is less than the third preset value (30dB), and the current time is within a preset time period (e.g., 11 p.m. to 6 a.m.), the user terminal presumes that the user is in a resting state.

[0086] Specifically, if the average loudness of the surrounding environmental sound signals within the past second preset time period is less than the third preset value, and the current time is within the preset time period, it means that the user is likely listening to the audio to be played while preparing to fall asleep and rest.

[0087] Step S750: When the user is in a resting state, the user terminal reduces the playback volume of the currently playing audio once every third preset time interval (e.g., 3 minutes).

[0088] Specifically, when a user enters a rest state, the playback volume of the currently playing audio can be reduced once every third preset time interval to help the user rest and thus improve the user experience.

[0089] In the eighth embodiment of the multi-source audio quality unified optimization method proposed in this invention, based on the fourth embodiment, the user terminal further includes a microphone capable of acquiring sound signals; step S440, followed by the following steps: Step S810: When the user terminal plays the audio to be played through the headphones and the current time is within a preset time period (e.g., 11 p.m. to 6 a.m.), the user terminal generates a monitoring command.

[0090] Specifically, when the user terminal plays the audio to be played through the headphones and the current time is within the preset time period, it indicates that the user is likely listening to the audio to be played and preparing to rest. Therefore, the system then collects ambient sound signals based on the microphone to determine whether the user has entered a resting state.

[0091] Step S820: The user terminal collects ambient sound signals through the microphone based on the monitoring command and marks them as ambient sound signals.

[0092] Step S830: The user terminal divides the ambient sound signal within the past fourth preset duration (e.g., 1 minute) into multiple sequentially adjacent sub-audio segments and marks them as second sub-audio segments. It then obtains the average loudness value of each second sub-audio segment and marks it as a second loudness value.

[0093] Specifically, each second sub-audio segment here has the same duration, for example, 1 second.

[0094] Step S840: The user terminal marks the second loudness value that meets the third condition within the past fourth preset time period as the first target value, wherein the third condition is: the two second loudness values ​​adjacent to the first target value are both less than the first target value.

[0095] Specifically, if a user falls asleep, the ambient sound signals here may reflect regular and even snoring. Regular snoring can be represented by loudness, specifically: The magnitude of the second loudness value in each second audio segment of the ambient sound signal within the past fourth preset time period is judged to find the first target value among the second loudness values. The two second loudness values ​​before and after the first target value are both less than the first target value, that is, the first target value is the point with the largest loudness, corresponding to the moment when the user's snoring is the loudest.

[0096] Step S850: When the number of the first target values ​​is at least 2, and the interval between the collection times of two adjacent first target values ​​is less than the fifth preset duration (5 seconds), the user terminal determines the second target value and the third target value corresponding to each first target value. The second target value corresponding to the first target value is a second loudness value that is earlier than the first target value and equal to the fourth preset value (here, the fourth preset value is set as the normal minimum loudness of snoring when a human is falling asleep normally, for example, 10dB). The third target value corresponding to the first target value is a second loudness value that is later than the first target value and equal to the fourth preset value.

[0097] Specifically, the fifth preset duration is set to the maximum normal interval between snoring sounds after a person falls asleep (e.g., 5 seconds) under normal circumstances; this setting is based on the premise that the number of the first target values ​​is at least 2, so as to ensure that the sample size of the first target values ​​is sufficient when making subsequent analysis and judgment.

[0098] Specifically, in this step, the second and third target values ​​corresponding to each first target value are determined. It can be seen that the second target value is the second loudness value corresponding to the start time of the user's snoring action, and the second target value is the second loudness value corresponding to the end time of the user's snoring action. The time period consisting of the second target value, the first target value, and the third target value corresponds to a complete snoring action of the user.

[0099] Step S860: The user terminal marks the second loudness value that is between the first target value and the corresponding second target value at the time of collection as the fourth target value corresponding to the first target value, and marks the second loudness value that is between the first target value and the corresponding third target value at the time of collection as the fifth target value corresponding to the first target value.

[0100] Specifically, the fourth target value is the loudness value from the start of snoring to the point of maximum snoring volume, so the fourth target value should increase over time; similarly, the fifth target value is the loudness value from the point of maximum snoring volume to the point of snoring end, so the fifth target value should decrease over time.

[0101] Step S870: The user terminal calculates the first slope and the second slope corresponding to each first target value: , , In the formula, The first slope corresponding to the j-th first target value; The second slope corresponds to the j-th first target value; j is a positive integer, and 1≤j≤J, where J is the number of first target values ​​(e.g., 3, meaning the user snores 3 times in the past fourth preset time period). Let $\mathbf(j)$ be the $\mathbf(o+1)$ third objective value corresponding to the $\mathbf(j)$ first objective value, where $o$ is a positive integer and $1 \leq \mathbf(j)$. , This represents the number of third objective values ​​corresponding to the j-th first objective value. The duration of the second sub-audio segment; Let be the (q+1)th fifth objective value corresponding to the j-th first objective value, where q is a positive integer and 1 ≤ q ≤ 1. , This represents the number of fifth objective values ​​corresponding to the j-th first objective value.

[0102] Specifically, the first slope in this step reflects the rate of change of loudness as the snoring volume increases during a snoring action; the second slope reflects the rate of change of loudness as the snoring volume decreases during a snoring action. If the user has rested and fallen asleep, the user's snoring action should be regular and uniform, so the first slope corresponding to each snoring action should be similar, and the second slope corresponding to each snoring action should also be similar.

[0103] Step S880: When the standard value between the first slopes corresponding to each first target value and the standard deviation between the second slopes corresponding to each first target value are both less than a preset threshold (e.g., 0.5), the user terminal presumes that the user is in a resting state.

[0104] Specifically, the standard value can reflect the degree of dispersion between each parameter. The smaller the standard value, the smaller the degree of dispersion between the corresponding parameters, that is, the closer they are. Therefore, when the standard values ​​between the first slopes corresponding to each first target value and the standard deviation between the second slopes corresponding to each first target value are both less than the preset threshold, it indicates that the loudness changes corresponding to each snoring action of the user are similar, and it can be inferred that the user is in a resting state.

[0105] Step S890: When the user is in a resting state, the user terminal reduces the playback volume of the currently playing audio once every third preset time interval.

[0106] This invention also proposes a multi-source audio quality unified optimization system, which applies a multi-source audio quality unified optimization method; the system includes an audio optimization terminal.

[0107] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0108] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for unified optimization of multi-source audio quality, characterized in that, An application to a multi-source audio quality unified optimization system; the system includes an audio optimization terminal; the method includes: The audio optimization terminal acquires multiple audio segments that need to be spliced ​​and mixed, and marks them as audio segments to be optimized; The audio optimization terminal determines whether format optimization needs to be performed; If format optimization is required, the audio optimization terminal determines the target format, marks the audio segments to be optimized that do not match the target format as audio segments to be converted, and converts the audio format of the audio segments to be converted to the target format; After format optimization is completed, or when format optimization is not required, the audio optimization terminal determines whether sampling rate optimization needs to be performed. If sampling rate optimization is required, the audio optimization terminal determines the target sampling rate based on each audio segment to be optimized, and performs resampling processing on the audio segments to be optimized based on the target sampling rate, so that the sampling rate of each audio segment to be optimized is consistent. When sampling rate optimization is completed, or when sampling rate optimization is not required, the audio optimization terminal determines whether loudness optimization needs to be performed. If loudness optimization is required, the audio optimization terminal obtains the average loudness value corresponding to each audio segment to be optimized, and performs loudness optimization on the audio segment to be optimized based on the average loudness value; When loudness optimization is completed, or when loudness optimization is not required, the audio optimization terminal splices the audio segments to be optimized end to end in a preset order to obtain the audio to be played.

2. The method for unified optimization of multi-source audio quality according to claim 1, characterized in that, If sampling rate optimization is required, the audio optimization terminal determines a target sampling rate based on each audio segment to be optimized, and performs resampling processing on the audio segments to be optimized based on the target sampling rate to ensure that the sampling rates of each audio segment to be optimized are consistent, including: The audio optimization terminal acquires the sampling rate of each audio segment to be optimized and marks it as the original sampling rate; The audio optimization terminal uses the most frequent original sampling rate among all the original sampling rates as the target sampling rate. The audio optimization terminal marks audio segments to be optimized with an original sampling rate higher than the target sampling rate as first audio segments, and audio segments to be optimized with an original sampling rate lower than the target sampling rate as second audio segments; The audio optimization terminal performs downsampling processing on the first audio segment to adjust the sampling rate of the first audio segment to the target sampling rate; The audio optimization terminal performs upsampling processing on the second audio segment to adjust the sampling rate of the second audio segment to the target sampling rate.

3. The method for unified optimization of multi-source audio quality according to claim 1, characterized in that, If loudness optimization is required, the audio optimization terminal obtains the average loudness value corresponding to each audio segment to be optimized, and performs loudness optimization on the audio segments to be optimized based on the average loudness value, including: The audio optimization terminal obtains the average loudness value corresponding to each audio segment to be optimized, and obtains the preset standard loudness value; The audio optimization terminal marks the audio segment to be optimized as the third audio segment, which is greater than the standard loudness value and whose difference from the standard loudness value is greater than a first preset value. The audio optimization terminal marks the audio segment to be optimized as the fourth audio segment, which is less than the standard loudness value and whose absolute value of the difference from the standard loudness value is greater than the first preset value. The audio optimization terminal obtains the difference between the standard loudness value and the average loudness value of the third audio segment, and uses it as the first loudness difference to attenuate the loudness of the third audio segment, with the attenuation amount being the first loudness difference. The audio optimization terminal obtains the difference between the standard loudness value and the average loudness value of the fourth audio segment, and uses it as the second loudness difference to perform loudness gain on the fourth audio segment, with the gain amount being the second loudness difference.

4. The method for unified optimization of multi-source audio quality according to claim 1, characterized in that, The system also includes an audio distribution terminal communicatively connected to the audio optimization terminal, and a user terminal communicatively connected to the audio distribution terminal; when loudness optimization is completed, or when loudness optimization is not required, the audio optimization terminal splices the audio segments to be optimized end-to-end in a preset order to obtain the audio to be played, and then further includes: The audio optimization terminal sends the audio to be played to the audio distribution terminal; The audio distribution terminal sends the audio to be played to the user terminal; The user terminal obtains the historical operation log of the current user's operation on the user terminal, and determines the target volume based on the historical operation log; The user terminal plays the audio to be played, and the volume of the playback is the target volume.

5. The method for unified optimization of multi-source audio quality according to claim 4, characterized in that, The historical operation information includes the average playback volume when the user uses the user terminal at different time periods. The user terminal obtains the historical operation log of the current user's operation on the user terminal, and determines the target volume based on the historical operation log, including: The user terminal marks the time period that the current moment falls into as the target time period; The user terminal obtains the average playback volume corresponding to the user's use of the user terminal during the target time period based on historical operation logs, and uses it as the target volume.

6. The method for unified optimization of multi-source audio quality according to claim 4, characterized in that, The user terminal plays the audio to be played, and the volume of the playback is the target volume, and then the following is also included: The user terminal determines whether it has received a volume adjustment command from the user within a first preset duration from the start of the audio to be played. If so, the user terminal marks the audio to be played this time as adjusted audio, obtains the volume of the adjusted audio after the user adjusts the volume minus the volume before the volume adjustment, and marks it as the volume adjustment value corresponding to the adjusted audio; The user terminal determines whether the number of adjusted audios in the first preset number of audios to be played in the past is greater than or equal to a second preset number, wherein the second preset number is less than the first preset number; If the quantity is greater than or equal to the second preset quantity, the user terminal determines whether the first condition or the second condition is met. The first condition is that the volume adjustment values ​​corresponding to the adjusted audio in the first preset quantity of audio to be played in the past are all positive values ​​and are all greater than the preset volume value. The second condition is that the volume adjustment values ​​corresponding to the adjusted audio in the first preset quantity of audio to be played in the past are all negative values ​​and are all less than the opposite value of the preset volume value. If the first condition is met, the user terminal increases the target volume by the amount of the increase, which is equal to the preset volume value. If the second condition is met, the user terminal will reduce the target volume by the amount of reduction, which is the preset volume value.

7. The method for unified optimization of multi-source audio quality according to claim 4, characterized in that, The user terminal also includes a microphone capable of collecting sound signals; The user terminal plays the audio to be played, and the volume of the playback is the target volume. This is preceded by: Before the user terminal starts playing the audio to be played, the user terminal divides the audio to be played into multiple sequentially adjacent sub-audio segments and marks them as the first sub-audio segments, obtains the average loudness value of each first sub-audio segment, and marks it as the first loudness value; The user terminal marks the first sub-audio segment whose first loudness value is less than the second preset value as the target audio segment; After the user terminal starts playing the audio to be played, the user terminal uses the microphone to collect the sound signal of the surrounding environment during each playback of the target audio segment, and obtains the average loudness value of the sound signal of the surrounding environment; When the average loudness value of the sound signals of the surrounding environment within the past second preset time period is less than the third preset value, and the current time is within the preset time period, the user terminal presumes that the user is in a resting state. When the user is in a resting state, the user terminal reduces the playback volume of the currently playing audio once every third preset time interval.

8. The method for unified optimization of multi-source audio quality according to claim 4, characterized in that, The user terminal also includes a microphone capable of collecting sound signals; The user terminal plays the audio to be played, and the volume of the playback is the target volume, and then the following is also included: When the user terminal plays the audio to be played through the headphones, and the current time is within a preset time period, the user terminal generates a monitoring command; The user terminal collects ambient sound signals through the microphone based on the monitoring command and marks them as ambient sound signals; The user terminal divides the ambient sound signal within the past fourth preset time period into multiple sequentially adjacent sub-audio segments and marks them as second sub-audio segments, obtains the average loudness value of each second sub-audio segment, and marks it as a second loudness value; The user terminal marks the second loudness value that meets the third condition within the past fourth preset time period as the first target value, wherein the third condition is: the two second loudness values ​​adjacent to the first target value are both less than the first target value; When the number of the first target values ​​is at least two, and the interval between the collection times of two adjacent first target values ​​is less than a fifth preset time, the user terminal determines the second target value and the third target value corresponding to each first target value, wherein the second target value corresponding to the first target value is a second loudness value that is earlier than the first target value and equal to a fourth preset value, and the third target value corresponding to the first target value is a second loudness value that is later than the first target value and equal to a fourth preset value. The user terminal marks the second loudness value that is between the first target value and the corresponding second target value at the time of collection as the fourth target value corresponding to the first target value, and marks the second loudness value that is between the first target value and the corresponding third target value at the time of collection as the fifth target value corresponding to the first target value; The user terminal calculates the first slope and the second slope corresponding to each first target value: ， ， In the formula, The first slope corresponding to the j-th first target value; Let be the second slope corresponding to the j-th first objective value; j is a positive integer, and 1≤j≤J, where J is the number of first objective values; Let $\mathbf(j)$ be the $\mathbf(o+1)$ third objective value corresponding to the $\mathbf(j)$ first objective value, where $o$ is a positive integer and $1 \leq \mathbf(j)$. , This represents the number of third objective values ​​corresponding to the j-th first objective value. The duration of the second sub-audio segment; Let be the (q+1)th fifth objective value corresponding to the j-th first objective value, where q is a positive integer and 1 ≤ q ≤ 1. , The number of fifth objective values ​​corresponding to the j-th first objective value; When the standard value between the first slopes corresponding to each first target value and the standard deviation between the second slopes corresponding to each first target value are both less than a preset threshold, the user terminal presumes that the user is in a resting state. When the user is in a resting state, the user terminal reduces the playback volume of the currently playing audio once every third preset time interval.

9. A multi-source audio quality unified optimization system, characterized in that, The system employs the multi-source audio quality unified optimization method as described in any one of claims 1-8; the system includes an audio optimization terminal.

Images