Automatic volume control device and automatic volume control method

Abstract

The present invention relates to an automatic volume control apparatus and an automatic volume control method. An automatic volume control apparatus based on a voice signal can include an object separator that separates an audio input signal into a voice signal and a non-voice signal, a voice signal processing section that processes the voice signal, determines a volume compensation value based on the voice signal and a set volume information, a non-voice signal processing section that processes the non-voice signal, and a volume controller that controls a volume of an audio output signal, which is obtained by adding a voice output signal and a non-voice output signal processed and output by the voice signal processing section and the non-voice signal processing section, based on the volume compensation value.

Description

Technical Field

[0001] This invention relates to volume control, and more specifically, to an automatic volume control based on voice signals. Background Technology

[0002] In recent years, with the emergence of various content platforms, viewers frequently switch between different platforms to select and consume their preferred content. Not only are the platforms providing content diverse, but the devices used to play that content are also varied, such as TVs, mobile phones, and tablets. However, when users play content on different devices, the volume differences between various content platforms, and the volume control differences of the playback devices themselves create an inconvenience, requiring them to adjust the volume each time they watch a selected program.

[0003] One reason users adjust the volume might be that the voices of characters in a program are not clear enough. When dialogue or important sounds are masked, it becomes difficult to grasp the content, causing audiovisual inconvenience. Therefore, adjusting the overall volume to a comfortable level can be achieved. Conversely, a reason for decreasing the volume might be that the overall sound is too loud or that a sudden increase in the volume of a particular sound effect or music causes discomfort. Thus, as mentioned above, there are inconveniences associated with adjusting the volume to a comfortable level, whether the volume is too low, too high, or suddenly increases or decreases.

[0004] Therefore, to eliminate potential discomfort caused by sound during the playback of various content, hardware manufacturers of devices that produce content playback equipment are researching automatic volume control technologies that enhance the speech played on content platforms while suppressing unnecessary noise or sudden volume changes. However, most automatic volume control technologies amplify small signals and attenuate excessively large signals, resulting in the problem of both the voices of characters and background noise in the content becoming louder or quieter simultaneously. This can lead to issues such as: while the overall volume may increase, the recognition performance of important speech, such as dialogue, may not improve; and the attenuation of large signals can cause the dynamics of background noise to disappear, leading to a decrease in the sense of presence. Summary of the Invention

[0005] Therefore, various embodiments of the present invention provide a method for classifying sounds contained in a wide variety of content into speech and non-speech, and implementing automatic volume control technology based on the volume of speech, thereby improving the speech recognition performance of audiovisual content and maintaining the dynamics of background noise.

[0006] The technical issues to be addressed by this invention are not limited to those mentioned above. Those skilled in the art can clearly understand other technical issues not mentioned through the following description.

[0007] According to various embodiments, an automatic volume control device based on a voice signal may include: an object separator that separates an audio input signal into a voice signal and a non-voice signal; a voice signal processing unit that processes the voice signal and determines a volume compensation value based on the voice signal and set volume information; a non-voice signal processing unit that processes the non-voice signal; and a volume controller that, based on the volume compensation value, controls the volume of an audio output signal obtained by adding the voice output signal and the non-voice output signal processed and output by the voice signal processing unit and the non-voice signal processing unit.

[0008] According to one embodiment, the object separator can use artificial intelligence algorithms to separate the audio input signal into speech signals and non-speech signals.

[0009] According to one embodiment, the speech signal processing unit may include: a speech volume analyzer that calculates the average power of the speech signal; a volume compensator that determines the volume compensation value based on the average power of the speech signal calculated by the speech volume analyzer and the set volume information; a speech automatic gain controller that calculates a gain for compensating for the magnitude of the speech signal based on the average power and cumulative power of the speech signal; a speech normalizer that normalizes the speech signal by reflecting the gain; and a speech equalizer that changes the timbre of the speech signal.

[0010] According to one embodiment, the voice volume analyzer can process the real-time input voice signal in units of frames of a specified size, and determine whether there is a voice signal in the frame. If it is determined that there is a voice signal, the average power of the current frame can be calculated, and the accumulated power can be calculated by accumulating the calculated average power.

[0011] According to one embodiment, the voice volume analyzer can calculate the cumulative power by multiplying the average power by a weight value and adding the value of the previously accumulated power multiplied by (1 - weight value); the weight value can have a value between 0 and 1.

[0012] According to one embodiment, the volume compensator can acquire user-set volume information and determine the volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the set volume information, so that the accumulated power corresponds to the volume information.

[0013] According to one embodiment, the volume compensator can store the volume of a voice signal previously set by the user, and determine the volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the stored volume of the voice signal, such that the accumulated power corresponds to the volume of the voice signal.

[0014] According to one embodiment, the volume compensator may have an identification device for confirming personal or biometric information, and based on the identification device, distinguish multiple users, distinguish multiple content platforms used by the multiple users, store the volume of the voice signal used by the multiple users on each of the multiple content platforms by time period, obtain the volume of a set voice signal based on the content platform currently in the audiovisual experience, the identified user, and the time period, and determine the volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the volume of the obtained voice signal, so that the accumulated power corresponds to the volume of the voice signal.

[0015] According to one embodiment, the automatic gain controller can compare the average power with the set volume information, and if the average power is less than the set volume, select and determine the smaller value among the difference between the set volume and the average power multiplied by the amplification ratio and the maximum gain as the gain; if the average power is greater than the set volume, select and determine the larger value among the difference between the set volume and the average power multiplied by the attenuation ratio and the minimum gain as the gain.

[0016] According to one embodiment, the non-speech signal processing unit may include: a sound field sensor processor that imparts a sound field effect to the non-speech signal; a non-speech volume analyzer that calculates an average power by analyzing the non-speech signal; a non-speech automatic gain controller that calculates a gain for compensating for the volume of the non-speech signal based on the average power and cumulative power of the non-speech signal analyzed by the non-speech volume analyzer; and a non-speech normalizer that normalizes the non-speech signal by reflecting the gain.

[0017] According to one embodiment, the non-speech volume analyzer can process the real-time input non-speech signal in units of a specified size (frame), and determine whether there is a non-speech signal in a frame. If it is determined that there is no non-speech signal, it is excluded from the calculation of the average power. If it is determined that there is a non-speech signal, the average power of the current frame is calculated, and the accumulated power is calculated by accumulating the calculated average power.

[0018] According to one embodiment, the non-speech automatic gain controller can, when comparing the average power and the target volume value of the non-speech signal, select and determine the larger of the difference between the target volume value and the average power multiplied by the attenuation ratio and the minimum gain if the average power is greater.

[0019] According to various embodiments, an automatic volume control method based on a speech signal may include: an object separation step of separating an audio input signal into a speech signal and a non-speech signal; a speech signal processing step of processing the speech signal and determining a volume compensation value based on the speech signal and set volume information; a non-speech signal processing step of processing the non-speech signal; and a volume control step of controlling the volume of an audio output signal obtained by adding the speech output signal and the non-speech output signal processed and output in the speech signal processing step and the non-speech signal processing step based on the volume compensation value.

[0020] According to one embodiment, the object separation step can utilize an artificial intelligence algorithm to separate the audio input signal into speech signals and non-speech signals.

[0021] According to one embodiment, the speech signal processing steps may include: a speech volume analysis step for calculating the average power of the speech signal; a volume compensation step for determining the volume compensation value based on the average power of the speech signal calculated by the speech volume analysis step and the set volume information; a speech automatic gain control step for calculating a gain for compensating the magnitude of the speech signal based on the average power and cumulative power of the speech signal; a speech normalization step for normalizing the speech signal by reflecting the gain; and a speech equalizer step for changing the timbre of the speech signal.

[0022] According to one embodiment, the voice volume analysis step may include: processing the real-time input voice signal in units of frames of a specified size; determining whether the voice signal exists within a frame; and if the voice signal is determined to exist, calculating the average power of the current frame and calculating the cumulative power by accumulating the calculated average power.

[0023] According to one embodiment, the voice volume analysis step may include calculating the cumulative power by multiplying the average power by a weight value and adding the value of multiplying the previously accumulated power by (1 - weight value); the weight value may have a value between 0 and 1.

[0024] According to one embodiment, the volume compensation step may include: a step of acquiring user-defined volume information; and a step of determining a volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the defined volume information, such that the accumulated power corresponds to the volume information.

[0025] According to one embodiment, the volume compensation step may include: storing the volume of a voice signal previously set by the user; and determining a volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the stored volume of the voice signal, such that the accumulated power corresponds to the volume of the voice signal.

[0026] According to one embodiment, the volume compensation step may include: distinguishing a plurality of users; distinguishing a plurality of content platforms used by the plurality of users; storing the volume of the voice signal used by the plurality of users on each of the plurality of content platforms by time period; obtaining a set volume of the voice signal based on the content platform currently in the audiovisual experience, the identified user, and the time period; and determining a volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the obtained volume of the voice signal, such that the accumulated power corresponds to the volume of the voice signal.

[0027] According to one embodiment, the automatic voice gain control step may include: when the average power is less than the set volume, selecting and determining the smaller value between the difference between the set volume and the average power multiplied by the amplification ratio and the maximum volume as the gain; and when the average power is greater than the set volume, selecting and determining the larger value between the difference between the set volume and the average power multiplied by the attenuation ratio and the minimum volume as the gain.

[0028] According to one embodiment, the non-speech signal processing step may include: a sound field effect processing step of imparting a sound field effect to the non-speech signal; a non-speech volume analysis step of calculating the average power of the non-speech signal by analyzing the non-speech signal; a non-speech automatic gain control step of calculating a gain to compensate for the volume of the non-speech signal based on the average power and cumulative power of the non-speech signal analyzed in the non-speech volume analysis step; and a non-speech normalization step of normalizing the non-speech signal by reflecting the gain.

[0029] According to one embodiment, the non-speech volume analysis step may include: processing the real-time input non-speech signal in units of frames of a specified size; determining whether a non-speech signal exists within a frame; and if it is determined that there is no non-speech signal, excluding it from the calculation of the average power, and if it is determined that there is a non-speech signal, calculating the average power of the current frame, and calculating the cumulative power by accumulating the calculated average power.

[0030] According to one embodiment, the non-speech automatic gain control step may include: when comparing the average power and the target volume value of the non-speech signal, if the average power is larger, selecting and determining the larger of the difference between the target volume value and the average power multiplied by the attenuation ratio and the minimum gain as the gain.

[0031] According to one embodiment of the present invention, an automatic volume control technique is implemented by analyzing the sound contained in a wide variety of content and based on the volume of the speech, thereby improving the performance of speech recognition.

[0032] According to one embodiment of the present invention, the content sound is amplified based on the volume of the speech without affecting the dynamics of the background sound, thereby maintaining the sense of presence when watching audiovisual content.

[0033] The effects achievable by this invention are not limited to those mentioned above, and those skilled in the art can clearly understand other effects not mentioned through the following description. Attached Figure Description

[0034] Figure 1 This is a diagram illustrating a simplified configuration of an automatic volume controller for an audio input signal according to an embodiment.

[0035] Figure 2 The histogram represents the frequency and application based on the magnitude of the audio input signal. Figure 1 The frequency diagram following the automatic volume control is based on the magnitude of the audio output signal.

[0036] Figure 3 It represents the waveform of the audio input signal and its applications. Figure 1 A graph showing the waveform of the audio output signal after the automatic volume control.

[0037] Figure 4 This is a diagram of a voice-normalized volume controller that separates and processes an audio input signal into speech and non-speech signals according to one embodiment.

[0038] Figure 5 Applications are represented by histograms Figure 4A graph showing the frequency of the audio output signal following the voice-standardized volume controller.

[0039] Figure 6 It indicates application Figure 4 A graph showing the waveform of the audio output signal after the voice-standardized volume controller.

[0040] Figure 7 This is a diagram of an automatic volume controller based on a speech signal, relating to various embodiments of the present invention, which separates and processes an audio signal input into a speech signal and a non-speech signal, and automatically controls the volume based on the magnitude of the speech signal.

[0041] Figure 8 The audio input signal is represented by a histogram. Figure 7 The diagram shows the frequency of the volume of the audio output signal after processing in the automatic volume controller 30 based on the voice signal.

[0042] Figure 9 It is shown by Figure 7 The diagram shows the waveform of the audio output signal after processing by an automatic volume controller based on voice signals.

[0043] Explanation of reference numerals in the attached figures

[0044] 10: Automatic volume controller; 20: Voice-standardized volume controller

[0045] 30: Automatic volume controller based on voice signals

[0046] 110: Audio Analyzer; 120: Automatic Gain Controller

[0047] 130: Equalizer 200: Object Separator

[0048] 300: Voice signal processor; 310: Voice volume analyzer

[0049] 320: Speech Normalizer; 330: Speech Equalizer

[0050] 400: Non-speech signal processor; 10: Non-speech volume analyzer

[0051] 420: Non-voice automatic gain controller; 500: Voice signal processor

[0052] 510: Voice volume analyzer; 520: Volume compensator

[0053] 530: Automatic Gain Controller for Speech 540: Speech Normalizer

[0054] 550: Voice equalizer; 600: Non-voice signal processor

[0055] 610: Sound field sensor processor; 620: Non-voice volume analyzer

[0056] 630: Non-speech automatic gain controller; 640: Non-speech normalizer

[0057] 700: Volume Control Detailed Implementation

[0058] The advantages, features, and implementation methods of the present invention will become clearer from the embodiments described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in many different forms. These embodiments are provided only to fully disclose the invention and to completely reveal the scope of the invention to those skilled in the art. The invention is defined solely by the scope of the claims. Throughout this specification, the same reference numerals denote the same constituent elements.

[0059] If a component is described as "connected to" or "coupled to" another component, it includes cases where it is directly connected to or coupled to the other component, and cases where another component is interspersed in between. Conversely, if a component is described as "directly connected to" or "directly coupled to" another component, it indicates that no other component is interspersed in between. "And / or" includes each of the mentioned items and all combinations of more than one.

[0060] The terminology used in this specification is for illustrative purposes and is not intended to limit the invention. Unless otherwise specified, singular expressions in this specification include plural expressions. The use of "comprises" and / or "comprising" in this specification does not exclude the presence or addition of more than one other constituent element, step, action, and / or element in addition to those mentioned.

[0061] Although terms such as "first" and "second" are used to describe various constituent elements, these constituent elements are not limited to these terms. These terms are used only to distinguish one constituent element from another.

[0062] Therefore, within the technical concept of this invention, the first constituent element mentioned below can be a second constituent element. Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in the sense that is commonly understood by those skilled in the art. Furthermore, unless explicitly defined, terms defined in commonly used dictionaries should not be idealized or over-interpreted.

[0063] The terms "unit" or "module" used in this embodiment refer to software or hardware components such as FPGAs or ASICs that perform certain functions. However, "unit" or "module" is not limited to the meaning of software or hardware. A "unit" or "module" can be configured to exist in addressable storage media or to execute on one or more processors. Thus, as an example, a "unit" or "module" can include software components, objects pointing to software components, class components, and components such as task components, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided by components and "units" or "modules" can be combined into fewer components and "units" or "modules," or can be separated into more components and "units" or "modules."

[0064] Figure 1 This is a diagram illustrating a simplified configuration of an automatic volume controller for an audio input signal according to one embodiment. Figure 2 The histogram represents the frequency and application of the volume of the audio input signal. Figure 1 The automatic volume control then displays a frequency graph based on the volume of the audio output signal. Figure 3 It represents the waveform of the audio input signal and its applications. Figure 1 A graph showing the waveform of the audio output signal after the automatic volume control.

[0065] The automatic volume controller 10 utilizes automatic volume control technology designed to eliminate audiovisual discomfort caused by volume differences in various audio environments. This technology automatically adjusts the volume balance to allow listeners to comfortably listen to audio. Developed to eliminate discomfort caused by volume differences between voices and background noise in content, volume differences between advertisements and content on a content platform, and volume differences between different pieces of content, automatic volume control technology can be applied to devices that play content (e.g., TVs, smartphones, tablets, etc.).

[0066] The automatic volume controller 10 can analyze the magnitude of the audio signal in real time and automatically amplify or attenuate the signal based on a set reference value before outputting it. By increasing the volume of soft sounds and decreasing the volume of loud sounds, the automatic volume controller 10 balances the overall volume, thereby enabling the listener to listen comfortably by reducing changes in the volume of the audio input.

[0067] Reference Figure 1 The automatic volume controller 10 may include an audio analyzer 110, an automatic gain controller 120, and an equalizer 130.

[0068] The audio analyzer 110 can analyze the volume, dynamic range, etc. of the audio input signal 1.

[0069] Audio analyzer 110 can determine the volume of the audio signal by analyzing the amplitude of the audio input signal 1. Audio analyzer 110 can also determine the peak value of the audio input signal 1 by analyzing its amplitude. When the audio input signal 1 contains a signal with a peak value exceeding the maximum peak value that audio analyzer 110 can process, clipping, a waveform distortion and deception phenomenon, may occur. Clipping is a distortion phenomenon that occurs when the amplitude of the audio signal exceeds the maximum limit that audio analyzer 110 can process. If clipping occurs, the sound becomes distorted or harsh, and distortion of higher harmonics may produce unpleasant high-frequency sounds that were not originally present. Furthermore, the deterioration in audio quality may cause ear fatigue and discomfort to the listener.

[0070] The audio analyzer 110 can analyze the dynamic range of an audio signal. Dynamic range represents the difference between the smallest and largest signal. The larger the dynamic range, the greater the volume variation of the audio signal; the smaller the dynamic range, the smaller the volume variation. If the dynamic range is too narrow, the sound may sound muffled; if the dynamic range is too wide, the volume difference is extreme, which may cause discomfort to the listener. Therefore, it is necessary to adjust the dynamic range as needed to output a natural audio signal.

[0071] The automatic gain controller 120 can determine the gain of the audio input signal 1 in real time based on the results of the audio analyzer 110. The gain can be a ratio value multiplied by the audio signal to amplify or attenuate it. When the gain is greater than 1, the audio signal can be amplified; when the gain is less than 1, the audio signal can be attenuated. According to one embodiment, when the overall volume of the audio input signal 1 is low, an average volume target value can be set, and the gain can be determined based on this value to amplify the volume of the audio input signal 1 by the same ratio.

[0072] To prevent clipping of the audio input signal 1, the automatic gain controller 120 can limit the volume of the audio input signal 1 to below its maximum peak value. Furthermore, if a sudden increase in volume over a short period is determined to indicate a wide dynamic range, the automatic gain controller 120 can narrow the dynamic range by amplifying the low volume and reducing the high volume. In other words, the automatic gain controller 120 can increase the volume by amplifying the low volume and can eliminate distorted sounds and sudden volume changes that may cause discomfort to the listener.

[0073] Equalizer 130 can adjust the timbre by amplifying or attenuating specific frequencies. While automatic gain controller 120 amplifies or reduces the overall volume of the audio signal, equalizer 130 can change the timbre by amplifying or reducing the volume of specific frequency bands. For example, equalizer 130 can amplify the frequency band corresponding to human voices to make speech clearer, and can remove high-frequency or low-frequency bands by applying a high-pass filter or a low-pass filter.

[0074] Reference Figure 2 Histogram 11 shows the audio input signal 1 received from the original content of the content platform, divided into speech signal and non-speech signal, according to the frequency of volume. Histogram 12 shows the application... Figure 1 The audio output signal 2 following the automatic volume controller 10 shown is divided into voice signal and non-voice signal, and is based on the frequency diagram of the volume.

[0075] Referring to histogram 11, in audio input signal 1, the average value of the non-speech signal is -36.98 dB, and the average value of the speech signal is -34.69 dB. Referring to histogram 12, in application... Figure 1 In the case of the automatic volume controller 10 shown, the average value of the non-speech signal of the audio output signal 2 is -38.80dB, and the average value of the speech signal is -37.98dB. That is, it can be confirmed that the automatic volume controller 10 reduces the overall volume.

[0076] Referring to histogram 11, it can be confirmed that in audio input signal 1, the volume of non-speech signals is approximately between -17dB and -53dB, and the volume of speech signals is approximately between -20dB and -47dB. Furthermore, it can be confirmed that in audio output signal 2 using the automatic volume controller 10, the volume of non-speech signals is approximately between -29dB and -53dB, and the volume of speech signals is approximately between -29dB and -47dB. That is, it can be confirmed that the automatic volume controller 10 reduces the dynamic range by decreasing the volume of audio signals with a volume range of -17dB to -28dB.

[0077] Reference Figure 3 When comparing the waveform 31 of the audio input signal 1 with the waveform 32 of the audio output signal 2 that has passed through the automatic volume controller 10, it can be seen that the volume is reduced in the larger volume part 33.

[0078] As a result, while viewers may hear more comfortable sounds, the sensitivity of the automatic volume controller 10 decreases when used with audio signals that are highly sensitive to background noise or sound effects, potentially making it difficult to perceive a sense of presence. Furthermore, since the volume of both speech and non-speech audio signals decreases at the same rate, it is impossible to determine whether the speech signal recognition performance has improved.

[0079] Figure 4 This is a diagram of a voice-normalized volume controller 20 according to an embodiment, which separates an audio input signal into a speech signal and a non-speech signal and processes them. Figure 5 Applications are represented by histograms Figure 4 After the voice-standardized volume controller 20, based on the frequency graph of the audio output signal volume, Figure 6 It indicates application Figure 4 A graph showing the waveform of the audio output signal after the voice-standardized volume controller.

[0080] Reference Figure 4 The voice-standardized volume controller 20 may include an object separator 200, a voice signal processor 300, a non-voice signal processor 400, and a volume controller 700.

[0081] The object separator 200 functions to separate the audio input signal 1 into a speech signal and a non-speech signal. According to one embodiment, the object separator 200 can utilize an artificial intelligence algorithm to separate the audio input signal 1 into a speech signal and a non-speech signal and output them. The speech signal output from the object separator 200 can be transmitted to the speech signal processor 300, while the non-speech signal can be transmitted to the non-speech signal processor 400.

[0082] The speech signal processor 300 may include a speech volume analyzer 310, a speech normalizer 320, and a speech equalizer 300. The speech volume analyzer 310 can analyze the amplitude of the speech signal to determine its magnitude. Furthermore, the speech volume analyzer 310 can determine the average volume of the entire speech signal by calculating its average power.

[0083] The speech normalizer 320 can normalize the speech signal based on the average power of the speech signal calculated by the speech volume analyzer 310, thereby adjusting the volume of the entire speech signal. That is, the speech normalizer 320 can increase or decrease the volume of the entire speech signal by the same ratio. Furthermore, the speech normalizer 320 can, through the normalization process, ensure that the volume of the speech signal is within the range between the minimum and maximum values ​​that the speech normalization volume controller 20 can handle.

[0084] Thus, the speech signal output by the speech normalizer 320 can be transmitted to the speech equalizer 330, where the process of modifying the tone can be performed. The speech equalizer 330 can reduce unpleasant-sounding frequency bands such as crackles and sibilants in the final speech signal, while highlighting the frequency bands corresponding to the speech, making the overall tone soft and penetrating.

[0085] The non-speech signal processor 400 may include a non-speech volume analyzer 410 and a non-speech automatic gain controller 420. The non-speech volume analyzer 410 can calculate at least one of the minimum peak value, maximum peak value, and average power of the non-speech signal output from the object separator 200. Based on these values, the non-speech automatic gain controller 420 can set a target value, determine the gain based on the target value, and amplify or attenuate the overall volume of the non-speech signal by multiplying the non-speech signal by the gain.

[0086] If the volume of the non-speech signal exceeds the maximum peak value that the speech normalization volume controller 20 can handle, the non-speech automatic gain controller 420 can limit the non-speech signal to below the maximum peak value to prevent clipping. Furthermore, if a sudden increase in volume within a short period indicates a wide instantaneous dynamic range, the non-speech automatic gain controller 420 can narrow the dynamic range by increasing the volume of the smaller sound and decreasing the volume of the larger sound. In other words, the non-speech automatic gain controller 420 can increase the volume by amplifying smaller sounds and eliminate factors that may cause discomfort to the listener, such as distorted sounds, background noise that suddenly increases in volume, and other changes in the volume of the non-speech signal.

[0087] The volume controller 700 can receive a signal, which includes the voice signal processed by the voice signal processor 300 and the non-voice signal processed by the non-voice signal processor 400, as input, and increase or decrease the volume of the input signal. The volume controller 700 performs the function of adjusting the volume of the final audio output signal 3, and the user can adjust the volume using a device such as a volume knob or buttons connected to the volume controller 700.

[0088] Reference Figure 5 Figure 13 shows the application Figure 4The audio output signal 3 following the voice-standardized volume controller 20 shown is divided into voice signal and non-voice signal, according to the frequency diagram of the volume.

[0089] Reference Figure 2 and Figure 5 The histograms 11, 12, and 13 shown indicate that the average volume of the non-speech signal of audio input signal 1 is -36.98 dB. (Application...) Figure 2 The average volume of the non-speech signal of the audio output signal 2 when the automatic volume controller 10 is shown is -38.80 dB. Figure 4 The average volume of the non-speech signal of the audio output signal 3 when the voice-normalized volume controller 20 is used is -42.70dB. On the other hand, the average volume of the speech signal of the audio input signal 1 is -34.69dB. Figure 2 The average volume of the audio output signal 2 of the automatic volume controller 10 shown is -37.98dB. (Application) Figure 4 The average volume of the audio output signal 3 of the voice-standardized volume controller 20 is -37.47dB.

[0090] It can be seen that, in the case of a voice signal, the volume of the audio output signals 2 and 3 of the automatic volume controller 10 and the voice-normalized volume controller 20 is reduced compared to the volume of the audio input signal 1, but the volume difference between the automatic volume controller 10 and the voice-normalized volume controller 20 is not significant. That is, it can be concluded that the automatic volume controller 10 and the voice-normalized volume controller 20 process the voice signal under the same or similar conditions.

[0091] As can be seen, in the case of non-speech signals, the average volume when the speech normalization volume controller 20 is applied is significantly reduced compared to the average volume of the audio output signal 2. That is, the speech normalization volume controller 20 applies different gains to the speech and non-speech signals, thereby allowing the listener to perceive a higher volume of the speech signal than the non-speech signal. Therefore, the speech normalization volume controller 20 can amplify the speech signal more than the non-speech signal.

[0092] The volume distribution of the audio signal is explained, refer to... Figure 2 Histogram 11 shows that in audio input signal 1, the volume of non-speech signals is approximately between -17dB and -53dB, while the volume of speech signals is approximately between -20dB and -47dB. (Refer to...) Figure 5Histogram 13 shows that in the audio output signal 3 using the voice-normalized volume controller 20, the volume of the non-speech signal is approximately between -32dB and -59dB, while the volume of the speech signal is approximately between -32dB and -47dB. That is, the voice-normalized volume controller 20 reduces the volume of the non-speech signal more than the volume of the speech signal, and reduces the dynamic range by decreasing the volume of the audio signal with a volume range of -17dB to -31dB.

[0093] Reference Figure 6 It can be seen that the waveform 34 of the audio output signal 3 passed through the voice standardization volume controller 20 reduces the volume of the larger part 310 in the audio input signal 10.

[0094] As a result, it can be determined that, compared to the automatic volume controller 10, the voice-normalized volume controller 20, which separates and processes audio into speech and non-speech signals, does not produce significantly different processing results because it performs similar processing on the speech signal. Conversely, it can be seen that the voice-normalized volume controller 20 makes the dynamic range of the non-speech signal similar to that of the audio input signal 1, while reducing the volume more than the speech signal. The voice-normalized volume controller 20 can process sounds such as background noise or effects into background noise that is closer to the original audio signal than the automatic volume controller 10, and because it processes the non-speech and speech signals separately, it can process them more effectively, matching their respective characteristics. However, adjusting the volume ratio between the speech and non-speech signals may not be possible.

[0095] Figure 7 This is a diagram illustrating the configuration of an automatic volume controller based on a voice signal, according to various embodiments of the present invention, which separates an audio input signal into a voice signal and a non-voice signal and processes them, and automatically controls the volume of the audio signal based on the volume of the voice signal.

[0096] Reference Figure 7 The automatic volume controller 30 based on voice signals may include an object separator 200, a voice signal processing unit 500, a non-voice signal processing unit 600, and a volume controller 700.

[0097] The automatic volume controller 30 based on voice signals includes at least one processor (not shown) or digital signal processor (DSP) and is capable of processing... Figure 7 The various functions are shown.

[0098] In the automatic volume controller 30 based on voice signals, the object separator 200 can separate the audio input signal into voice signals and non-voice signals and output them. According to one embodiment, the object separator 200 can utilize an artificial intelligence algorithm that separates the audio signal of the played content, i.e., the audio input signal, into voice signals and non-voice signals.

[0099] The speech signal processing unit 500 can process the speech signal separated and output by the object separator 200. The speech signal processing unit 500 may include a speech volume analyzer 510, a volume compensator 520, a speech automatic gain controller 530, a speech normalizer 540, and a speech equalizer 550.

[0100] The voice volume analyzer 510 can calculate the average power of the real-time input voice signal. At this time, the voice volume analyzer 510 can process the real-time input voice signal in units of frames. According to one embodiment, if the voice volume analyzer 510 determines that there is no voice signal in a frame due to its low calculated average power, it can exclude the average power calculated for that frame from subsequent calculations.

[0101] The voice volume analyzer 510 can calculate the accumulated power by accumulating the calculated average power. According to one embodiment, the voice volume analyzer 510 can calculate the accumulated power as (current average power × weight value) + previous accumulated power × (1 - weight value). Here, the weight value is a value representing the degree to which the currently calculated average power is reflected in the accumulated power, and can have a value between 0 and 1. The accumulated power can also be referred to as smoothing power, which can be used to reduce the magnitude of volume changes when the volume of the voice signal changes abruptly.

[0102] The Voice Volume Analyzer 510 can determine the minimum or maximum peak value of a voice signal by analyzing its amplitude.

[0103] The volume compensator 520 can monitor the cumulative power calculated by the voice volume analyzer 510 based on the analysis of the voice signal. Furthermore, the volume compensator 520 can read the volume value set by the user. The volume compensator 520 can compensate for the volume of the entire audio signal by increasing or decreasing it when it is determined that the current voice signal volume is unsuitable for the listener, or when it differs from the user-set volume value.

[0104] The volume compensator 520 can be connected to at least one of the following devices: a camera, microphone, fingerprint recognition, or other biometric identification device that can distinguish the user of the device using the playback content, and a device that can verify personal information.

[0105] According to one embodiment, the volume compensator 520 can store the average volume in a user's personal account when the user, logged into a device with a personal account for playing content, adjusts the volume. Later, the overall volume can be set based on the volume stored for each user, according to the identified individual. Therefore, personalized volume adjustment can be achieved.

[0106] According to one embodiment, the volume compensator 520 can store the average volume in the personal account of the user who logged into the device playing the content, according to the different volume preferences of the user in different time periods. Then, the overall volume can be set according to the volume of each user in different time periods, based on the identified user and time.

[0107] According to one embodiment, the volume compensator 520 can store the average volume used by the platform in the personal account of a user who has logged into a personal account on a device playing content, when the user adjusts and sets the volume according to the content platform. When connecting to the platform later, the overall volume can be set based on the volume stored by each person and according to the identified individual.

[0108] The automatic gain controller 530 can amplify or attenuate the voice signal. The gain can be set to amplify or attenuate the signal. The gain of the voice signal can be determined based on the average power result of the voice volume analyzer 510 and the target volume value determined by the volume compensator 520.

[0109] The automatic gain controller 530 can compare the average power of the voice volume analyzer 510 with the target volume value of the volume compensator 520, and if the average power is low, select the smaller value between the difference between the target volume value and the average power multiplied by the amplification ratio and the maximum gain. Here, the maximum gain can be the maximum value that the device can maximize to amplify the magnitude of the voice signal.

[0110] The automatic gain controller 530 can compare the average power of the voice volume analyzer 510 with the target volume value of the volume compensator 520, and if the average power is higher, select the larger of the following values: the difference between the target volume value and the average power multiplied by the attenuation ratio, and the minimum gain. Here, the minimum gain can be the minimum value that can attenuate the voice signal to the maximum extent.

[0111] The gain determined by the automatic gain controller 530 is passed to the speech normalizer 540 and undergoes a speech signal normalization process. During speech signal normalization, the clarity of the speech signal can be improved by increasing the overall volume. Furthermore, as a process of processing the speech signal to make it comfortable for the listener to hear, in the case of a speech signal exceeding its maximum volume, it can be attenuated to a value below the maximum volume to prevent potential distortion. In the case of a speech signal that is inaudible due to a value below its minimum volume, the volume of the speech signal can be increased by amplifying it above the minimum volume. Thus, the speech signal normalization process can be understood as a process of processing the speech signal to achieve a volume that is pleasing to the listener.

[0112] The speech signal output from the speech normalizer 540 is transmitted to the speech equalizer 550. The speech equalizer 550 reduces unwanted noise in the speech signal and adjusts human voices to produce a clearer and more pleasant sound. The speech equalizer 550 improves the clarity of the speech signal by reducing sharp and harsh sounds in the 4kHz–6kHz frequency band, removing clipping sounds in the 150Hz frequency band, and boosting the human voice portion in the 300Hz–500Hz frequency band. The speech equalizer 330 can provide an optimized speech signal to the listener by enhancing or attenuating specific frequency bands.

[0113] The non-speech signal processing unit 600 may include a sound field sensor processor 610, a non-speech volume analyzer 620, a non-speech automatic gain controller 630, and a non-speech normalizer 640.

[0114] The non-speech signals separated from the audio signal are mostly background noise or sound effects used to enhance the sense of presence. Therefore, producers can add sound or sound field effects to immerse viewers in the atmosphere or scene of the content. A sound field processor 610 can be added to make the sound more vivid. The sound field processor 610 can include sound field processing technologies and algorithms, such as Dolby Atmos, DTS:X, and 3D Audio.

[0115] The non-speech volume analyzer 620 can calculate the average power of the real-time input non-speech signal. At this time, the real-time input non-speech signal can be processed frame by frame. Therefore, it can be determined whether there is a non-speech signal within a frame, and frames without signals can be excluded from the calculation. By accumulating and reflecting the average power thus calculated, the accumulated power can be calculated. The accumulated power can be calculated using (current average power × weight value) + previous accumulated power × (1 - weight value). Here, the weight value is a value representing the degree to which the currently calculated average power is reflected in the accumulated power, and can have a value between 0 and 1. This accumulated power can also be called smoothing power; when the volume of the non-speech signal changes drastically, the accumulated power can be used to reduce the magnitude of the volume change.

[0116] The non-speech volume analyzer 620 can determine the minimum or maximum peak value of a non-speech signal by analyzing the amplitude of the speech signal. According to one embodiment, the non-speech volume analyzer 620 can help reduce or increase the volume when the accumulated power is calculated and a non-speech signal exceeding the peak range is input.

[0117] The non-speech automatic gain controller 630 can amplify or attenuate speech signals. In this case, the gain can be set for either amplification or attenuation. The gain of the non-speech signal can be determined based on the average power calculation result of the non-speech volume analyzer 620 and the target volume value of the non-speech signal.

[0118] The non-speech automatic gain controller 630 can compare the average power of the non-speech volume analyzer 620 with the target volume value, and if the average power is lower, select the smaller value between the difference between the target volume value and the average power multiplied by the amplification ratio and the maximum gain. Here, the maximum gain can be the maximum value that the device can maximize to amplify the magnitude of the non-speech signal.

[0119] The non-speech automatic gain controller 630 can compare the average power of the non-speech volume analyzer 620 with the target volume value, and if the average power is higher, select the larger of the following values: the difference between the target volume value and the average power multiplied by the attenuation ratio, and the minimum gain. Here, the minimum gain can be the minimum value that minimizes the magnitude of the non-speech signal.

[0120] The gain determined by the non-speech automatic gain controller 630 can be passed to the non-speech normalizer 640, allowing the non-speech normalizer 640 to be used when normalizing non-speech signals. The normalization process of non-speech signals is the process of making the entire non-speech signal comfortable for the listener to hear. In the case of non-speech signals with a volume above the maximum peak value, distortion that may occur in the non-speech signal can be prevented by attenuating it to a value below the maximum peak value.

[0121] The normalization process for non-speech signals can be performed within the scope of preventing distortion caused by the non-speech signal reaching its maximum peak volume, and minimizing damage to the dynamic range of the non-speech signal. That is, it can refer to adjusting the volume to a level that will not cause discomfort to the listener and will not impair the sound field effect intended by the producer.

[0122] The volume controller 700 can receive an audio signal, which is the sum of the audio signal processed by the audio signal processing unit 500 and the non-audio signal processed by the non-audio signal processing unit 600, as input. Based on the target audio signal volume value output from the volume compensator 520 of the audio signal processing unit 500, it ultimately determines the output volume of the audio signal. The volume controller 700 adjusts the volume of the entire audio signal taking into account the target audio signal volume value. The listener can adjust the volume of the content using a volume knob or button connected to the volume controller 700.

[0123] Figure 8 Histograms are used to represent the results of... Figure 7 The diagram shown illustrates the frequency of the volume of the audio output signal 4 output by the automatic volume controller 30 based on the voice signal after processing the audio input signal 1. Figure 9 It is shown by Figure 7 The diagram shows the waveform of the audio output signal 4 output after processing by the automatic volume controller 30 based on the voice signal.

[0124] Reference Figure 2 , Figure 5 as well as Figure 8 The average volume of the audio output signal and non-speech signal of the audio input signal 1, the automatic volume controller 10, the voice-normalized volume controller 20 and the automatic volume controller 30 based on the voice signal according to one embodiment is shown in Table 1 below.

[0125] Table 1

[0126]

[0127] Referring to Table 1, it can be seen that the automatic volume controller 30 based on speech signals reduces the non-speech signal in the audio input signal from -36.98 to -40.14, and increases the speech signal from -34.69 to -32.71. Therefore, when viewers use the automatic volume controller 30 based on speech signals to listen to content, they can hear a clearer and louder speech signal, while conversely, they can hear relatively quieter background noise. In particular, when comparing the average volume of the speech signal and the non-speech signal of the audio input signal 1 and each of the audio output signals 2, 3, and 4, the automatic volume controller 30 based on speech signals proposed in this specification makes the speech signal louder than the non-speech signal, thereby improving the viewing and listening environment for viewers.

[0128] Reference Figure 2 , Figure 5 as well as Figure 8 Table 2 shows the volume distribution of voice signals and non-voice signals of audio output signals 2, 3, and 4 of audio input signal 1, automatic volume controller 10, voice-normalized volume controller 20, and voice signal-based automatic volume controller 30 according to an embodiment.

[0129] Table 2

[0130]

[0131] Referring to Table 2, it can be seen that, compared with the aforementioned automatic volume controller 10 and voice-normalized volume controller 20, the automatic volume controller 30 based on voice signals has a wider dynamic range for both voice and non-voice signals. Figure 9 The waveform 35 shown confirms that in the automatic volume controller 30 based on the voice signal, the change in the amplitude representing the dynamic range is similar to the change in the amplitude of the audio input signal 1.

[0132] Additionally, refer to Table 2, refer to Figure 8 As can be seen, the volume distribution of the voice signal in the audio output signal 4 of the automatic volume controller 30 based on the voice signal is similar to or slightly narrower than that of the audio input signal, while the volume distribution of the non-voice signal, although similar to that of the audio input signal 1, is shifted to the left compared to the voice signal of the audio input signal 1. That is, in the case of a non-voice signal, the volume output by the automatic volume controller 30 based on the voice signal is lower than that of the audio input signal 1. However, this is only one embodiment, and through the operation of the viewer, the volume of at least one of the voice signal and the non-voice signal of the automatic volume controller 30 based on the voice signal can be higher than that of the audio input signal 1.

[0133] The automatic volume controller 30 based on voice signals proposed in this specification can not only match the volume of the voice signal to the requirements of the listener, but also automatically adjust the volume of non-voice signals so that the voice signal can be heard clearly, while improving the sound field effect by minimizing the damage to the dynamic range of non-voice signals.

[0134] As described above, although the present invention has been illustrated with limited embodiments and accompanying drawings, the present invention is not limited to the above embodiments, and modifications or variations may be made to the described matters. Therefore, the scope of the present invention should not be limited to the illustrated embodiments, but should be determined by the scope of the following claims and their equivalents.

[0135] Those skilled in the art will understand that the invention can be implemented in modified forms without departing from the essential characteristics described. Therefore, the disclosed methods should be considered from an illustrative perspective, not a limiting one. It should be interpreted that the scope of the invention is defined by the claims rather than the foregoing description, and all modifications within the equivalent scope are part of the invention.

Claims

1. An automatic volume control device, which is an automatic volume control device based on voice signals, wherein, include: An object separator separates an audio input signal into a speech signal and a non-speech signal; The voice signal processing unit processes the voice signal and determines a volume compensation value based on the voice signal and set volume information. The non-speech signal processing unit processes the non-speech signals; as well as A volume controller, based on the volume compensation value, controls the volume of the audio output signal, which is the sum of the voice output signal and the non-voice output signal processed and output by the voice signal processing unit and the non-voice signal processing unit.

2. The automatic volume control device according to claim 1, wherein, The object separator uses artificial intelligence algorithms to separate the audio input signal into speech signals and non-speech signals.

3. The automatic volume control device according to claim 1, wherein, The speech signal processing unit includes: A voice volume analyzer calculates the average power of the voice signal; A volume compensator determines the volume compensation value based on the average power of the voice signal calculated by the voice volume analyzer and the set volume information. An automatic gain controller for speech calculates a gain to compensate for the magnitude of the speech signal based on the average power and cumulative power of the speech signal. A speech normalizer normalizes the speech signal by reflecting the gain; and A voice equalizer changes the timbre of the voice signal.

4. The automatic volume control device according to claim 3, wherein, The voice volume analyzer processes the real-time input voice signal in units of frames of a specified size, and determines whether there is a voice signal within the frame. If a voice signal is detected, the average power of the current frame is calculated, and the cumulative power is calculated by accumulating the calculated average power.

5. The automatic volume control device according to claim 4, wherein, The voice volume analyzer calculates the cumulative power by multiplying the average power by a weight value and adding the value of multiplying the previously accumulated power by (1 - weight value); The weight values ​​have values ​​between 0 and 1.

6. The automatic volume control device according to claim 3, wherein, The volume compensator acquires the volume information set by the user and determines the volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the set volume information, so that the accumulated power corresponds to the volume information.

7. The automatic volume control device according to claim 3, wherein, The volume compensator stores the volume of the voice signal previously set by the user, and determines the volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the stored volume of the voice signal, so that the accumulated power corresponds to the volume of the voice signal.

8. The automatic volume control device according to claim 7, wherein, The volume compensator has an identification device for confirming personal or biometric information, and distinguishes multiple users and multiple content platforms used by the multiple users based on the identification device. It stores the volume of the voice signal used by the multiple users on each of the multiple content platforms by time period. It obtains the volume of the set voice signal based on the content platform in the current audiovisual experience, the identified user, and the time period. It determines the volume compensation value for increasing or decreasing the overall volume by comparing the accumulated power with the volume of the obtained voice signal, so that the accumulated power corresponds to the volume of the voice signal.

9. The automatic volume control device according to claim 3, wherein, The automatic gain controller compares the average power with the set volume information. If the average power is less than the set volume, it selects and determines the smaller value among the difference between the set volume and the average power multiplied by the amplification ratio and the maximum gain. If the average power is greater than the set volume, it selects and determines the larger value among the difference between the set volume and the average power multiplied by the attenuation ratio and the minimum gain.

10. An automatic volume control method, which is an automatic volume control method based on voice signals. in, include: The object separation step involves separating the audio input signal into speech signals and non-speech signals. The voice signal processing steps involve processing the voice signal and determining the volume compensation value based on the voice signal and the set volume information. Non-speech signal processing steps for processing the non-speech signal; and A volume control step that controls the volume of the audio output signal obtained by adding the processed and output audio output signal and the non-audio output signal in the speech signal processing step and the non-audio signal processing step based on the volume compensation value.

Images