Audio playback control method and control apparatus

Abstract

The application discloses an audio playing control method, comprising: acquiring first audio data; acquiring an environmental noise decibel value; performing sound source separation on the first audio data to obtain a plurality of second audio data with different frequency structures; adjusting the volume of the plurality of second audio data according to the environmental noise decibel value; performing audio mixing on the second audio data after volume adjustment; and adjusting the playing volume of the second audio data after audio mixing according to the environmental noise decibel value. The audio playing control method provided by the application can realize sound source separation of playing audio data, and adjust the volume of, for example, target sound source data and environmental sound source data according to the environmental noise condition, thereby improving the automatic adjustment capability of the system. The volume of sound of different sound sources is adjusted, so that sound containing substantial content such as a human voice sound source is highlighted in a noisy environment, and all kinds of sound sources are balanced in a quiet environment, giving people a sense of stability and improving the auditory experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio data processing, and in particular to an audio playing control method and an audio playing control device. BACKGROUND

[0002] At present, the volume of the audio output by the electronic devices or smart devices with audio output function on the market is usually adjusted manually by the user, or adjusted in the manner of system recommended volume value in combination with the detection of the audio playing state and the surrounding scene or environment, such as: the volume of the Bluetooth (or other wireless type) earphone is adjusted in the manner of button adjustment or automatic recommended optimal volume value; the volume of the car audio is automatically increased or decreased according to the vehicle speed to cover the noise in the car; the audio and video playing device acquires the currently played audio and video content, and automatically adjusts the volume according to the content.

[0003] How to automatically adjust the volume of the system through an adaptive scheme under various noise level environmental conditions to achieve the ideal playing level of the user in real time is a technical problem to be solved. SUMMARY

[0004] In view of at least one defect of the prior art, in a first aspect, the present application provides an audio playing control method, comprising:

[0005] acquiring first audio data;

[0006] acquiring an environmental noise decibel value;

[0007] performing sound source separation on the first audio data to obtain a plurality of second audio data with different frequency structures;

[0008] adjusting the volume of the plurality of second audio data according to the environmental noise decibel value;

[0009] mixing the second audio data after volume adjustment;

[0010] adjusting the playing volume of the second audio data after audio mixing according to the environmental noise decibel value.

[0011] According to the first aspect of the present application, the second audio data comprises:

[0012] one or more of vocal sound source data, drum sound source data, bass sound source data and other audio data.

[0013] According to the first aspect of the present application, the sound source separation on the first audio data to obtain a plurality of second audio data with different frequency structures further comprises:

[0014] sampling the first audio data;

[0015] analyzing frequency structure of the sampling result;

[0016] separating a plurality of third audio data according to different frequency structures;

[0017] waveform shaping the plurality of third audio data to obtain the plurality of second audio data.

[0018] According to the first aspect of the present application, wherein the volume adjustment of the plurality of second audio data according to the environmental noise decibel value comprises:

[0019] when the environmental noise decibel value is greater than a first threshold value, if the volume of the human voice sound source data is greater than or equal to the volume of the drum sound source data, increasing the volume of the human voice sound source data;

[0020] when the environmental noise decibel value is greater than the first threshold value, if the volume of the human voice sound source data is less than the volume of the drum sound source data, adjusting the volume of the human voice sound source data to be equal to the volume of the drum sound source data.

[0021] According to the first aspect of the present application, wherein the volume adjustment of the plurality of second audio data according to the environmental noise decibel value comprises:

[0022] when the environmental noise decibel value is less than or equal to a first threshold value, if the environmental noise decibel value is less than a second threshold value, adjusting the volumes of the human voice sound source data, the drum sound source data, the bass sound source data and the other audio data to be equal;

[0023] when the environmental noise decibel value is less than or equal to the first threshold value, if the environmental noise decibel value is greater than or equal to the second threshold value, maintaining the original volumes of the human voice sound source data, the drum sound source data, the bass sound source data and the other audio data.

[0024] According to the first aspect of the present application, wherein the adjustment of the playback volume of the second audio data after audio mixing according to the environmental noise decibel value comprises:

[0025] when the environmental noise decibel value is greater than a third threshold value, calculating a first decibel value, the first decibel value being equal to the environmental noise decibel value plus a first enhancement decibel value;

[0026] when the environmental noise decibel value is less than or equal to a third threshold value, calculating a second decibel value, the second decibel value being equal to the environmental noise decibel value plus a second enhancement decibel value; wherein the first enhancement decibel value is greater than the second enhancement decibel value;

[0027] According to the volume decibel corresponding table, a volume value corresponding to the first decibel value or the second decibel value is searched, and a playing volume of the currently played audio data is adjusted based on a search result.

[0028] According to a first aspect of the present application, the method further comprises:

[0029] reading a volume value of a current system in real time;

[0030] when receiving a system switching instruction, adjusting the volume value of the system after switching, so that the decibel value of the currently played audio data remains unchanged.

[0031] According to the first aspect of the present application, when the system switching instruction is received, the volume value of the system after switching is adjusted, comprising:

[0032] adjusting the volume value of the system after switching according to the volume decibel corresponding table; or

[0033] adjusting the volume value of the system after switching according to the corresponding relationship between the volume values of the system before and after switching and the maximum volume value of the system.

[0034] According to a second aspect, the present application further provides an audio playing control device, comprising:

[0035] an audio data acquisition module configured to acquire first audio data;

[0036] an environmental noise acquisition module configured to acquire an environmental noise decibel value;

[0037] a sound source separation module configured to

[0038] perform sound source separation on the first audio data to obtain a plurality of second audio data with different frequency structures;

[0039] a first volume adjustment module configured to adjust the volume of the plurality of second audio data according to the environmental noise decibel value;

[0040] an audio mixing module configured to mix the second audio data after volume adjustment;

[0041] a second volume adjustment module configured to adjust the playing volume of the second audio data after audio mixing according to the environmental noise decibel value.

[0042] According to a third aspect, the present application further provides an electronic device, comprising:

[0043] a processor; and

[0044] a memory storing a computer program, when the computer program is executed by the processor, the processor executes the method as introduced in the first aspect above.

[0045] The audio playing control method provided by the present application can realize sound source separation of playing audio data, and adjust the sound emitted by the target sound source and the environmental sound source according to the environmental noise, thereby improving the automatic adjustment capability of the system. On the basis of realizing sound source separation of playing audio data, the volume of the sound of different sound sources is adjusted, so that the sound containing substantial content such as the human voice sound source is highlighted in a noisy environment, and all kinds of sound sources are balanced in a quiet environment, giving people a sense of stability and improving the auditory experience. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without departing from the scope of the present application.

[0047] Figure 1 An audio playing control method provided by an embodiment of the present application is shown;

[0048] Figure 2 An audio playing control device provided by an embodiment of the present application is shown;

[0049] Figure 3 An audio playing control device provided by an embodiment of the present application is shown;

[0050] Figure 4 An audio playing control device provided by an embodiment of the present application is shown;

[0051] Figure 5 An audio playing control device provided by an embodiment of the present application is shown;

[0052] Figure 6 An audio playing control device provided by an embodiment of the present application is shown;

[0053] Figure 7 An electronic device provided by an embodiment of the present application is shown;

[0054] Figure 8 An audio playing control device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0055] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are some of the embodiments of the present application but not all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative efforts should fall into the scope of the present application.

[0056] The same video or audio file, when played in different environmental noise backgrounds, the user's playing appeal and the experience brought to the user by the same playing level are not different. For example, when playing an advertisement in a crowded noisy environment or on a public transport tool such as a subway, the drum, background sound, etc. in the advertisement make the human voice content difficult to distinguish, and the user usually pays more attention to the human voice content in the advertisement, and then obtains substantive information, and ignores the background sound, music, etc. In a live scene, the host will generally explain to the user and render the atmosphere under the background music, and promote the purchase desire. If the voice of the host can be enhanced and the background music, etc. can be weakened, the interference of the background music on the audience can be reduced. In a quiet audiovisual environment, such as a cinema or a family, when a movie is played by an audio and video playing device, appropriately enhancing the background bass and reducing the loud sound (such as the explosion sound in the movie) can make all kinds of sounds more balanced. In a quiet environment, the charm, color, and emotional taste of all kinds of timbres are enhanced, and the sound with high loudness is not too strong, and the sound with low loudness is enhanced, giving a comfortable feeling to the user.

[0057] Based on the above problems, the present application provides an audio playing control method, which can separate the sound emitted by different sound sources (for example, can be separated into target sound source sound and environmental sound source sound), and then perform enhancement or weakening operation on the sound emitted by different sound sources. For example, when the environment is relatively quiet, the human voice is reduced and the environmental sound is enhanced to improve the auditory comfort; when the environment is relatively noisy, the target sound source sound is enhanced to highlight the substantive content.

[0058] According to one embodiment of the present application, as shown in Figure 1 The present application provides an audio playing control method 10, including steps S101 to S106. Wherein:

[0059] In step S101, first audio data is acquired. In some embodiments, the first audio data being played or about to be played by the electronic device is acquired by a component having communication and storage functions.

[0060] In step S102, an environmental noise decibel value is obtained. In some embodiments, an environmental noise sensor (e.g., including a microphone) is disposed at or within a preset distance range of the electronic device. The environmental noise sensor collects sounds around the electronic device and converts them into an electrical signal. After filtering out the electrical signal corresponding to the audio data being played by the electronic device, the environmental noise signal is obtained. Based on the environmental noise signal, the environmental noise decibel value is calculated.

[0061] In step S103, the first audio data is subjected to sound source separation to obtain a plurality of second audio data having different frequency structures. For example, the first audio data is separated into human voice sound source data and environmental sound source data.

[0062] In step S104, the volume of the plurality of second audio data is adjusted based on the environmental noise decibel value. In some embodiments, the volume ratio of the second audio data having different frequency structures is adjusted based on the environmental noise decibel value at or near the electronic device. For example, in a noisy environment, the volume of the human voice sound source data is increased, and the volume of the environmental sound source data is decreased. In a relatively quiet environment, the volumes of the human voice sound source data and the environmental sound source data are adjusted to be equal.

[0063] In step S105, the second audio data after volume adjustment is subjected to audio mixing. In some embodiments, the second audio data after volume adjustment is subjected to audio mixing to generate audio data for playing by the electronic device.

[0064] In step S106, the playing volume of the second audio data after audio mixing is adjusted based on the environmental noise decibel value. In some embodiments, the playing volume of the mixed second audio data is adaptively adjusted as a whole based on the environmental noise decibel value at or near the electronic device.

[0065] The audio playing control method 10 provided by the above embodiments of the present application can realize sound source separation of the playing audio data and adjust the sound emitted by, for example, the target sound source and the environmental sound source based on the environmental noise condition, thereby improving the automatic adjustment capability of the system. Moreover, the audio playing control method 10 can automatically adjust the output volume of the electronic device based on the noise condition (noisy or quiet) of the surrounding environment, thereby reducing user intervention and improving the intelligent level of the electronic device.

[0066] According to an embodiment of the present application, in the audio playing control method 10, the sound source separation of the first audio data in step S103 to obtain a plurality of second audio data having different frequency structures includes:

[0067] separating one or more of vocal sound source data, drum sound source data, bass sound source data, and other audio data from the first audio data by the trained machine learning component.

[0068] The trained machine learning component is configured to read the first audio data and separate the first audio data. The machine learning component is pre-trained by a large amount of audio data to obtain model parameters, so as to have certain "cognition" for sound source data of different frequency structures (different sound sources or different timbres), and to be able to identify one or more sound source data from a mixture of a plurality of sound source data.

[0069] According to one embodiment of the present application, in the audio playing control method 10, the sound source separation of the first audio data in step S103 further comprises:

[0070] Firstly, the first audio data being played or about to be played is sampled, for example, the waveform of the first audio data in the first few seconds of playing is sampled, so that the machine learning model is adapted to the prominent frequencies in the first audio data, including vocal frequency, drum frequency, bass frequency, etc. The machine learning component generates a separate waveform for each sound element (prominent frequency).

[0071] Secondly, the frequency structure of the sampling result is analyzed, and a higher proportion of sound source frequency structure is inferred according to the above-mentioned sound elements (prominent frequencies). Different sound sources play the same frequency sound, but the frequency and number of overtones are different, which is the reason why different sound sources can be distinguished (different sound sources have different timbres). Therefore, according to the analyzed frequency structure, different sound sources can be separated.

[0072] Thirdly, a plurality of third audio data is separated according to different frequency structures. The model separates the sound sources according to the analyzed frequency structure of different sound sources to form a plurality of third audio data with different frequency structures.

[0073] Fourthly, the plurality of third audio data is waveform shaped to obtain the plurality of second audio data. The waveform corresponding to different sound sources is finely processed to remove burrs to obtain the shaped sound source waveform. Optionally, the machine learning component generates a plurality of audio files to store the plurality of second audio data.

[0074] According to an embodiment of the present application, in the audio playing control method 10, the input audio data is separated by a trained machine learning component to separate at least vocal sound source data, drum sound source data, and bass sound source data. Optionally, after the vocal sound source data, the drum sound source data, and the bass sound source data are filtered out from the input audio data, the remaining audio data is stored as an audio file as other audio data.

[0075] According to an embodiment of the present application, the trained machine learning component comprises a machine learning model based on a Demucs algorithm.

[0076] Those skilled in the art can understand that other machine learning models or components suitable for implementing sound source separation can also be used to implement the present application, and the present application does not limit this.

[0077] According to an embodiment of the present application, in the audio playing control method 10, the volume adjustment of the second audio data according to the environmental noise decibel value in step S104 comprises:

[0078] When the environmental noise decibel value is greater than the first threshold value, if the volume of the vocal sound data is greater than or equal to the volume of the drum sound data, the volume of the vocal sound data is increased.

[0079] When the environmental noise decibel value is greater than the first threshold value, if the volume of the vocal sound data is less than the volume of the drum sound data, the volume of the vocal sound data is adjusted to be equal to the volume of the drum sound data.

[0080] The first threshold value is a noise standard value V n When the environmental noise decibel value is greater than the noise standard value V n , it indicates that the electronic device is currently in a noisy environment, and the target sound source, such as the vocal sound source, should be highlighted. If the volume of the vocal sound source data is greater than or equal to the volume of the drum sound source data, the volume of the vocal sound source data is increased to highlight the sound emitted by the vocal sound source. If the volume of the vocal sound source data is less than the volume of the drum sound source data, the volume of the vocal sound source data is adjusted to be equal to the volume of the drum sound source data, i.e., the original volume ratio of the separated vocal sound source data and drum sound source data is not changed, and the sound emitted by the vocal sound source is highlighted to enable the user to still recognize the target vocal sound in a noisy environment.

[0081] According to an embodiment of the present application, in the audio playing control method 10, the volume adjustment of the second audio data according to the environmental noise decibel value in step S104 comprises:

[0082] When the environmental noise decibel value is less than or equal to the first threshold value, if the environmental noise decibel value is less than the second threshold value, the volumes of the vocal sound source data, the drum sound source data, the bass sound source data and the other audio data are adjusted to be equal.

[0083] When the environmental noise decibel value is less than or equal to the first threshold value, if the environmental noise decibel value is greater than or equal to the second threshold value, the original volumes of the vocal sound source data, the drum sound source data, the bass sound source data and the other audio data are maintained.

[0084] The first threshold value adopts a noise standard value V n When the environmental noise decibel value is less than or equal to the noise standard value V n , it indicates that the electronic device is currently in a relatively quiet external environment, at this time, the external environment is divided into two levels:

[0085] If the environmental noise decibel value is less than the second threshold value (for example, the second threshold value adopts half of the noise standard value V n / 2), it indicates that the electronic device is currently in an extremely quiet external environment, and the volumes of the vocal sound source data, the drum sound source data, the bass sound source data and the other audio data are adjusted to be equal to each other. When the electronic device is in an extremely quiet external environment, the decibel values of each sound source are greatly different, the sound content is extremely unbalanced, which will affect the taste, color, and emotional taste of the timbre, and may also bring a sense of instability to the user. Therefore, when the electronic device is in an extremely quiet external environment, the volumes of each sound source are adjusted to be equal, to improve the comfort of hearing. Alternatively, the equalizer parameter values of each sound source data (second audio data) after separation are adjusted to make the volumes of each sound source data equal.

[0086] Alternatively, the actual decibel value of the currently played audio data can also be obtained by the sensor 120, and then the appropriate volume is set according to the actual decibel value, the volumes of each sound source are adjusted, and the sound is balanced for the sound source with a lower decibel value.

[0087] If the environmental noise decibel value is greater than or equal to the second threshold value (for example, the second threshold value adopts half of the noise standard value V n / 2), it indicates that the electronic device is neither in a noisy environment nor in an extremely quiet external environment, and no adjustment is made to the volume ratio of the vocal sound source data, the drum sound source data, the bass sound source data and the other audio data. The vocal sound source data, the drum sound source data, the bass sound source data and the other audio data are mixed at the original volume ratio.

[0088] The audio playing control method 10 provided by the one or more embodiments of the present application can realize sound source separation of playing audio data, and adjust the volume of different sound sources, so as to highlight the sound source containing substantial content such as human voice in a noisy environment, and make all kinds of sound sources balanced in a quiet environment, giving people a sense of stability and improving the auditory experience.

[0089] According to an embodiment of the present application, in the audio playing control method 10, the step S106 of adjusting the playing volume of the second audio data after mixing according to the environmental noise decibel value comprises:

[0090] When the environmental noise decibel value is greater than the third threshold value, a first decibel value is calculated, and the second decibel value is equal to the environmental noise decibel value plus a first enhanced decibel value.

[0091] When the environmental noise decibel value is less than or equal to the third threshold value, a second decibel value is calculated, and the second decibel value is equal to the environmental noise decibel value plus a second enhanced decibel value.

[0092] Wherein, the first enhanced decibel value is greater than the second enhanced decibel value.

[0093] According to an embodiment of the present application, the above method further comprises: based on the volume decibel correspondence table, searching for the volume value corresponding to the first decibel value or the second decibel value, and adjusting the volume of the currently playing audio data.

[0094] In the early stage of running of the electronic device, a volume decibel correspondence table corresponding to different operating systems of the electronic device is obtained, that is, a volume decibel correspondence table. For devices that do not open the volume decibel correspondence table, the corresponding relationship between the volume and the decibel value is tested to obtain the volume decibel correspondence table, and the device model and the volume decibel correspondence table are entered into a cloud server. When the electronic device is turned on, the device model is obtained, the cloud server queries the volume decibel correspondence table corresponding to the model and caches the query result to a local data table.

[0095] When the electronic device plays sound, the surrounding environmental decibel value S2 is obtained, S2 is compared with the noise standard value V n , when S2 is greater than or equal to V n , the decibel value of the playing volume of the electronic device is enhanced, and the playing volume decibel value of the electronic device is adjusted to S=S2+T1; when S2 is less than V nAt this time, the decibel value of the playing volume of the electronic device is slightly enhanced or weakened, and the playing volume decibel value of the electronic device is adjusted to S=S2+T2, and when S2+T2 is less than the original playing sound decibel value of the device, the sound decibel value is weakened. Alternatively, according to the value of S, the volume decibel corresponding table is searched to obtain the corresponding volume value V3, and the output volume of the current system of the electronic device is calibrated to V3. And after switching the system, still based on the volume decibel corresponding table, the volume value V4 corresponding to S in the system after switching is inquired, and the output volume value of the corresponding system is set to V4. After adjustment, the playing volume decibel value of the electronic device is always greater than the ambient noise decibel value.

[0096] According to one embodiment of the present application, the noise standard value V n is: the daytime equivalent noise value of residential area is 55 decibels, and the nighttime equivalent noise value is 45 decibels; the daytime equivalent noise value of mixed industrial and commercial area is 60 decibels, and the nighttime equivalent noise value is 55 decibels.

[0097] T1 is an enhancement value when the ambient noise is greater than the noise standard value, and the enhancement value is large.

[0098] T2 is an enhancement value when the ambient noise is less than the noise standard value, and the enhancement value is small. That is

[0099] T1>T2

[0100] The audio playing control method 10 provided by one or more embodiments of the present application adjusts the system volume according to the environmental background to avoid playing sound too small in noisy environment, such as subway and shopping mall environment, and the device output sound is too small, and the user cannot obtain effective content. Or the playing sound is too large in a quiet environment, such as a relatively quiet environment in a family, and the device sound output is large, which may disturb the people or cause burden or impact to the user's hearing.

[0101] According to one embodiment of the present application, the audio playing control method 10 further comprises:

[0102] According to the volume decibel corresponding table, the volume value of the system after switching is adjusted.

[0103] Alternatively, according to the corresponding relationship between the volume value of the system before and after switching and the maximum volume value of the system, the volume value of the system after switching is adjusted.

[0104] After the electronic device is powered on, a current system volume decibel value is used for volume output. When the electronic device performs system switching, a first volume value V1 of a current first system is obtained, a corresponding decibel value S1 is searched based on a volume decibel correspondence table, the second volume value V2 of the switched second system is determined in the volume decibel correspondence table according to the decibel value, and the second system sets the output volume value as V2. If the device does not support volume decibel correspondence table searching for the time being, when the system is switched, the first volume value V1 of the current first system is obtained, and the following is calculated:

[0105] P1 = V1 / V max1

[0106] After switching to the second system, the output volume value of the second system is set as:

[0107] V2 = P1*V max2

[0108] The unified adjustment entrance is used to adjust the volume value TV1 of the current system (for example, the first system) when the user adjusts the volume, and the corresponding decibel S2 is searched according to the correspondence table. According to the decibel value S2, the volume value TV2 of the corresponding system is searched in the volume decibel correspondence table under other systems (for example, under the second system and the third system), and the volume output value is set as TV2 in the system, so that the output decibel values of all systems are always consistent. If the system does not support volume decibel correspondence table searching, TV2 is obtained according to the above-mentioned equal proportion calculation mode.

[0109] The audio playing control method 10 provided by the above-mentioned embodiments of the present application realizes the synchronous adjustment of the multi-system volume, avoids the situation that the playing sound is too large or too small after the system is switched, makes the system volume change smooth, keeps the playing sound consistent as a whole, and improves the user experience.

[0110] According to one embodiment of the present application, as shown in Figure 2 The present application provides an audio playing control device 100, which comprises a storage module 110, a sensor 120 and a processing module 130. Wherein:

[0111] The storage module 110 is configured to obtain first audio data. The storage module 110, for example, comprises a component with communication and storage functions, and in some embodiments, the storage module 110 receives and stores the first audio data that the electronic device is playing or will play.

[0112] The sensor 120 is configured to obtain an ambient noise decibel value. In some embodiments, the sensor 120, for example, comprises a microphone, is arranged at the electronic device or within a preset distance range of the electronic device. The sensor 120 collects sounds around the electronic device and converts them into an electrical signal. After filtering out the electrical signal corresponding to the audio data being played by the electronic device, the electrical signal is an ambient noise signal, based on which the ambient noise decibel value can be calculated.

[0113] The processing module 130 is coupled with the storage module 110 and the sensor 120, and is configured to:

[0114] The processing module 130 separates the first audio data into second audio data with different frequency structures, for example, vocal source data and ambient source data. The processing module 130 adjusts the volume ratio of the second audio data with different frequency structures according to the ambient noise decibel value at or near the electronic device, for example, in a noisy environment, the volume of the vocal source data is increased and the volume of the ambient source data is decreased; and in a relatively quiet environment, the volumes of the vocal source data and the ambient source data are adjusted to be balanced.

[0115] The existing electronic devices with audio / video playback function generally cannot automatically adjust the volume according to the surrounding environment and the output audio / video, and enhance or reduce the output volume to achieve sound balance in hearing.

[0116] Based on the above technical problems, the processing module 130 in the audio playback control device 100 provided by the present application is further configured to adjust the playback volume of the second audio data after audio mixing according to the ambient noise decibel value.

[0117] The audio playback control device 100 provided by the above embodiments of the present application can realize sound source separation of the played audio data, and adjust the sound emitted by, for example, the target sound source and the ambient sound source according to the ambient noise condition, thereby improving the automatic adjustment capability of the system. The size of the output volume of the electronic device can also be automatically adjusted according to the noise condition (noisy or quiet) of the surrounding environment, which reduces user intervention and improves the intelligent level of the electronic device.

[0118] According to one embodiment of the present application, as shown in Figure 3 The processing module 130 in the audio playback control device 100 comprises:

[0119] The trained machine learning component 131 is configured to separate one or more of human voice sound source data, drum sound source data, bass sound source data from the first audio data.

[0120] The trained machine learning component 131 is coupled with the storage module 110 and is configured to read the first audio data and separate the first audio data. The machine learning component 131 is pre-trained by a large amount of audio data to obtain model parameters, so as to have certain "cognition" for sound source data of different frequency structures (different sound sources or different timbres) and to be able to identify one or more sound source data from a mixture of a plurality of sound source data.

[0121] According to one embodiment of the present application, the machine learning component 131 is configured to:

[0122] First, the first audio data being played or about to be played is sampled, for example, the waveform of the first few seconds of the first audio data is sampled, so that the machine learning model is adapted to the prominent frequencies in the first audio data, including human voice frequency, drum sound frequency, bass frequency, etc. The machine learning component 131 generates a separate waveform for each sound element (prominent frequency).

[0123] Second, the frequency structure of the sampling result is analyzed, and a higher proportion of sound source frequency structure is inferred according to the above-mentioned sound elements (prominent frequencies). Different sound sources play the same frequency sound, but the frequency and number of overtones are different, which is the reason why different sound sources can be distinguished (different sound sources have different timbres). Therefore, according to the analyzed frequency structure, different sound sources can be separated.

[0124] Third, a plurality of third audio data is separated according to different frequency structures. The model separates the sound sources according to the analyzed frequency structure of different sound sources to form a plurality of third audio data with different frequency structures.

[0125] Fourth, the plurality of third audio data is waveform shaped to obtain a plurality of second audio data. The waveform corresponding to different sound sources is finely processed to remove burrs to obtain the shaped sound source waveform. Optionally, the machine learning component 131 generates a plurality of audio files corresponding to the storage of a plurality of second audio data.

[0126] According to an embodiment of the present application, in the audio playing control apparatus 100, the machine learning component 131 is configured to separate the input audio data into a plurality of audio data including at least vocal sound source data, drum sound source data, and bass sound source data. Optionally, after filtering out the vocal sound source data, the drum sound source data, and the bass sound source data from the input audio data, the remaining audio data is stored as an audio file as other audio data.

[0127] According to an embodiment of the present application, the machine learning component 131 comprises a machine learning model based on the Demucs algorithm.

[0128] Those skilled in the art can understand that other machine learning models or components suitable for implementing sound source separation can also be used to implement the present application, and the present application does not limit this.

[0129] According to an embodiment of the present application, in the audio playing control apparatus 100, the processing module 130 is further configured to:

[0130] Receive the noise detection signal output by the sensor 120, and determine the environmental noise decibel value according to the noise detection signal.

[0131] Mix the plurality of second audio data at a preset volume ratio according to the environmental noise decibel value.

[0132] The sensor 120 collects sound at a preset position, and in some embodiments, the sensor 120 collects sound at or near the electronic device and converts it into an electrical signal, which is the noise detection signal. The processing module 130 filters the noise detection signal, filters out the electrical signal corresponding to the audio data currently played by the electronic device, and then calculates the environmental decibel value (representing the noise condition at the electronic device) according to the remaining environmental noise signal. The processing module 130 mixes the plurality of second audio data at a preset volume ratio according to the noise condition at the electronic device.

[0133] According to an embodiment of the present application, in the audio playing control apparatus 100, the processing module 130 is further configured to:

[0134] When the environmental noise decibel value is greater than a first threshold value, if the volume of the vocal data is greater than or equal to the volume of the drum data, the volume of the vocal data is increased.

[0135] When the environmental noise decibel value is greater than a first threshold value, if the volume of the vocal data is less than the volume of the drum data, the volume of the vocal data is adjusted to be equal to the volume of the drum data.

[0136] wherein the first threshold value adopts a noise standard value V n When the environmental noise decibel value is greater than the noise standard value V n , it indicates that the electronic device is currently in a noisy environment, and prominent adjustment should be made to the target sound source, for example, to the human voice sound source. The volume of the human voice sound source data and the drum sound source data is judged. If the volume of the human voice sound source data is greater than or equal to the volume of the drum sound source data, the volume of the human voice sound source data is increased to highlight the sound emitted by the human voice sound source. If the volume of the human voice sound source data is less than the volume of the drum sound source data, the volume of the human voice sound source data is adjusted to be equal to the volume of the drum sound source data, that is, the original volume ratio of the separated human voice sound source data and drum sound source data is not changed much, and the sound emitted by the human voice sound source is highlighted, so that the user can still identify the target human voice in a noisy environment.

[0137] According to one embodiment of the present application, in the audio playing control device 100, the processing module 130 is further configured to:

[0138] When the environmental noise decibel value is less than or equal to the first threshold value, if the environmental noise decibel value is less than the second threshold value, the volumes of the human voice sound source data, the drum sound source data, the bass sound source data and the other audio data are adjusted to be equal.

[0139] When the environmental noise decibel value is less than or equal to the first threshold value, if the environmental noise decibel value is greater than or equal to the second threshold value, the human voice sound source data, the drum sound source data, the bass sound source data and the other audio data are mixed at the original volume ratio.

[0140] wherein the first threshold value adopts a noise standard value V n When the environmental noise decibel value is less than or equal to the noise standard value V n , it indicates that the electronic device is currently in a relatively quiet external environment, and at this time, the external environment is divided into two levels:

[0141] If the environmental noise decibel value is less than the second threshold value (for example, the second threshold value adopts half of the noise standard value V nIf / 2), it indicates that the electronic device is currently in an extremely quiet external environment. In this case, adjust the volumes of the human voice source data, the drum sound source data, the bass sound source data, and the other audio data to be equal. When the electronic device is in an extremely quiet external environment, and the decibel values ​​of each sound source differ significantly, the sound content is extremely unbalanced, affecting the timbre, color, and emotional quality of the sound, and may also cause a sense of unease for the user. Therefore, when the electronic device is in an extremely quiet external environment, adjusting the volume of each sound source to be equal improves auditory comfort. Optionally, the equalization parameters of the separated sound source data (second audio data) can be adjusted to make the volume of each sound source data equal.

[0142] Optionally, the sensor 120 can also acquire the actual decibel value of the currently playing audio data, and then set an appropriate volume based on the actual decibel value to adjust the volume of each sound source, thereby enhancing the effect of sound sources with lower decibel values ​​and balancing the sound.

[0143] If the ambient noise decibel value is greater than or equal to the second threshold (the second threshold is, for example, half of the noise standard value V), n / 2) indicates that the electronic device is neither in a noisy environment nor in an extremely quiet external environment, so the volume ratio of the human voice source data, the drum sound source data, the bass sound source data, and the other audio data is not adjusted. The human voice source data, the drum sound source data, the bass sound source data, and the other audio data are mixed at their original volume ratios.

[0144] The audio playback control device 100 provided in one or more embodiments of the present invention, based on the separation of audio data sources, adjusts the volume of different sound sources, so that in noisy environments, sounds containing substantial content such as human voices are highlighted, and in quiet environments, various sound sources are balanced, giving people a sense of security and improving the listening experience.

[0145] According to one embodiment of the present invention, such as Figure 4 As shown, in the audio playback control device 100, the processing module 130 is further configured to:

[0146] When the ambient noise decibel value is greater than the third threshold, the first decibel value is calculated, and the second decibel value is equal to the ambient noise decibel value plus the first enhancement decibel value.

[0147] When the ambient noise decibel value is less than or equal to the third threshold, a second decibel value is calculated, which is equal to the ambient noise decibel value plus the second enhanced decibel value.

[0148] Wherein, the first enhanced decibel value is greater than the second enhanced decibel value.

[0149] According to the above embodiment of the present application, the processing module 130 is further configured to:

[0150] Based on the volume decibel corresponding table, the volume value corresponding to the first decibel value or the second decibel value is searched, and the volume of the currently played audio data is adjusted.

[0151] In the initial stage of the operation of the electronic device, a corresponding table of volume values and decibel values of different operating systems corresponding to the current electronic device, i.e., a volume decibel corresponding table, is obtained. For devices that do not open the volume decibel corresponding table, the corresponding relationship between the volume and the decibel value is tested to obtain the volume decibel corresponding table, and the device model and the volume decibel corresponding table are entered into a cloud server. When the electronic device is turned on, the device model is obtained, the cloud server queries the volume decibel corresponding table corresponding to the model and caches the query result to a local data table.

[0152] When the electronic device plays sound, the ambient environment decibel value S2 is obtained, S2 is compared with the noise standard value V n , and when S2 is greater than or equal to V n , the decibel value of the playing volume of the electronic device is enhanced, the playing volume decibel value of the electronic device is adjusted to S=S2+T1; when S2 is less than V n , the decibel value of the playing volume of the electronic device is slightly enhanced or weakened, the playing volume decibel value of the electronic device is adjusted to S=S2+T2, and when s2+T2 is less than the original playing sound decibel value of the device, the sound decibel value is weakened. Alternatively, according to the value of s, the volume decibel corresponding table is searched to obtain the corresponding volume value v3, and the output volume of the current system of the electronic device is calibrated to v3. Moreover, after switching the system, the volume decibel corresponding table is still used to query the volume value V4 corresponding to s in the system after the switching, and the output volume value of the corresponding system is set to V4. After the adjustment, the playing volume decibel value of the electronic device is always greater than the ambient environment noise decibel value.

[0153] According to one embodiment of the present application, the noise standard value V n is: the daytime equivalent noise value of the residential area is 55 decibels, and the nighttime equivalent noise value is 45 decibels; the daytime equivalent noise value of the mixed industrial and commercial area is 60 decibels, and the nighttime equivalent noise value is 55 decibels.

[0154] T1 is an enhancement value when the ambient environment noise is greater than the noise standard value, and the enhancement value is large.

[0155] T2 is an enhancement value when the ambient environment noise is less than the noise standard value, and the enhancement value is small. That is

[0156] T1>T2

[0157] The audio playing control device 100 provided by the one or more embodiments of the present application can adaptively adjust the system volume according to the environment background, so as to avoid that the playing sound is too small in a noisy environment, such as a subway and a shopping mall, and the user cannot obtain effective content, or the playing sound is too large in a quiet environment, such as a relatively quiet environment in a family, and the device sound output is too large, which may disturb the people around or cause burden or impact to the user's hearing.

[0158] On a multi-system fusion electronic device, it is usually difficult to realize playing in a unified playing sound size (playing sound decibel value at the device), and usually the current volume value of the system is used as the output value. The volume adjustment is also the adjustment of the current volume value of the system. After switching the system, the current volume value of the switched system is used for volume output. This causes the playing sound to be large or small when switching the system, which affects the user's hearing experience, and in the case of using a headphone or other playing component, it may also cause hearing loss.

[0159] Based on the above technical problems, the present application provides an audio playing control device, which can realize multi-system volume adjustment linkage, unified adjustment entrance and simplified volume adjustment component.

[0160] According to one embodiment of the present application, as shown in Figure 5 In the audio playing control device 100 as introduced in the above embodiment, the processing module 130 is configured to:

[0161] read the volume value of the current system in real time.

[0162] When receiving a system switching instruction, adjust the volume value of the system after switching, so that the first decibel value of the currently playing audio data remains unchanged.

[0163] According to one embodiment of the present application, in the audio playing control device 100, the processing module 130 is further configured to:

[0164] adjust the volume value of the system after switching according to the volume decibel corresponding table.

[0165] Or, adjust the volume value of the system after switching according to the corresponding relationship between the volume values of the systems before and after switching and the maximum volume value of the system.

[0166] After the electronic device is powered on, a current system volume decibel value is used for volume output, when the electronic device is switched, a first volume value V1 of a current first system is acquired, a corresponding decibel value S1 is searched based on a volume decibel correspondence table, the second volume value V2 of the second system after switching is determined in the volume decibel correspondence table according to the decibel value, and the second system sets the output volume value as V2. If the device does not support the volume decibel correspondence table search temporarily, when the system is switched, the first volume value V1 of the current first system is acquired, and the following is calculated:

[0167] P1 = V1 / V max1

[0168] After switching to the second system, the output volume value of the second system is set as:

[0169] V2 = P1*V max2

[0170] The input is uniformly adjusted, when the user adjusts the volume, the volume value TV1 of the current system (for example, the first system) is adjusted, and the corresponding decibel S2 is searched according to the correspondence table. According to the decibel value S2, the volume value TV2 of the corresponding system is inquired in the volume decibel correspondence table under other systems (for example, under the second system and the third system), and the volume output value is set as TV2 in the system, so that the output decibel values of all systems are always guaranteed to be consistent. If the system does not support the volume decibel correspondence table search, TV2 is acquired according to the above-mentioned equal proportion calculation mode.

[0171] The audio playing control device provided by the one or more embodiments of the present application realizes the synchronous adjustment of the multi-system volume, avoids that the playing sound is too large or too small after the system is switched, makes the system volume change smooth, the playing sound is always consistent as a whole, and the user experience is improved.

[0172] It is easy for those skilled in the art to understand that the control order of the processing module 130 in the audio playing control device 100 in the one or more embodiments of the present application is not limited, the separated sound source-sound source volume adjustment, the volume adjustment according to the environmental noise and the volume adjustment of the switching system can be performed simultaneously, or the control program can be executed sequentially. These changed embodiments are within the protection scope of the present application.

[0173] According to one embodiment of the present application, as Figure 6 shown, the present application further provides an audio playing control device 200, which comprises a storage module 210, a sensor 220 and a processing module 230. Wherein:

[0174] The storage module 210 is coupled with a machine learning component in the cloud, and is configured to receive and store the plurality of second audio data separated based on different frequency structures. In some embodiments, the storage module 210 receives first audio data that is being played or is about to be played by the electronic device, uploads the first audio data to the machine learning component in the cloud, which is configured to separate the first audio data based on different frequency structures, and the storage module 210 downloads and stores the plurality of second audio data separated based on different frequency structures.

[0175] The sensor 220 is configured to detect an environmental noise decibel value at a preset location. In some embodiments, the sensor 120, for example, includes a microphone, is disposed at the electronic device (the preset location) or within a preset distance range of the electronic device. The sensor 120 collects sounds around the electronic device and converts the sounds into an electrical signal. After filtering out an electrical signal corresponding to audio data being played by the electronic device, the electrical signal is an environmental noise signal. Based on the environmental noise signal, an environmental noise decibel value at the preset location can be calculated.

[0176] The processing module 230 is coupled with the storage modules 210 and 220, respectively, and is configured to mix the plurality of second audio data at a preset volume ratio based on the environmental noise decibel value. The processing module 130 adjusts the volume ratio of the second audio data with different frequency structures based on the environmental noise decibel value at the preset location (in some embodiments, at or near the electronic device), for example, in a noisy environment, the volume of the vocal sound source data is increased, and the volume of the environmental sound source data is decreased; and in a relatively quiet environment, the volumes of the vocal sound source data and the environmental sound source data are adjusted to be balanced.

[0177] According to one embodiment of the present application, as shown in Figure 7 The present application also provides an electronic device 300, which includes the audio playing control device 100 as described in one or more embodiments above, i.e., the storage module 110 and the processing module 130 are integrated in the electronic device 300, the processing module 130 runs on the electronic device 300, and the preset location includes a location where the electronic device 300 is located, i.e., the sensor 120 detects an environmental noise decibel value at the location where the electronic device 300 is located. The electronic device 300 further includes a playing module 310. Wherein:

[0178] The playing module 310 is coupled with the storage module 110, and is configured to play the mixed plurality of second audio data.

[0179] The electronic device 300 provided by the above embodiments of the present application can automatically adjust the volume according to the environmental noise by real-time processing of the audio data by the audio playing control device 100, highlight the target sound source in a noisy environment, and balance various sound sources in a quiet environment, thereby improving the automatic adjustment capability of the electronic device as an audio and video playing device and providing a user with a better auditory experience.

[0180] According to one embodiment of the present application, as shown in Figure 8 The present application also provides an audio playing control device 400, which comprises an audio data acquisition module 410, an environmental noise acquisition module 420, a sound source separation module 430, a first volume adjustment module 440, an audio mixing module 450 and a second volume adjustment module 460.

[0181] The audio data acquisition module 410 is configured to acquire first audio data.

[0182] The environmental noise acquisition module 420 is configured to acquire an environmental noise decibel value.

[0183] The sound source separation module 430 is configured to perform sound source separation on the first audio data to obtain a plurality of second audio data with different frequency structures.

[0184] The first volume adjustment module 440 is configured to adjust the volume of the plurality of second audio data according to the environmental noise decibel value.

[0185] The audio mixing module 450 is configured to mix the second audio data after volume adjustment.

[0186] The second volume adjustment module 460 is configured to adjust the playing volume of the second audio data after audio mixing according to the environmental noise decibel value.

[0187] The specific limitations in the audio playing control device 400 provided by the above embodiments of the present application are similar to those in the audio playing control method 10 introduced in one or more embodiments above, and can refer to the description of the audio playing control method 10 above, which will not be described here again.

[0188] The audio playing control method and the audio playing control device provided by one or more embodiments of the present application can first classify sound sources of a sound file when an electronic device plays sound, classify the sound into, for example, a target sound source and an environmental sound source, then acquire a noisy condition of sound around the electronic device, calculate an enhancement or weakening value of the classified sound source through a noisy value, and perform enhancement or weakening processing on the target sound source and the environmental sound source according to the calculated value. The classified sound source is enhanced or weakened, the bass of the sound resource is improved, the treble is balanced, and the sound source is amplified or weakened according to the environment, so that the accuracy and comfort of the user's hearing are improved.

[0189] The present application also provides an electronic device, comprising:

[0190] a processor; and

[0191] a memory storing a computer program, when the computer program is executed by the processor, the processor executes the method as described in any one of the above embodiments.

[0192] The above describes the embodiments of the present application in detail, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, the changes or deformations made by the person skilled in the art according to the idea of the present application, based on the specific implementation manners and application scope of the present application, all belong to the scope of protection of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. An audio playback control method, characterized in that, include: Obtain the first audio data; Obtain the ambient noise level in decibels; The first audio data is subjected to sound source separation to obtain multiple second audio data with different frequency structures; Adjusting the volume of multiple second audio data based on the ambient noise decibel value includes adjusting the volume ratio of second audio data with different frequency structures based on the ambient noise decibel value. Mix the second audio data after volume adjustment; Adjust the playback volume of the second audio data after audio mixing according to the ambient noise decibel value; The second audio data includes: One or more of the following: human voice source data, drum sound source data, bass sound source data, and other audio data; The step of adjusting the volume of multiple second audio data based on the ambient noise decibel value includes: When the ambient noise decibel value is greater than the first threshold, if the volume of the human voice source data is greater than or equal to the volume of the drum sound source data, then the volume of the human voice source data is increased. When the ambient noise decibel value is greater than the first threshold, if the volume of the human voice source data is less than the volume of the drum sound source data, then the volume of the human voice source data is adjusted to be equal to the volume of the drum sound source data. When the ambient noise decibel value is less than or equal to the first threshold, if the ambient noise decibel value is less than the second threshold, then the volume of the human voice source data, the drum sound source data, the bass sound source data, and the other audio data are adjusted to be equal. When the ambient noise decibel value is less than or equal to the first threshold, if the ambient noise decibel value is greater than or equal to the second threshold, the original volume of the human voice source data, the drum sound source data, the bass sound source data, and the other audio data is maintained.

2. The method as described in claim 1, wherein, The step of separating the sound source from the first audio data to obtain multiple second audio data with different frequency structures further includes: The first audio data is sampled; Analyze the frequency structure of the sampling results; Multiple third audio data were separated based on different frequency structures; Waveform shaping is performed on the plurality of third audio data to obtain the plurality of second audio data.

3. The method according to any one of claims 1-2, wherein, The step of adjusting the playback volume of the second audio data after audio mixing based on the ambient noise decibel value includes: When the ambient noise decibel value is greater than the third threshold, a first decibel value is calculated, and the first decibel value is equal to the ambient noise decibel value plus the first enhancement decibel value; When the ambient noise decibel value is less than or equal to the third threshold, a second decibel value is calculated, which is equal to the ambient noise decibel value plus a second enhanced decibel value; wherein, the first enhanced decibel value is greater than the second enhanced decibel value; Based on the volume decibel correspondence table, find the volume value corresponding to the first decibel value or the second decibel value, and adjust the playback volume of the currently playing audio data based on the search result.

4. The method according to any one of claims 1-2, further comprising: Real-time reading of the current system volume value; When a system switching command is received, the volume of the system after the switch is adjusted so that the decibel value of the currently playing audio data remains unchanged.

5. The method as described in claim 4, wherein adjusting the volume value of the system after the switch when a system switching command is received includes: Adjust the system volume value after switching according to the volume decibel correspondence table; or Adjust the system volume after switching based on the correspondence between the system volume before and after switching and the system's maximum volume.

6. An audio playback control device, characterized in that, include: The audio data acquisition module is configured to acquire the first audio data. An environmental noise acquisition module is configured to acquire environmental noise decibel values. The sound source separation module is configured to perform sound source separation on the first audio data to obtain multiple second audio data with different frequency structures; The first volume adjustment module is configured to adjust the volume of a plurality of second audio data according to the ambient noise decibel value, including adjusting the volume ratio of the second audio data with different frequency structures according to the ambient noise decibel value. The audio mixing module is configured to mix the second audio data after volume adjustment. The second volume adjustment module is configured to adjust the playback volume of the second audio data after audio mixing according to the ambient noise decibel value. The second audio data includes: One or more of the following: human voice source data, drum sound source data, bass sound source data, and other audio data; The step of adjusting the volume of multiple second audio data based on the ambient noise decibel value includes: When the ambient noise decibel value is greater than the first threshold, if the volume of the human voice source data is greater than or equal to the volume of the drum sound source data, then the volume of the human voice source data is increased. When the ambient noise decibel value is greater than the first threshold, if the volume of the human voice source data is less than the volume of the drum sound source data, then the volume of the human voice source data is adjusted to be equal to the volume of the drum sound source data. When the ambient noise decibel value is less than or equal to the first threshold, if the ambient noise decibel value is less than the second threshold, then the volume of the human voice source data, the drum sound source data, the bass sound source data, and the other audio data are adjusted to be equal. When the ambient noise decibel value is less than or equal to the first threshold, if the ambient noise decibel value is greater than or equal to the second threshold, the original volume of the human voice source data, the drum sound source data, the bass sound source data, and the other audio data is maintained.

7. An electronic device, comprising: processor; as well as A memory storing a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-5.

Images