Audio playing processing method and device, equipment, medium and product

Abstract

The invention provides an audio playing processing method and device, equipment, a medium and a product, the method is applied to an audio system engine, the audio system engine comprises a sound mixing component Audio Flinger of an Android system, and the method comprises the following steps: obtaining a playing state of audio data to be processed, the audio data to be processed are written into an audio cache region of the audio system engine by the application program; according to the playing state of the to-be-processed audio data, fade-in processing or fade-out processing is carried out on the to-be-processed audio data, and an audio processing result is obtained; and carrying out sound mixing according to the audio processing result, and outputting to audio playing equipment for playing. According to the method and the device, a high-quality audio transition effect can be realized on the basis of simplifying a development process, the development process is simplified, cross-platform transplantation can be realized, and the universality is relatively high.

Description

Technical Field

[0001] The present application relates to the field of audio data processing, and in particular to an audio playback processing method, device, equipment, medium and product. Background Art

[0002] As smart devices continue to evolve, audio interaction scenarios are becoming more diverse and complex, including voice assistants, navigation prompts, system notification sounds, phone calls, etc. In these interactions, fade-in or fade-out processing can significantly improve the smoothness of audio playback and user experience.

[0003] At present, the main ways to achieve this function include improving the application level or integrating the fade-in and fade-out function through underlying hardware optimization. However, these methods are relatively complex and have low versatility. Summary of the invention

[0004] Based on the above technical status, the present application provides an audio playback processing method, device, equipment, medium and product, which can achieve high-quality audio transition effects on the basis of simplifying the development process, simplify the development process, and can be transplanted across platforms with high versatility.

[0005] In order to achieve the above technical objectives, this application specifically proposes the following technical solutions:

[0006] According to a first aspect of an embodiment of the present application, there is provided an audio playback processing method, which is applied to an audio system engine, and the method comprises: obtaining the playback status of audio data to be processed, wherein the audio data to be processed is written into an audio buffer area of ​​the audio system engine by an application; fading in or fading out the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result; mixing is performed according to the audio processing result, and outputting the audio to an audio playback device for playback.

[0007] In some implementations, the audio system engine includes an Android system mixing component AudioFlinger and an audio hardware abstraction layer, and the audio data to be processed includes multiple audio data to be processed; wherein, according to the playback status of the audio data to be processed, the audio data to be processed is faded in or faded out to obtain an audio processing result, including: for each audio data to be processed, when the mixing component AudioFlinger identifies that the playback status of the audio data to be processed is a first preset state, the audio data to be processed is faded in to obtain the faded-in audio corresponding to the audio data to be processed, and the first preset state is any one of a start playback state and a paused and continued playback state; mixing according to the audio processing result, and outputting it to the audio playback device for playback through the audio hardware abstraction layer, includes: mixing according to each audio data to be processed and the corresponding faded-in audio through the mixing component AudioFlinger, and outputting it to the audio playback device for playback through the audio hardware abstraction layer.

[0008] In some implementations, the audio system engine includes an Android system mixing component AudioFlinger and an audio hardware abstraction layer, and the audio data to be processed includes multiple audio data to be processed; wherein, according to the playback status of the audio data to be processed, the audio data to be processed is faded in or faded out to obtain an audio processing result, including: for each audio data to be processed, when the mixing component AudioFlinger identifies that the playback status of the audio data to be processed is a second preset state, the audio data to be processed is faded out to obtain a second audio processing result, the second audio processing result includes faded-out audio, and the second preset state is any one of a stopped playback state and a paused playback state; mixing according to the audio processing result and outputting it to an audio playback device for playback through an audio hardware abstraction layer includes: mixing according to each audio data to be processed and the corresponding faded-out audio through the mixing component AudioFlinger, and outputting it to an audio playback device for playback through the audio hardware abstraction layer.

[0009] In some implementations, when the audio mixing component AudioFlinger identifies that the playback state of the audio data to be processed is a second preset state, the audio data to be processed is faded out to obtain a second audio processing result, including: when the audio mixing component AudioFlinger identifies that the playback state of the audio data to be processed is a paused playback state, the played audio data is faded out according to the unplayed audio data in the audio data to be processed to obtain faded-out audio.

[0010] In some implementations, the audio processing result includes fade-in audio or fade-out audio, and the method further includes: when the playback of the fade-in audio or the fade-out audio is started after the playback is interrupted, controlling the volume of the fade-in audio or the fade-out audio to gradually change from the playback volume before the interruption to the target playback volume.

[0011] In some implementations, the method further includes: obtaining first configuration information of a fade-in function and a fade-out function, the first configuration information including turning on the fade-in function and the fade-out function, turning on the fade-in function or turning on the fade-out function; wherein, performing fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result includes: when the first configuration information is to turn on the fade-in function, performing fade-in processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result; when the first configuration information is to turn on the fade-out function, performing fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result; when the first configuration information is to turn on the fade-in function and the fade-out function, performing fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result.

[0012] In some implementations, the method further includes: obtaining second configuration information of a fade-in function and a fade-out function, the second configuration information including a fade-in duration of the fade-in function and / or a fade-out duration of the fade-out function; wherein, fading-in or fading-out the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result includes: fading-in the audio data to be processed according to the playback status of the audio data to be processed and the fade-in duration to obtain an audio processing result; and / or fading-out the audio data to be processed according to the playback status of the audio data to be processed and the fade-out duration to obtain an audio processing result.

[0013] According to the second aspect of the embodiment of the present application, an audio playback processing device is provided, including: an acquisition unit, used to obtain the playback status of audio data to be processed, and the audio data to be processed is written into the audio buffer area of ​​the audio system engine by the application; a fade-in and fade-out processing unit, used to fade-in or fade-out the audio data to be processed according to the playback status of the audio data to be processed, to obtain an audio processing result; a mixing unit, used to mix according to the audio processing result, and output it to an audio playback device for playback.

[0014] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor; the memory is connected to the processor and is used to store programs; the processor is used to implement the audio playback processing method described in the first aspect by running the program in the memory.

[0015] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the audio playback processing method as described in the first aspect is implemented.

[0016] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, cause the processor to execute: the audio playback processing method as described in the first aspect.

[0017] The embodiments of the present application provide an audio playback processing method, device, equipment, medium and product. The method is applied to an audio system engine. By obtaining the playback status of the audio data to be processed, the audio data to be processed is written by the application into the audio buffer area of ​​the audio system engine. Then, according to the playback status of the audio data to be processed, the audio data to be processed is faded in or faded out to obtain an audio processing result, and mixed according to the audio processing result, and finally output to the audio playback device through the audio hardware abstraction layer for playback. Since the present application uses the audio system engine of AOSP as the implementation basis, fade-in and fade-out processing can be implemented without relying on a specific hardware platform interface, ensuring cross-platform portability and high versatility; in addition, since the entire processing process is at the operating system level, high-quality audio transition effects can be achieved without the need for the application to modify the code, simplifying the development process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0019] Figure 1 It is a schematic diagram of the framework of an audio system engine in the related art;

[0020] Figure 2 A flowchart of an audio playback processing method provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a user interface for setting a fade-in and fade-out function provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a user interface for setting the fade-in and fade-out duration provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram showing the comparison of the mixing process before and after optimization provided in the embodiment of the present application;

[0024] Figure 6 A schematic diagram showing the comparison of effects before and after adding fade-in and fade-out processing provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of the structure of an audio playback processing device provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solution provided in the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, servers (including cloud servers), or packaged into software programs to be run. When the hardware device executes the processing of the technical solution of the embodiments of the present application, or the above-mentioned software program is run, the automatic splitting of the target task and the automatic calling of the application program interface required for the task can be achieved to complete the purpose of the target task. The embodiments of the present application only exemplarily introduce the specific processing of the technical solution of the present application, and do not limit the specific implementation form of the technical solution of the present application. Any technical implementation form that can execute the processing of the technical solution of the present application can be adopted by the embodiments of the present application.

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] Before introducing this application solution, the relevant technologies are first introduced:

[0030] In audio interaction scenarios, many applications lack smooth fade-in and fade-out effects, which may cause abrupt sounds or even popping sounds when the audio starts playing or is suddenly interrupted. This phenomenon seriously affects the user's listening experience, especially in closed environments such as in a car, in a theater, or when listening through headphones.

[0031] There are currently two main solutions to the above problems. The following introduces the two solutions in combination with the audio framework:

[0032] Figure 1 FIG. 1 is a schematic diagram of a framework of an audio system engine in the related art. Figure 1 As shown, the framework includes an application layer, a Java framework layer, a native framework layer, and an audio hardware abstraction layer (Hardware Abstraction Layer, HAL).

[0033] Among them, the application layer includes various applications used by users, such as music players, video applications, game applications, etc. Open Bores Operating Excursion (OBOE) and Open Source Audio and Video Player (ExoPlayer) are API interfaces provided by the Android system for applications (APPs). APPs can use OBOE or ExoPlayer to implement audio playback functions. OBOE is mainly used for applications that require playback delay, such as voice communication, game sound effects, or navigation prompts. ExoPlayer is used for media players to play audio and video content.

[0034] The Java framework layer is an audio application framework layer, including a first AudioTrack component and a first MediaPlayer component. The first AudioTrack component is an API class of the Android application framework, responsible for outputting playback data.

[0035] The Native framework layer includes the audio mixing component AudioFlinger. AudioFlinger is responsible for the audio mixing process of audio data and includes the second AudioTrack component, the Track component, and the Play thread component.

[0036] Each APP corresponds to a first AudioTrack component. Each first AudioTrack component corresponds to a mixing component AudioFlinger, that is, each first AudioTrack component corresponds to a second AudioTrack component and a Track component. Multiple Track components correspond to a Play thread component.

[0037] When the APP needs to play audio, it calls the AudioTrack component of AOSP. Specifically, the APP can write the audio data to be processed in the audio buffer area applied by AudioFlinger by directly calling the second AudioTrack component or indirectly calling the second AudioTrack component through the first AudioTrack component.

[0038] The Track component periodically takes the audio data to be processed from the audio buffer for mixing, and transmits the mixed audio data to the audio hardware abstraction layer through the Playthread component. The audio hardware abstraction layer provides an interface for interacting with the underlying hardware to ensure that the audio data can be correctly transmitted to the audio device for playback.

[0039] In addition, for media playback apps, since the corresponding first MediaPlayer component can play audio data in multiple formats, such as Mp3, flac, wma, ogg, wav, etc., while the AudioTrack component can only play decoded PCM data streams, the first MediaPlayer component needs to create the corresponding second MediaPlayer component, NuPlayer component, MediaExtractor component, MediaCodec component, and OMX component in the Native framework layer to decode the audio and video data to obtain audio data, and then hand it over to AudioFlinger for mixing.

[0040] The above process is the current processing process for audio playback. Based on the above audio playback processing process, one of the solutions for fade-in and fade-out processing is to make improvements at the application layer. Specifically, each application uses the Android system's VolumeShaper API or self-developed algorithms to adjust the volume of the player to achieve a natural audio transition effect. This method can directly improve the user experience of a specific application, but it will increase the complexity of software development, and for most applications, it is challenging to push them to make corresponding changes. In addition, this method requires each application to implement the fade-in and fade-out function separately, and it is difficult to ensure that all applications can achieve consistent user experience standards.

[0041] Another solution is to process audio streams through underlying hardware optimization. For example, a specially designed algorithm is integrated into the digital signal processor (DSP) or power amplifier in the audio hardware abstraction layer to uniformly process all audio streams and ensure that each sound event has a fade-in and fade-out effect. Although this method can provide wider compatibility and better performance, it helps to maintain audio consistency. However, its implementation depends on the support of device hardware, and not all devices currently have this capability. In addition, its control accuracy of a single audio track is limited, which may not meet the needs of some advanced application scenarios.

[0042] In view of this, the embodiments of the present application are dedicated to providing an audio playback processing method, device, equipment, medium and product. By modifying the AudioTrack component of the Android Open Source Project (AOSP), the mixing-related logic of AudioFlinger can be adjusted, and fade-in and fade-out processing can be performed separately when each audio data to be processed is played, without the need for the application to cooperate with the modification code, simplifying the implementation process of the fade-in and fade-out function, and can be ported to various Android versions of the system, with high versatility. In addition, since its processing process is in the AudioFlinger mixing stage, close to the hardware layer, it can achieve a low-latency effect, further improve the fade-in and fade-out effect, and enhance the overall fluency of the audio playback. Detailed description is given one by one in the following embodiments.

[0043] Exemplary Methods

[0044] Figure 2 Flow chart of an audio playback processing method provided in an embodiment of the present application. Figure 2 As shown, the audio playback processing method provided in this embodiment includes steps S101 to S103:

[0045] S101, obtaining the playing status of the audio data to be processed, where the audio data to be processed is written into the audio buffer area of ​​the audio system engine by the application.

[0046] The execution subject of the method of this embodiment is the audio system engine, specifically the mixing component AudioFlinger in the Android system.

[0047] The applications in step S101 include audio applications (such as music players, podcast platforms, voice memos, etc.), audio and video applications (such as online video streaming services, local media players, live broadcast platforms, etc.), communication applications (such as VoIP call software or voice and video call functions in instant messaging tools) or game applications (background music, character dialogues, environmental sound effects, etc. in games) and other applications that need to play sound.

[0048] Continue reading Figure 1 When the application is ready to play audio, it calls the corresponding API interface (such as Figure 1 The first AudioTrack component in the application program) requests a playback channel from the audio mixer component AudioFlinger. Afterwards, after the audio mixer component AudioFlinger receives the audio data to be processed from the application program, it stores it in an internal audio buffer for further processing.

[0049] During this process, the Track component in AudioFlinger is responsible for managing the playback status of the audio data to be processed, which includes but is not limited to different situations such as starting to play, playing, pausing or ending playback.

[0050] When the user triggers a playback control command (such as start, pause, or stop), the playback status corresponding to these operations will be passed to the audio mixer component AudioFlinger. Based on the received playback status, the audio mixer component AudioFlinger can accurately grasp the playback status of the current audio data to be processed and make appropriate responses accordingly. This ensures that when multiple applications play audio at the same time, all audio outputs are managed in an orderly and efficient manner, improving the user experience.

[0051] S102: Perform fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed, to obtain an audio processing result.

[0052] Fading in or out can smoothly change the loudness of a sound over time to achieve a more natural audio effect.

[0053] Fade-in refers to gradually increasing the volume of an audio signal, starting from silence or a very low volume and rising smoothly to the desired normal playback volume over a certain period of time. Fade-in processing can make the introduction of audio softer and less abrupt.

[0054] Fade-out refers to gradually reducing the volume of an audio signal, from the normal playback volume to silence or a very low volume over a period of time, and finally stopping. In contrast to fade-in, fade-out can convey a feeling of letting the audio fade away, avoiding an abrupt end. When playing multiple audio clips continuously, fade-out can make the current audio smoothly transition to the next audio to ensure overall continuity.

[0055] When the mixing component AudioFlinger identifies that the playback state of the audio data to be processed is a preset state, the audio data to be processed is faded in or faded out to improve the audio playback effect. Specifically, when the mixing component AudioFlinger implements step S102, at least one of the following implementation methods can be used:

[0056] In some implementations, step S102 includes:

[0057] Step a1: when it is identified that the playing state of the audio data to be processed is the first preset state, fade-in processing is performed on the audio data to obtain fade-in audio corresponding to the audio data to be processed, and the first preset state is any one of the start playing state and the continue playing state after pausing.

[0058] When an audio file starts playing for the first time, its playback state is "start playing state". In order to avoid the abrupt feeling caused by the sudden appearance of audio, a fade-in effect can be added to the audio data to be processed. For example, the data of the first n seconds (such as 3 seconds) can be intercepted at the beginning of the audio data to be processed, and the volume of the n seconds of data can be gradually increased using an audio processing algorithm to generate fade-in audio, so as to achieve a change from complete silence to the target playback volume. The target playback volume is the playback volume of the audio stream after the first n seconds, so as to ensure that the audio after the fade-in processing matches the volume of the subsequent audio stream, providing a smooth listening experience.

[0059] For example, when a user opens a music player and selects a song, after pressing the play button, the music does not immediately start at full volume, but gradually rises from a silent or very low volume to the normal playback volume, making the introduction of audio more natural and providing users with an auditory transition.

[0060] When the audio playback is paused and resumed, its playback state is "continued playback state after pause". In order to make the music continuous and unobtrusive before and after the pause, the present embodiment may also add a fade-in effect when the audio is replayed. Specifically, the first n seconds of data may be intercepted at the beginning of the restarted audio data, and the volume of the n seconds of data may be gradually increased using an audio processing algorithm to generate fade-in audio, so as to achieve a change from complete silence after the pause to the target playback volume, thereby ensuring that the audio after the fade-in processing matches the volume of the subsequent audio stream, providing a smooth listening experience.

[0061] For example, a movie is currently playing, and the movie is paused because of a phone ring. After hanging up the phone, the user returns to the video player and clicks the play button. At this time, the audio mixing component AudioFlinger recognizes that the current state is "continue playing after pause" and starts the fade-in process, so that the dialogue or background music in the movie gradually returns to the normal level from a lower volume.

[0062] Whether it is the first playback or the resumption of playback after a pause, fading in the audio data to be processed can provide users with a smoother and more comfortable listening experience.

[0063] In some implementations, step S102 includes:

[0064] Step b1: when it is identified that the playback state of the audio data to be processed is the second preset state, fade-out processing is performed on the audio data to be processed to obtain a second audio processing result, wherein the second audio processing result includes fade-out audio, and the second preset state is any one of a stop playback state and a pause playback state.

[0065] When the application or user decides to completely terminate the audio playback, the playback state changes to the "stop playback state". At this time, in order to avoid the abruptness caused by the sudden interruption of the audio, the present embodiment can apply a fade-out effect. For example, in a music player, if the user chooses to switch songs or turn off the player, the currently playing audio will not disappear immediately, but will gradually decrease from the normal volume until it is completely silent. The gradual reduction in volume makes the end of the audio appear smoother, thereby improving the comfort of the audience. In the stop playback state, the audio data n seconds after the end of the audio data to be processed can be used for fade-out processing.

[0066] When the user temporarily interrupts the audio playback, the playback state changes to "pause playback state". For example, when watching a movie or video, if the user needs to temporarily leave to answer a phone call or handle other things, they can choose to pause the playback. At this time, the audio mixer will fade out the audio, so that the background music or dialogue sound gradually weakens instead of being cut off immediately, thereby maintaining the continuity of the scene.

[0067] When implementing step b1, if the mixing component AudioFlinger identifies that the play state of the audio data to be processed is a paused play state, the played audio data is faded out according to the unplayed audio data in the audio data to be processed to obtain faded-out audio.

[0068] In order to avoid stopping the audio output immediately when pausing, the played part (i.e., the played audio data) can be faded out according to the audio content that has not been played (i.e., the unplayed audio data), and the volume of the currently played audio can be gradually reduced, so that the normal playback volume can be smoothly transitioned to complete silence, so that the end of the audio stream will not appear abrupt and will not affect the overall listening experience. Even in the paused state, the audio can maintain its coherence and integrity, thereby providing users with a more natural and comfortable end-of-listening effect.

[0069] In some embodiments, in order to further improve the user experience, when the user triggers a pause operation, the fade-out strategy can be determined by evaluating the content features of the audio data portion that has not been played in the audio data.

[0070] Specifically, audio content features may include music structure, dialogue rhythm, or other key audio elements. Audio content features can be used to determine how the currently playing audio ends in order to maintain the coherence and integrity of the overall audio content. For example, if the audio content feature is a first content feature (such as the unplayed data contains an important conversation or music climax), the first fade-out duration can be selected to ensure that this part of the content will not be suddenly interrupted; conversely, if the audio content feature is a second content feature (such as the unplayed audio is relatively bland), the second fade-out duration can be selected to quickly respond to the user's pause request. The first fade-out duration is greater than the second fade-out duration. This customized processing method ensures that no matter how the audio content changes, the fade-out effect can adapt well to the audio features and enhance the user's listening experience.

[0071] In some embodiments, in order to ensure that the audio fade-in process or the audio fade-out process can be seamlessly connected after being interrupted, the audio playback processing method of this embodiment may also include: when the playback is started after the fade-in audio or fade-out audio playback is interrupted, controlling the volume of the fade-in audio or fade-out audio to gradually change from the playback volume before the interruption to the target playback volume.

[0072] When the playback of the fade-in audio or fade-out audio is interrupted and then restarted, the volume can be controlled to gradually and smoothly transition to the target playback volume starting from the playback volume before the interruption.

[0073] Specifically, when an interruption is detected during the fade-in or fade-out process (such as network delay or other external factors), the current volume value and time point will be recorded; when the audio playback is restarted, the volume will be gradually adjusted from the recorded interruption volume value until the target playback volume is reached, thereby avoiding discomfort to the audience caused by sudden sound changes.

[0074] Whether fading in or fading out, when an interruption occurs, the fade-in will be restarted based on the current volume value. In this way, even if there is a short interruption during playback, the user will not feel an obvious volume change or jump, thus maintaining the smoothness and consistency of the audio experience.

[0075] For example, suppose a user is listening to a song. When the song starts playing and fades in, the playback is interrupted by an incoming call. You need to record the volume value and fade-in progress at the interruption time. When the user answers the call and returns to the app to continue playing, the volume is gradually increased from the recorded volume value until it is fully restored to the normal playback volume.

[0076] Through the above mechanism, the audio playback processing method of this embodiment can provide a seamless and natural audio recovery effect when an interruption occurs during the fade-in or fade-out process.

[0077] In some embodiments, in order to provide a personalized user experience, configuration options for fade-in and fade-out processing can also be set for each application, allowing users to customize audio effects according to their own needs. The following is an introduction to this configuration function in conjunction with the accompanying drawings:

[0078] Figure 3 This is a schematic diagram of the fade-in and fade-out function settings provided in an embodiment of the present application. Figure 3 As shown in the figure, the user can choose to turn on or off the fade-in function, the fade-out function, or turn on both at the same time. When the black dot is on the left side of the selection box, it means it is turned on; when the black dot is on the right side of the selection box, it means it is turned off. For example, APP1 turns on the fade-in function and turns off the fade-out function; APP2 turns off the fade-in function and turns on the fade-out function; APP3 turns on both the fade-in function and the fade-out function.

[0079] Based on the above configuration information, before executing step S102 of the audio playback processing method of this embodiment, it is necessary to obtain first configuration information (i.e., the selection of whether to enable the fade-in and / or fade-out function), and perform corresponding processing according to different types of first configuration information, which may specifically include the following implementation methods:

[0080] In some implementations, if the first configuration information is identified as turning on the fade-in function, the audio data to be processed is faded in according to the playback state of the audio data to be processed to obtain an audio processing result. Specifically, when the playback state of the audio data to be processed is the first preset state, the audio data to be processed is faded in to obtain an audio processing result. For example, if the user chooses to turn on only the fade-in function, the fade-in effect is automatically applied when the audio starts playing, thereby achieving a smooth volume increase effect when the audio starts playing.

[0081] In some implementations, if the first configuration information is identified as turning on the fade-out function, the audio data to be processed is faded out according to the playback state of the audio data to be processed to obtain an audio processing result. Specifically, when the playback state of the audio data to be processed is the second preset state, the audio data to be processed is faded out to obtain an audio processing result. For example, if the user chooses to turn on only the fade-out function, the fade-out effect is automatically applied when the audio ends or pauses, thereby achieving a smooth volume reduction effect when the audio ends or pauses.

[0082] In some implementations, if it is identified that the first configuration information is to turn on the fade-in function and the fade-out function at the same time, the audio data to be processed is faded in or faded out according to the playback state of the audio data to be processed to obtain an audio processing result. Specifically, when the playback state of the audio data to be processed is the first preset state, the audio data to be processed is faded in to obtain an audio processing result; when the playback state of the audio data to be processed is the second preset state, the audio data to be processed is faded out to obtain an audio processing result. For example, if the user chooses to turn on the two functions at the same time, a fade-in effect is applied when the audio starts playing, and a fade-out effect is applied when the audio ends or pauses.

[0083] In order to further enhance the personalized experience, in some embodiments, the user may be provided with the option to set the fade-in duration and fade-out duration of each application. Figure 4 As shown, users can set only the fade-in duration, only the fade-out duration, or both the fade-in duration and the fade-out duration for each application according to their personal preferences by selecting the "minus (-)" and "plus (+)" in the selection box, or directly entering a number. The fade-in duration refers to the duration of the fade-in process. The fade-out duration refers to the duration of the fade-out process.

[0084] Based on the configuration information of the fade-in duration and the fade-out duration, before executing step S102 of the audio playback processing method of this embodiment, it is necessary to obtain second configuration information and perform corresponding processing according to different types of second configuration information, which may specifically include the following implementation methods:

[0085] In some implementations, if it is recognized that the second configuration information includes a fade-in duration, the audio data to be processed is faded-in according to the playback state of the audio data to be processed and the set fade-in duration, to obtain an audio processing result. Specifically, when the playback state of the audio data to be processed is the first preset state, the audio data to be processed is faded-in according to the set fade-in duration, to obtain an audio processing result. For example, if the user chooses to set only the fade-in duration, the set fade-in duration is automatically applied when the audio starts playing.

[0086] In some implementations, if it is recognized that the second configuration information includes a fade-out duration, the audio data to be processed is faded out according to the playback state of the audio data to be processed and the set fade-out duration, to obtain an audio processing result. Specifically, when the playback state of the audio data to be processed is the second preset state, the audio data to be processed is faded out according to the set fade-out duration, to obtain an audio processing result. For example, if the user chooses to set only the fade-out duration, the set fade-out duration is automatically applied when the audio starts playing.

[0087] In some implementations, if it is identified that the second configuration information includes a fade-in duration and a fade-out duration, the audio data to be processed is faded in according to the playback state of the audio data to be processed and the set fade-in duration to obtain an audio processing result, or the audio data to be processed is faded out according to the playback state of the audio data to be processed and the set fade-out duration to obtain an audio processing result. Specifically, when the playback state of the audio data to be processed is a first preset state, the audio data to be processed is faded in according to the set fade-in duration to obtain an audio processing result, and when the playback state of the audio data to be processed is a second preset state, the audio data to be processed is faded out according to the set fade-out duration to obtain an audio processing result. For example, if the user chooses to turn on both functions at the same time, the fade-in duration is applied when the audio starts playing, and the fade-out duration is applied when the audio ends or pauses.

[0088] It should be noted that when the user chooses to set only the fade-in duration, when the playback state of the audio data to be processed is the second preset state, the audio data to be processed is faded-in according to the default fade-in duration to obtain the audio processing result. When the user chooses to set only the fade-out duration, when the playback state of the audio data to be processed is the first preset state, the audio data to be processed is faded-in according to the default fade-in duration to obtain the audio processing result.

[0089] The above embodiments introduce fading in or fading out of a single audio data. However, in actual application scenarios, multiple audio data often exist at the same time and need to be processed synchronously. In order to ensure that the fade-in or fade-out between the various audio data to be processed does not affect each other, the mixing component applies for an independent audio buffer for each audio data, so that each audio stream has its own independent buffer.

[0090] After the audio mixing component obtains the audio data to be processed and the corresponding playback status from each audio buffer, it can perform corresponding fade-in or fade-out processing on the audio data that meets the preset status according to the obtained playback status. For example, the audio data that meets the first preset status is faded in to generate a smooth audio introduction effect; and the audio data that meets the second preset status is faded out to ensure that the audio stream can end naturally.

[0091] In order to ensure that the processing of each audio stream is independent of each other, each audio stream also corresponds to an independent processing path. For example, the music playback application and the notification prompt tone each have an independent audio buffer area. When the user is playing music, the phone receives a notification prompt tone of a new message. At this time, the mixing component recognizes that the current playback state is "start playback state", so the prompt tone is faded in; at the same time, the background music in the music playback application may be in "pause playback state", so it is faded out. Because the processing of the two audio streams is completely independent, the user hears a clear notification prompt tone, while the music gradually fades to silence, and then can resume playback according to the user's operation.

[0092] Through the above mechanism, the audio mixing component in this embodiment can not only process multiple audio data efficiently, but also ensure that the fade-in or fade-out processing of each audio stream is independent and not affected by other audio streams.

[0093] Continue reading Figure 2 After step S102, the audio playback processing method of this embodiment further includes step S103.

[0094] S103: Mix the audio according to the audio processing result, and output it to an audio playback device for playback.

[0095] After the mixing component has completed the fade-in or fade-out processing of the processed audio data, the mixing operation can be performed based on the fade-in or fade-out audio data (i.e., the audio processing result). Mixing refers to the process of merging multiple audio streams into a single audio stream, which includes adjusting the relative volume and balance of each audio source, and applying audio effects such as equalization and echo cancellation to ensure that all audio elements coexist harmoniously and provide a clear and immersive listening experience.

[0096] Specifically, step S103 includes: mixing the audio data to be processed and the corresponding fade-in audio through the mixing component AudioFlinger, and outputting them to the audio playback device for playback through the audio hardware abstraction layer; or, mixing the audio data to be processed and the corresponding fade-out audio through the mixing component AudioFlinger, and outputting them to the audio playback device for playback through the audio hardware abstraction layer.

[0097] The audio mixer component merges the faded-in or fade-out audio data with other playing audio streams to generate a final audio stream containing all necessary audio information, and transmits it to the actual audio playback device (such as the built-in speaker of a mobile phone, Bluetooth headset, etc.) through the HAL layer.

[0098] The fade-in and fade-out function of this embodiment can cover the following usage scenarios: switching audio sources, up and down songs, seekto (moving the audio or video playback position to a specified time point), pausing playback, starting playback, abnormal application crash, etc.

[0099] The beneficial effects of this embodiment are described in detail below in conjunction with the accompanying drawings:

[0100] Figure 5 This is a schematic diagram of the comparison of the mixing process before and after optimization provided in the embodiment of the present application. Figure 5 As shown, the left side is a schematic diagram of the audio processing state before optimization, and the right side is a schematic diagram of the audio processing state after optimization.

[0101] Among them, IDLE represents the idle state, RESUMING represents the resume state, STOPPED represents the stopped state, PAUSED represents the paused state, ACTIVE represents the active state, that is, the playing state, and PAUSING represents the paused state.

[0102] The start operation can be used to switch from the idle state (IDLE) to the resume state (RESUMING); the start operation can be used to switch from the resume state (RESUMING) to the active state (ACTIVE); the pause operation can be used to switch from the active state (ACTIVE) to the paused state (PAUSING); the pause operation can be used to switch from the paused state (PAUSING) to the paused state (PAUSED); the start operation can be used to switch from the paused state (PAUSED) to the active state (ACTIVE); the stop / destroy operation can be used to switch from the active state (ACTIVE) to the stopped state (STOPPED); the stop / destroy operation can be used to switch from the paused state (PAUSING) to the stopped state (STOPPED); the stop / destroy operation can be used to switch from the paused state (PAUSED) to the stopped state (STOPPED); the release of related resources (destroy) can be used to switch from the stopped state (STOPPED) to the idle state (IDLE).

[0103] When the application is playing audio, if the user switches sources or skips tracks, the audio playback state will switch from active to stopped. Also, if the user pauses or seeks to, the audio playback state will switch from active to paused, and then to paused. In these two scenarios, plosive sounds are likely to occur.

[0104] According to the comparison between the left and right sides, this embodiment eliminates the plosive sound by performing a fade-out process when the audio switches from the paused state to the paused state when switching sources, skipping tracks, pausing or seeking to operations by the user.

[0105] Figure 6 This is a schematic diagram comparing the effects before and after adding fade-in and fade-out processing provided in the embodiment of the present application. Figure 6 As shown in the figure, it can be seen that before adding the fade-in process, when encountering the next song or seek to scene, the audio will be directly cut off and end. After adding the fade-in and fade-out process, when encountering the next song or seek to scene, the audio will not be directly cut off, but gradually weakened to smoothly transition to the end.

[0106] The audio playback processing method provided in the embodiment of the present application is applied to the audio system engine, specifically using the mixing component AudioFlinger in the audio system engine as the implementation basis, so that the need to rely on a specific hardware platform interface can be bypassed, ensuring cross-platform portability and high versatility; in addition, since the entire process is implemented at the operating system level, there is no need for each application to modify the code separately to support the fade-in and fade-out function, so that no matter which application the user uses, they can enjoy consistent and high-quality audio transition effects, thereby simplifying the development process and reducing maintenance costs. In addition, the solution can cover a variety of audio playback scenarios, such as music switching, pause / resume, etc., further enhancing the user's auditory experience. And because AudioFlinger is close to the hardware (hzl) layer, a low-latency fade-in and fade-out effect can be achieved.

[0107] Exemplary Devices

[0108] Corresponding to the above-mentioned audio playback processing method, the embodiment of the present application also provides an audio playback processing device. Figure 7 Schematic diagram of an audio playback process provided by an embodiment of the present application. Figure 7 As shown, the audio playback processing device provided in the embodiment of the present application includes: an acquisition unit 701, a fade-in and fade-out processing unit 702 and a mixing unit 703; the acquisition unit 701 is used to obtain the playback status of the audio data to be processed, and the audio data to be processed is written into the audio buffer area of ​​the audio system engine by the application; the fade-in and fade-out processing unit 702 is used to fade-in or fade-out the audio data to be processed according to the playback status of the audio data to be processed, and obtain an audio processing result; the mixing unit 703 is used to mix according to the audio processing result, and output it to the audio playback device for playback.

[0109] In some embodiments, the audio data to be processed includes multiple audio data to be processed; wherein, the fade-in and fade-out processing unit 702 performs fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result, including: for each audio data to be processed, when it is identified that the playback status of the audio data to be processed is a first preset state, fade-in processing is performed on the audio data to be processed to obtain fade-in audio corresponding to the audio data to be processed, and the first preset state is any one of a start playback state and a paused and continued playback state; the mixing unit 703 performs mixing according to the audio processing result, and outputs it to the audio playback device for playback through the audio hardware abstraction layer, including: mixing according to each audio data to be processed and the corresponding fade-in audio, and outputting it to the audio playback device for playback through the audio hardware abstraction layer.

[0110] In some embodiments, the audio data to be processed includes multiple audio data to be processed; wherein, the fade-in and fade-out processing unit 702 performs fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result, including: for each audio data to be processed, when it is identified that the playback status of the audio data to be processed is a second preset state, fade-out processing is performed on the audio data to be processed to obtain a second audio processing result, wherein the second audio processing result includes faded-out audio, and the second preset state is any one of a stopped playback state and a paused playback state; the mixing unit 703 performs mixing according to the audio processing result, and outputs it to the audio playback device for playback through the audio hardware abstraction layer, including: performing mixing according to each audio data to be processed and the corresponding fade-out audio, and outputting it to the audio playback device for playback through the audio hardware abstraction layer.

[0111] In some embodiments, when the fade-in and fade-out processing unit 702 identifies that the playback status of the audio data to be processed is a second preset status, it fades out the audio data to be processed to obtain a second audio processing result, including: when it is identified that the playback status of the audio data to be processed is a paused playback status, it fades out the played audio data according to the unplayed audio data in the audio data to be processed to obtain faded-out audio.

[0112] In some embodiments, the audio processing result includes fade-in audio or fade-out audio, and the fade-in and fade-out processing unit 702 is also used to: when the playback of the fade-in audio or the fade-out audio is interrupted and then started, control the volume of the fade-in audio or the fade-out audio to gradually change from the playback volume before the interruption to the target playback volume.

[0113] In some embodiments, the acquisition unit 701 is also used to: acquire first configuration information of the fade-in function and the fade-out function, the first configuration information includes turning on the fade-in function and the fade-out function, turning on the fade-in function or turning on the fade-out function; wherein the fade-in and fade-out processing unit 702 performs fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result, including: when the first configuration information is to turn on the fade-in function, the audio data to be processed is faded in according to the playback status of the audio data to be processed to obtain an audio processing result; when the first configuration information is to turn on the fade-out function, the audio data to be processed is faded out according to the playback status of the audio data to be processed to obtain an audio processing result; when the first configuration information is to turn on the fade-in and fade-out functions, the audio data to be processed is faded in or faded out according to the playback status of the audio data to be processed to obtain an audio processing result.

[0114] In some embodiments, the acquisition unit 701 is also used to: acquire second configuration information of the fade-in function and the fade-out function, the second configuration information including the fade-in duration of the fade-in function and / or the fade-out duration of the fade-out function; wherein the fade-in and fade-out processing unit 702 performs fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result, including: performing fade-in processing on the audio data to be processed according to the playback status of the audio data to be processed and the fade-in duration to obtain an audio processing result; and / or performing fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed and the fade-out duration to obtain an audio processing result.

[0115] The acquisition unit 701, the fade-in and fade-out processing unit 702 and the mixing unit 703 correspond to the mixing component AudioFlinger in the audio system engine. Specifically, the acquisition unit 701 and the fade-in and fade-out processing unit 702 correspond to the Track component in the mixing component AudioFlinger, and the mixing unit 703 corresponds to the Playthread component in the mixing component AudioFlinger.

[0116] The audio playback processing device provided in this embodiment belongs to the same application concept as the audio playback processing method provided in the above embodiments of this application, and can execute the audio playback processing method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the audio playback processing method. For technical details not fully described in this embodiment, please refer to the specific processing content of the audio playback processing method provided in the above embodiments of this application, which will not be repeated here.

[0117] The functions implemented by the above acquisition unit 701, fade-in and fade-out processing unit 702 and mixing unit 703 may be implemented by the same or different processors respectively, and the embodiment of the present application is not limited thereto.

[0118] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0119] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0120] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0121] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0122] Exemplary Electronic Devices

[0123] The present application embodiment provides an electronic device, see Figure 8 As shown, the electronic device includes:

[0124] Memory 200 and processor 210;

[0125] The memory 200 is connected to the processor 210 and is used to store programs;

[0126] The processor 210 is used to implement the audio playback processing method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0127] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0128] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0129] A bus may include a pathway that transfers information between components of a computer system.

[0130] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0131] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0132] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0133] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0134] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0135] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0136] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any audio playback processing method provided in the above embodiments of the present application.

[0137] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the audio playback processing method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the embodiment introduction of the above-mentioned audio playback processing method.

[0138] Exemplary computer program products and storage media

[0139] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the audio playback processing method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0140] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0141] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the audio playback processing method according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically the steps of the above audio playback processing method can be implemented.

[0142] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0143] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0144] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0145] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0146] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0147] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0148] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0149] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0150] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0151] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0152] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio playback processing method, characterized in that: Applied to an audio system engine, the method comprises: Acquire the playing status of the audio data to be processed, where the audio data to be processed is written into the audio buffer area of ​​the audio system engine by the application; According to the playback status of the audio data to be processed, fade-in processing or fade-out processing is performed on the audio data to be processed to obtain an audio processing result; Mixing is performed according to the audio processing result, and output to an audio playback device for playback.

2. The method according to claim 1, characterized in that The audio system engine includes an audio mixing component AudioFlinger and an audio hardware abstraction layer of the Android system, and the audio data to be processed includes a plurality of audio data to be processed; The step of performing fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result includes: For each audio data to be processed, when the mixing component AudioFlinger identifies that the playback state of the audio data to be processed is a first preset state, the audio data to be processed is faded in to obtain faded-in audio corresponding to the audio data to be processed, wherein the first preset state is any one of a start playback state and a paused and continued playback state; The mixing according to the audio processing result and outputting to the audio playback device through the audio hardware abstraction layer for playback includes: The audio mixing component AudioFlinger performs mixing according to each of the audio data to be processed and the corresponding fade-in audio, and outputs the audio to the audio playback device for playback through the audio hardware abstraction layer.

3. The method according to claim 1, characterized in that The audio system engine includes an audio mixing component AudioFlinger and an audio hardware abstraction layer of the Android system, and the audio data to be processed includes a plurality of audio data to be processed; The step of performing fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result includes: For each audio data to be processed, when the mixing component AudioFlinger identifies that the playback state of the audio data to be processed is a second preset state, the audio data to be processed is faded out to obtain a second audio processing result, wherein the second audio processing result includes faded-out audio, and the second preset state is any one of a stop playback state and a pause playback state; The mixing according to the audio processing result and outputting to the audio playback device through the audio hardware abstraction layer for playback includes: The audio mixing component AudioFlinger performs mixing according to each of the audio data to be processed and the corresponding faded-out audio, and outputs the audio to the audio playback device for playback through the audio hardware abstraction layer.

4. The method according to claim 3, characterized in that When the mixing component AudioFlinger identifies that the playing state of the audio data to be processed is the second preset state, fading out the audio data to be processed to obtain a second audio processing result includes: When the audio mixing component AudioFlinger identifies that the playing state of the audio data to be processed is a paused playing state, the played audio data is faded out according to the unplayed audio data in the audio data to be processed to obtain faded-out audio.

5. The method according to claim 1, characterized in that The audio processing result includes fade-in audio or fade-out audio, and the method further includes: When the fade-in audio or the fade-out audio is interrupted and then started to play, the volume of the fade-in audio or the fade-out audio is controlled to gradually change from the playback volume before the interruption to the target playback volume.

6. The method according to any one of claims 1 to 5, characterized in that The method further includes: acquiring first configuration information of a fade-in function and a fade-out function, wherein the first configuration information includes enabling the fade-in function and the fade-out function, enabling the fade-in function, or enabling the fade-out function; The step of performing fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result includes: In the case where the first configuration information is to enable the fade-in function, fade-in processing is performed on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result; In the case where the first configuration information is to enable the fade-out function, fade-out processing is performed on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result; In the case where the first configuration information is to enable the fade-in function and the fade-out function, the audio data to be processed is faded-in or faded-out according to the playback status of the audio data to be processed to obtain an audio processing result.

7. The method according to any one of claims 1 to 5, characterized in that The method further includes: acquiring second configuration information of a fade-in function and a fade-out function, wherein the second configuration information includes a fade-in duration of the fade-in function and / or a fade-out duration of the fade-out function; The step of performing fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed to obtain an audio processing result includes: According to the playing state of the audio data to be processed and the fade-in duration, fade-in processing is performed on the audio data to be processed to obtain an audio processing result; and / or, According to the playing state of the audio data to be processed and the fade-out duration, fade-out processing is performed on the audio data to be processed to obtain an audio processing result.

8. An audio playback processing device, characterized in that: include: An acquisition unit, used for acquiring the playing status of the audio data to be processed, wherein the audio data to be processed is written into the audio buffer area of ​​the audio system engine by the application program; A fade-in and fade-out processing unit, used for performing fade-in processing or fade-out processing on the audio data to be processed according to the playback status of the audio data to be processed, to obtain an audio processing result; The audio mixing unit is used to mix the audio according to the audio processing result and output it to the audio playing device for playing.

9. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 7 by running the program in the memory.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, cause the processor to implement the method as claimed in any one of claims 1 to 7.

Images