A method and device for optimizing audio streaming media playback

Abstract

The present invention provides an audio streaming media optimized playback method and device, which solves the problems of audio streaming media in real-time, cross-platform compatibility and limited audio control processing capabilities in the prior art. By applying technologies such as API, Vue combined function encapsulation, custom audio buffer management and dynamic update of audio data source, cross-platform compatibility is ensured, processing delay of audio streaming data is reduced and precise control of playback progress is achieved. At the same time, the maintainability and scalability of the code are also improved.

Description

Technical Field

[0001] The present invention relates to the technical field of audio streaming media, and in particular to a method and device for optimizing the playback of audio streaming media. Background Art

[0002] With the rapid development of information technology, audio streaming has become an indispensable and important part of modern Internet applications. Audio streaming is widely used in many fields such as online music, voice calls, online education, live broadcasts, etc., which greatly enriches the digital experience of users. However, how to achieve efficient, stable and cross-platform compatible audio streaming playback in Web applications has always been a major challenge faced by technology developers.

[0003] The existing technologies mainly include the following implementation methods:

[0004] One: HTML5 <audio>Tag plays the audio stream directly:

[0005] <audio>As a standard audio playback element introduced by HTML5, the tag is widely used in Web audio playback because of its simplicity and ease of use. Developers only need to embed it in HTML <audio>Tags and set the audio source URL to achieve basic audio playback. This method does not rely on external plug-ins and has good cross-browser compatibility. However, <audio>Tags have obvious delay problems when processing real-time audio streams, which makes it difficult to meet application scenarios that require low latency and high real-time performance. In addition, their control and processing capabilities for audio data are limited, and they cannot meet complex audio processing requirements, such as dynamic audio splicing and real-time audio adjustment.

[0006] Two: Media Source Extensions (MSE):

[0007] MSE is an API that extends the functionality of media elements, allowing developers to dynamically add media data streams to the media buffer through JavaScript, thereby achieving more flexible audio and video playback control. With MSE, developers can implement advanced features such as customized buffering strategies and dynamically adjust playback content. This has significant advantages in implementing adaptive streaming media playback (such as DASH, HLS). However, MSE's support varies on different browsers and devices, especially on mobile devices (such as iOS devices), where MSE has poor compatibility, limiting its widespread use in cross-platform applications. In addition, the implementation complexity of MSE is high, which increases the difficulty of development and maintenance.

[0008] 3. Third-party plug-ins and libraries (such as WebRTC, Flash plug-ins):

[0009] In order to make up for the shortcomings of HTML5 and MSE in real-time audio stream processing, some developers choose to use third-party plug-ins or libraries, such as WebRTC (Web Real-Time Communication) and Flash plug-ins. These technologies can provide more powerful real-time audio processing capabilities and support low-latency voice calls, real-time audio streaming and other functions. As an open source project, WebRTC provides point-to-point real-time communication capabilities and is widely used in scenarios that require real-time interaction, such as video conferencing and online games. However, the use of third-party plug-ins or libraries also brings many problems. First, the plug-in has a strong dependency, which may cause users to install additional plug-ins, affecting the user experience. Secondly, the Flash plug-in has been gradually eliminated due to security issues and lack of support from mobile devices, limiting its application prospects in modern Web applications. In addition, the integration and maintenance of third-party libraries increase the complexity of development work, and there may be differences in compatibility and performance between different libraries.

[0010] Four: Custom audio processing based on Web Audio API:

[0011] The Web Audio API provides developers with rich audio processing capabilities and supports real-time generation, processing, and playback of audio. By building a complex network of audio processing nodes, developers can implement advanced features such as audio effects, dynamic volume adjustment, and real-time synthesis of audio data. This approach can theoretically provide a highly customized audio playback solution that is suitable for application scenarios that require complex audio processing. However, the learning curve of the Web Audio API is steep, and developers need to have a high level of audio processing knowledge. In addition, although the Web Audio API is well supported in modern browsers, there may still be compatibility issues in some older versions of browsers or on specific devices, affecting its universality in cross-platform applications. Summary of the invention

[0012] The present invention provides an audio streaming media optimized playback method and device to solve the problems of limited real-time performance, cross-platform compatibility and audio control processing capability of audio streaming media in the prior art.

[0013] In a first aspect, the present invention provides a method for optimizing the playback of audio streaming media, which specifically comprises the following steps:

[0014] Step S1, creating audio context and gain nodes;

[0015] Step S2, receiving and acquiring audio stream data, adding a custom file header to each audio stream data, and decoding the audio stream data to form decoded audio stream data;

[0016] Step S3, storing the decoded audio stream data in an array to form an audio buffer AudioBuffer;

[0017] Step S4: merging the received multiple (in this application, the "multiple" means at least two) audio stream data through the audio buffer AudioBuffer to form new audio stream data, and constructing a controller for controlling the playback state of the audio player;

[0018] Step S5: When playing audio, dynamically update the audio stream data and complete seamless splicing of the audio stream data segments according to the current audio playing time and the time deviation of the audio buffer;

[0019] Step S6: when the audio is played, paused, or ended, or when the component is uninstalled, release the audio resources;

[0020] Step S7: Manage the audio playback status through the responsive system of the front-end framework, dynamically update the playback progress and volume, and bind and display them with the UI in real time.

[0021] Wherein, in step S1, the gain node is connected to the destination.

[0022] The destination is a terminal device (audio player) for outputting audio.

[0023] Preferably, in step S1, an audio context is created through AudioContext.

[0024] Preferably, in step S2, the custom file header includes a timestamp and a segment number; wherein the timestamp is used to mark the start time of playing the audio segment, and the segment number is used to ensure the order of merging the audio stream data.

[0025] Preferably, in step S2, audio stream data is received through WebSocket technology or other real-time communication technology, and the received audio stream data format is ArrayBuffer; wherein, when the audio stream data is transmitted through fragmentation, it is necessary to add necessary file header information before each audio stream data fragment to ensure that the decoder can correctly parse the audio stream data format.

[0026] Preferably, in step S2, the audio stream data received in the ArrayBuffer format is decoded into audio stream data in the AudioBuffer format by audioContext.decodeAudioData.

[0027] Preferably, in step S4, the controller for controlling the playback state of the audio player adopts play (for controlling audio playback), pause (for controlling audio pause playback), playFromStart (for controlling audio playback from the beginning) and other methods to control the playback state of the audio player.

[0028] Preferably, in step S4, during the process of merging multiple audio stream data, each audio stream data includes custom file header information; wherein the custom file header information includes a timestamp, sequence and other necessary information for representation, which significantly improves the efficiency of audio splicing, ensures that each audio stream data is merged at an appropriate position and time, and avoids errors in the merging order of audio stream data or playback errors during audio playback.

[0029] Preferably, in step S5, seamless splicing of audio stream data segments is achieved by updateSourceAndResume to ensure the continuity of audio playback; the functions implemented by updateSourceAndResume specifically include:

[0030] A. Interrupt the playback of the current audio stream data audio through stop;

[0031] B. Merge the latest received audio stream data with the current audio stream data to form new audio stream data;

[0032] C. Continue the audio playback from the interrupted audio stream data position through start;

[0033] Among them, stop is used to interrupt the current audio playback, and start is used to start playing the audio from the position where the current audio playback is interrupted.

[0034] Preferably, in step S6, when the audio is playing, paused, ended, or the component is uninstalled, in the onUnmounted lifecycle hook of the component, pause is called to stop the audio playback of the audio player, and the audio context is closed to release resources and prevent memory leaks, thereby ensuring that performance degradation is not caused by untimely resource release.

[0035] Preferably, in step S7, steps S1 to S4 are encapsulated and managed through the reactive function and ref function of the vue framework, and the state variables of the audio player are defined and managed; wherein, the state variables of the audio player include isPlaying, currentTime, duration, and volume (isPlaying indicates that the audio is playing, currentTime indicates the current playing time, duration indicates the playing time, and volume indicates the volume); through the responsive system of the vue framework, real-time updating and interface display of the audio player status are realized.

[0036] More preferably, the state variables are bound and UI (UserInterface) elements are updated through the responsive API of the vue framework to achieve real-time update and interface display of the audio player state, which specifically includes the following steps:

[0037] Step 1. Create the state variable of the audio player through the reactive function;

[0038] Step 2. Bind UI elements and state variables through v-bind (v-bind is a directive in the vue framework, used to dynamically bind one or at least two attributes) and v-model (v-model is a directive used to create two-way data binding, mainly used for form input, selection and text area elements, synchronizing user input with the data attributes of the vue instance) to control the audio playback;

[0039] Step 3. During the playback process, the state variables of the audio player are updated in real time, and the changes in the state variables are processed through the watch function or computed function in Vue to display the audio playback status and audio playback volume in real time.

[0040] Preferably, a method for optimizing the playback of audio streaming media further includes:

[0041] Step S8, adding the function interface of obtaining the Blob URL, setting the playback volume of the audio player and resetting the audio player.

[0042] The merged audio buffer AudioBuffer is encoded into a Blob URL in WAV format by createWavBlob, so that the user can download it or use it in other components.

[0043] Among them, the gain value of the gain node is controlled by setVolume to adjust the playback volume of the audio player.

[0044] Among them, the buffer is cleared by reset, the audio player state is reset, and resources are released.

[0045] In a second aspect, the present invention further provides an audio streaming media optimized playback device, which specifically includes the following modules:

[0046] Create a module to create audio context and gain nodes;

[0047] The decoding processing module is used to receive and obtain audio stream data, add a custom file header to each audio stream data, and decode the audio stream data to form decoded audio stream data;

[0048] A buffer construction module, used for storing the decoded audio stream data in an array to form an audio buffer AudioBuffer;

[0049] An audio data merging and playing state processing module, used to merge the received multiple audio stream data through the audio buffer AudioBuffer to form new audio stream data, and to construct a controller for controlling the playing state of the audio player;

[0050] The audio playback seamless splicing module is used to dynamically update the audio stream data and complete the seamless splicing of the audio stream data segments during audio playback according to the current audio playback time and the time deviation of the audio buffer;

[0051] The audio resource release module is used to release audio resources when the audio is played, paused, ended, or when the component is uninstalled;

[0052] The audio playback status real-time display module is used to manage the audio playback status through the responsive system of the front-end framework, dynamically update the playback progress and volume, and bind and display it with the UI in real time.

[0053] Wherein, in the creation module, the gain node is connected to the destination.

[0054] The destination is a terminal device (audio player) for outputting audio.

[0055] Preferably, in the creation module, an audio context is created through AudioContext.

[0056] Preferably, in the decoding processing module, the custom file header includes a timestamp and a segment number; wherein the timestamp is used to mark the start time of playing the audio segment, and the segment number is used to ensure the order of merging the audio stream data.

[0057] Preferably, in the decoding processing module, audio stream data is received through WebSocket technology or other real-time communication technology, and the received audio stream data format is ArrayBuffer; wherein, when the audio stream data is transmitted through fragmentation, it is necessary to add necessary file header information before each audio stream data fragment to ensure that the decoder can correctly parse the audio stream data format.

[0058] Preferably, in the decoding processing module, the audio stream data received in the ArrayBuffer format is decoded into the audio stream data in the AudioBuffer format by means of audioContext.decodeAudioData.

[0059] Preferably, in the audio data merging and playing state processing module, the controller for controlling the playing state of the audio player adopts play (for controlling audio playing), pause (for controlling audio pause playing), playFromStart (for controlling audio playing from the beginning) and other methods to control the playing state of the audio player.

[0060] Preferably, in the audio data merging and playback status processing module, during the process of merging multiple audio stream data, each audio stream data includes custom file header information; wherein the custom file header information includes a timestamp, sequence and other necessary information for representation, which significantly improves the efficiency of audio splicing, ensures that each audio stream data is merged at an appropriate position and time, and avoids errors in the merging order of audio stream data or playback errors during audio playback.

[0061] Preferably, in the audio playback seamless splicing module, the seamless splicing of the audio stream data segments is achieved by updateSourceAndResume to ensure the continuity of the audio playback; the functions implemented by updateSourceAndResume specifically include:

[0062] A. Interrupt the playback of the current audio stream data audio through stop;

[0063] B. Merge the latest received audio stream data with the current audio stream data to form new audio stream data;

[0064] C. Continue the audio playback from the interrupted audio stream data position through start;

[0065] Among them, stop is used to interrupt the current audio playback, and start is used to start playing the audio from the position where the current audio playback is interrupted.

[0066] Preferably, in the audio resource release module, when the audio is playing, paused, ended, or the component is uninstalled, in the onUnmounted lifecycle hook of the component, pause is called to stop the audio playback of the audio player, and the audio context is closed to release resources and prevent memory leaks, thereby ensuring that performance degradation is not caused by untimely resource release.

[0067] Preferably, in the audio playback status real-time display module, the creation module, the decoding processing module, the buffer construction module and the audio data merging and playback status processing module are encapsulated and managed through the reactive function and ref function of the vue framework, and the state variables of the audio player are defined and managed; wherein, the state variables of the audio player include isPlaying, currentTime, duration, and volume; and through the responsive system of the vue framework, real-time update and interface display of the audio player status are realized.

[0068] More preferably, the state variables are bound and UI elements are updated through the responsive API of the vue framework to achieve real-time update and interface display of the audio player status, specifically including the following submodules:

[0069] The first processing submodule is used to create a state variable of the audio player through a reactive function;

[0070] The second processing submodule is used to bind UI elements and state variables through v-bind and v-model to control the playback of audio;

[0071] The third processing submodule is used to update the state variables of the audio player in real time during the playback process, and process the changes of the state variables through the watch function or computed function in Vue to display the audio playback status and audio playback volume in real time.

[0072] Preferably, an audio streaming media optimized playback device further includes:

[0073] The function interface adds a module for adding the function interfaces for obtaining the Blob URL, setting the playback volume of the audio player, and resetting the audio player.

[0074] The merged audio buffer AudioBuffer is encoded into a Blob URL in WAV format by createWavBlob, so that the user can download it or use it in other components.

[0075] Among them, the gain value of the gain node is controlled by setVolume to adjust the playback volume of the audio player.

[0076] Among them, the buffer is cleared by reset, the audio player state is reset, and resources are released.

[0077] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an audio streaming media optimized playback method as described in any one of the first aspects of the present application.

[0078] In a fourth aspect, the present invention further provides an electronic device, comprising: a memory storing a computer program; and a processor communicatively connected to the memory, for executing an audio streaming media optimized playback method described in any one of the first aspects of the present application when the computer program is called.

[0079] Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages:

[0080] The present invention provides an audio streaming media optimized playback method and device, which solves the problems of audio streaming media in real-time, cross-platform compatibility and limited audio control processing capabilities in the prior art. By applying technologies such as API, Vue combined function encapsulation, custom audio buffer management and dynamic update of audio data source, cross-platform compatibility is ensured, processing delay of audio streaming data is reduced and precise control of playback progress is achieved. At the same time, the maintainability and scalability of the code are also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The accompanying drawings, which constitute part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention. In the accompanying drawings:

[0082] Figure 1 The present invention is a flowchart of an audio streaming media optimized playback method according to a preferred embodiment of the present invention.

[0083] Figure 2 It is a structural schematic diagram of an audio streaming media optimized playback device according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0084] The present invention provides an audio streaming media optimized playback method and device. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0085] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0086] Embodiment 1:

[0087] like Figure 1 As shown, the audio streaming media optimized playback method described in this embodiment specifically includes the following steps:

[0088] Step S1, creating an audio context and a gain node; wherein the gain node is connected to a destination; the destination is a terminal device (audio player) for outputting audio; optionally, creating an audio context through AudioContext.

[0089] Step S2, receiving and acquiring audio stream data, adding a custom file header to each audio stream data, and decoding the audio stream data to form decoded audio stream data; wherein, the audio stream data is received through WebSocket technology or other real-time communication technology, and the received audio stream data format is ArrayBuffer; when the audio stream data is transmitted by fragmentation, it is necessary to add necessary custom file header information before each audio stream data fragment (the custom file header information includes a timestamp and a fragment number; wherein the timestamp is used to mark the start time of the playback of the audio fragment, and the fragment number is used to ensure the order of merging the audio stream data), so as to ensure that the decoder can correctly parse the audio stream data format.

[0090] Among them, the audio stream data received in the ArrayBuffer format is decoded into the audio stream data in the AudioBuffer format by audioContext.decodeAudioData.

[0091] Step S3: storing the decoded audio data in an array to form an audio buffer AudioBuffer.

[0092] Step S4: merging the received multiple audio stream data through the audio buffer AudioBuffer to form new audio stream data, and constructing a controller for controlling the playback state of the audio player.

[0093] The controller for controlling the playing state of the audio player adopts play, pause, playFromStart and other methods to control the playing state of the audio player.

[0094] In the process of merging multiple audio stream data, each audio stream data includes custom file header information; wherein, the custom file header information includes a timestamp, sequence and other necessary information for representation, which significantly improves the efficiency of audio splicing, ensures that each audio stream data is merged at the appropriate position and time, and avoids errors in the merging order of audio stream data or playback errors during audio playback.

[0095] Step S5: When playing audio, dynamically update the audio stream data and complete seamless splicing of the audio stream data segments according to the current audio playing time and the time deviation of the audio buffer.

[0096] Among them, the updateSourceAndResume method is used to achieve seamless splicing of audio stream data segments to ensure the continuity of audio playback; the functions implemented by the updateSourceAndResume specifically include:

[0097] A. Interrupt the playback of the current audio stream data through stop.

[0098] B. Merge the latest received audio stream data with the current audio stream data to form new audio stream data.

[0099] C. Continue audio playback from the interrupted audio stream data position through start.

[0100] Among them, stop is used to interrupt the current audio playback, and start is used to start playing the audio from the position where the current audio playback is interrupted.

[0101] Step S6, releasing audio resources when the audio is playing, paused, ended, or when the component is uninstalled; wherein, when the audio is playing, paused, ended, or when the component is uninstalled, in the onUnmounted lifecycle hook of the component, calling pause is used to stop the audio playback of the audio player, and the audio context is closed to release resources and prevent memory leaks, thereby ensuring that performance degradation is not caused by untimely release of resources.

[0102] Step S7: Manage the audio playback status through the responsive system of the front-end framework, dynamically update the playback progress and volume, and bind and display them with the UI in real time.

[0103] Among them, steps S1 to S4 are encapsulated and managed through the reactive function and ref function of the vue framework, and the state variables of the audio player are defined and managed; wherein the state variables of the audio player include isPlaying, currentTime, duration, and volume; through the responsive system of the vue framework, the real-time update and interface display of the audio player state are realized; through the responsive API of the vue framework, the state variables are bound and the UI elements are updated to realize the real-time update and interface display of the audio player state, which specifically includes the following steps:

[0104] Step 1. Create the state variable of the audio player through the reactive function.

[0105] Step 2. Bind UI elements and state variables through v-bind and v-model to control the audio playback.

[0106] Step 3. During the playback process, the state variables of the audio player are updated in real time, and the changes in the state variables are processed through the watch function or computed function in Vue to display the audio playback status and audio playback volume in real time.

[0107] Optionally, a method for optimizing the playback of audio streaming media further includes:

[0108] Step S8, adding the function interface of obtaining the Blob URL, setting the playback volume of the audio player and resetting the audio player.

[0109] Among them, the merged audio buffer AudioBuffer is encoded into a Blob URL in WAV format through the createWavBlob method, so that users can download it or use it in other components; the gain value of the gain node is controlled through the setVolume method to adjust the playback volume of the audio player; the buffer is cleared through the reset method, the audio player status is reset, and resources are released.

[0110] Embodiment 2:

[0111] like Figure 2 As shown, the audio streaming media optimized playback device described in this embodiment specifically includes the following modules:

[0112] A creation module is used to create an audio context and a gain node; wherein the gain node is connected to a destination; and the destination is a terminal device (audio player) for outputting audio.

[0113] Among them, the audio context is created through AudioContext.

[0114] The decoding processing module is used to receive and obtain audio stream data, add a custom file header to each audio stream data, and decode the audio stream data to form decoded audio stream data; wherein, the audio stream data is received through WebSocket technology or other real-time communication technology, and the received audio stream data format is ArrayBuffer; wherein, when the audio stream data is transmitted through fragmentation, it is necessary to add necessary custom file header information (including metadata such as timestamp, audio segment number, etc., for improving the efficiency of audio stream data splicing) before each audio stream data fragment to ensure that the decoder can correctly parse the audio stream data format.

[0115] Among them, the audio stream data received in the ArrayBuffer format is decoded into the audio stream data in the AudioBuffer format by audioContext.decodeAudioData.

[0116] The buffer construction module is used to store the decoded audio stream data in an array to form an audio buffer AudioBuffer.

[0117] The audio data merging and playing state processing module is used to merge the received multiple audio stream data through the audio buffer AudioBuffer to form new audio stream data, and to build a controller for controlling the playing state of the audio player.

[0118] The controller for controlling the playing state of the audio player adopts play, pause, playFromStart and other methods to control the playing state of the audio player.

[0119] In the process of merging multiple audio stream data, each audio stream data includes custom file header information; wherein, the custom file header information includes a timestamp, sequence and other necessary information for representation, which significantly improves the efficiency of audio splicing, ensures that each audio stream data is merged at the appropriate position and time, and avoids errors in the merging order of audio stream data or playback errors during audio playback.

[0120] The audio playback seamless splicing module is used to dynamically update the audio stream data and complete the seamless splicing of the audio stream data segments during audio playback according to the current audio playback time and the time deviation of the audio buffer.

[0121] Among them, the updateSourceAndResume method is used to achieve seamless splicing of audio stream data segments to ensure the continuity of audio playback; the functions implemented by the updateSourceAndResume specifically include:

[0122] A. Interrupt the playback of the current audio stream data audio through stop;

[0123] B. Merge the latest received audio stream data with the current audio stream data to form new audio stream data;

[0124] C. Continue the audio playback from the interrupted audio stream data position through start;

[0125] Among them, stop is used to interrupt the current audio playback, and start is used to start playing the audio from the position where the current audio playback is interrupted.

[0126] The audio resource release module is used to release audio resources when the audio is played, paused, ended, or when the component is uninstalled.

[0127] Among them, when the audio is playing, paused, ended, or the component is uninstalled, in the onUnmounted lifecycle hook of the component, pause is called to stop the audio playback of the audio player, and the audio context is closed to release resources and prevent memory leaks, ensuring that performance will not be degraded due to untimely resource release.

[0128] The audio playback status real-time display module is used to manage the audio playback status through the responsive system of the front-end framework, dynamically update the playback progress and volume, and bind and display it with the UI in real time.

[0129] Among them, the creation module, the decoding processing module, the buffer construction module and the audio data merging and playback status processing module are encapsulated and managed through the reactive function and ref function of the vue framework, and the state variables of the audio player are defined and managed; wherein, the state variables of the audio player include isPlaying, currentTime, duration, and volume; through the responsive system of the vue framework, the real-time update and interface display of the audio player status are realized.

[0130] Among them, the state variables are bound and UI elements are updated through the responsive API of the vue framework to achieve real-time update and interface display of the audio player status, which specifically includes the following sub-modules:

[0131] The first processing submodule is used to create a state variable of the audio player through a reactive function.

[0132] The second processing submodule is used to bind UI elements and state variables through v-bind and v-model to control the playback of audio.

[0133] The third processing submodule is used to update the state variables of the audio player in real time during the playback process, and process the changes of the state variables through the watch function or computed function in Vue to display the audio playback status and audio playback volume in real time.

[0134] Optionally, an audio streaming media optimized playback device further includes:

[0135] The function interface adds a module for adding the function interfaces for obtaining the Blob URL, setting the playback volume of the audio player, and resetting the audio player.

[0136] Among them, the merged audio buffer AudioBuffer is encoded into a Blob URL in WAV format through the createWavBlob method, so that users can download it or use it in other components; the gain value of the gain node is controlled through the setVolume method to adjust the playback volume of the audio player; the buffer is cleared through the reset method, the audio player status is reset, and resources are released.

[0137] The present invention provides an audio streaming media optimized playback method and device, which have the following advantages: (1) avoiding dependence on incompatible technologies: not using MSE and third-party plug-ins with poor compatibility on mobile browsers (especially iOS devices) to ensure cross-platform compatibility; (2) optimizing decoding and merging efficiency: decoding is performed immediately after receiving the audio data, and an efficient algorithm is used to merge the audio buffer to reduce processing delay; (3) fine-grained playback control: by managing variables such as currentTime and isPlaying, precise control of playback progress is achieved, and dynamic adjustment and seamless playback are supported; (4) improving code maintainability and scalability: using Vue's combined functions to encapsulate complex logic into modular functions, the code structure is clear, and it is easy to maintain and expand.

[0138] The specific embodiments of the present invention are described in detail above, but they are only examples, and the present invention is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications and substitutions made to the present invention are also within the scope of the present invention. Therefore, the equalization changes and modifications made without departing from the spirit and scope of the present invention should be included in the scope of the present invention.< / audio> < / audio> < / audio> < / audio>

Claims

1. A method for optimizing the playback of audio streaming media, characterized in that: The specific steps include: Step S1, creating an audio context and gain node; Step S2, receiving and acquiring audio stream data, adding a custom file header to each audio stream data, and then decoding the audio stream data to form decoded audio stream data; Step S3, storing the decoded audio stream data in an array to form an audio buffer AudioBuffer; Step S4, merging the received multiple audio stream data through the audio buffer AudioBuffer to form new audio stream data, and constructing a controller for controlling the playback state of the audio player; Step S5: When playing audio, dynamically update the audio stream data and complete seamless splicing of the audio stream data segments according to the current audio playing time and the time deviation of the audio buffer; Step S6: when the audio is played, paused, or ended, or when the component is uninstalled, release the audio resources; Step S7: Manage the audio playback status through the responsive system of the front-end framework, dynamically update the playback progress and volume, and bind with the UI in real time; Wherein, in step S1, the gain node is connected to a destination; the destination is a terminal device for outputting audio; In step S7, steps S1 to S4 are encapsulated and managed through the reactive function and ref function of the vue framework, and the state variables of the audio player are defined and managed; wherein the state variables of the audio player include isPlaying, currentTime, and duration; and the real-time update and interface display of the audio player status are realized through the responsive system of the vue framework; The state variables are bound and UI elements are updated through the responsive API of the Vue framework to achieve real-time update and interface display of the audio player status, which specifically includes the following steps: Step 1. Create the state variable of the audio player through the reactive function; Step 2. Bind UI elements and state variables through v-bind and v-model to control the playback of audio; Step 3. During the playback process, the state variables of the audio player are updated in real time, and the changes in the state variables are processed through the watch function or computed function in Vue to display the audio playback status and audio playback volume in real time.

2. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: In step S1, an audio context is created through AudioContext.

3. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: In step S2, the custom file header includes a timestamp and a segment number; wherein the timestamp is used to mark the start time of playing the audio segment, and the segment number is used to ensure the order of merging audio stream data.

4. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: In step S2, audio stream data is received through WebSocket technology or other real-time communication technology, and the received audio stream data format is ArrayBuffer; wherein, when the audio stream data is transmitted through fragmentation, it is necessary to add file header information before each audio stream data fragment to ensure that the decoder correctly parses the audio stream data format.

5. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: In step S2, the audio stream data received in the ArrayBuffer format is decoded into audio stream data in the AudioBuffer format by audioContext.decodeAudioData.

6. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: In step S4, the controller for controlling the playing state of the audio player adopts the play, pause, and playFromStart modes to control the playing state of the audio player.

7. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: In step S4, during the process of merging multiple audio stream data, each audio stream data includes custom file header information.

8. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: In step S5, the seamless splicing of the audio stream data segments is achieved by updateSourceAndResume to ensure the continuity of the audio playback; the functions implemented by updateSourceAndResume specifically include: A. Interrupt the playback of the current audio stream data audio through stop; B. Merge the latest received audio stream data with the current audio stream data to form new audio stream data; C. Continue the audio playback from the interrupted audio stream data position through start; Among them, stop is used to interrupt the current audio playback, and start is used to start playing the audio from the position where the current audio playback is interrupted.

9. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: In step S6, when the audio is playing, paused, ended, or the component is uninstalled, in the onUnmounted lifecycle hook of the component, pause is called to stop the audio playback of the audio player, and the audio context is closed to release resources and prevent memory leaks, ensuring that performance will not be degraded due to untimely resource release.

10. The method for optimizing the playback of audio streaming media according to claim 1, characterized in that: Also includes: Step S8, adding the function interface of obtaining Blob URL, setting the playback volume of the audio player and resetting the audio player; Among them, the merged audio buffer AudioBuffer is encoded into a Blob URL in WAV format through the createWavBlob method; the gain value of the gain node is controlled through the setVolume method to adjust the playback volume of the audio player; the buffer is cleared through the reset method, the audio player state is reset, and resources are released.

11. An audio streaming media optimized playback device, characterized in that: Used to execute the audio streaming media optimized playback method according to claim 1, the device specifically comprises the following modules: Create a module to create audio context and gain nodes; The decoding processing module is used to receive and obtain audio stream data, add a custom file header to each audio stream data, and decode the audio stream data to form decoded audio stream data; A buffer construction module, used for storing the decoded audio stream data in an array to form an audio buffer AudioBuffer; An audio data merging and playing state processing module, used to merge the received multiple audio stream data through the audio buffer AudioBuffer to form new audio stream data, and to construct a controller for controlling the playing state of the audio player; The audio playback seamless splicing module is used to dynamically update the audio stream data and complete the seamless splicing of the audio stream data segments during audio playback according to the current audio playback time and the time deviation of the audio buffer; The audio resource release module is used to release audio resources when the audio is played, paused, ended, or when the component is uninstalled; The audio playback status real-time display module is used to manage the audio playback status through the responsive system of the front-end framework, dynamically update the playback progress and volume, and bind to the UI in real time; Wherein, in the creation module, the gain node is connected to a destination; the destination is a terminal device for outputting audio; In the audio playback status real-time display module, the creation module, the decoding processing module, the buffer construction module and the audio data merging and playback status processing module are encapsulated and managed through the reactive function and ref function of the vue framework, and the state variables of the audio player are defined and managed; wherein the state variables of the audio player include isPlaying, currentTime, duration, and volume; through the responsive system of the vue framework, the real-time update and interface display of the audio player status are realized; The state variables are bound and UI elements are updated through the responsive API of the Vue framework to achieve real-time update and interface display of the audio player status, which specifically includes the following sub-modules: The first processing submodule is used to create a state variable of the audio player through a reactive function; The second processing submodule is used to bind UI elements and state variables through v-bind and v-model to control the playback of audio; The third processing submodule is used to update the state variables of the audio player in real time during the playback process, and process the changes of the state variables through the watch function or computed function in Vue to display the audio playback status and audio playback volume in real time.

Images