Audio playing method, device and system, and audio transmission method and device

Abstract

This invention provides an audio playback method, comprising: acquiring the data transmission status between the client and a server; upon receiving an audio playback request from a user, generating an audio data request corresponding to the current data transmission status; sending the audio data request to the server and receiving target audio stream data returned by the server based on the audio data request; and decoding and playing the target audio stream data according to the decoding parameters corresponding to the audio data request. By generating the corresponding audio data request through the current data transmission status and the audio playback request, sending the audio data request corresponding to the data transmission status from the client, and having the server return the target audio stream data corresponding to the data transmission status, the method eliminates the need for the server to determine the data transmission status, thereby improving data transmission efficiency and ultimately audio playback efficiency.

Description

Technical Field

[0001] This invention relates to the field of data transmission, and more particularly to an audio playback method, apparatus and system, and an audio transmission method and apparatus. Background Technology

[0002] In existing audio data transmission and playback technologies, audio data interaction between the client and server typically follows fixed transmission patterns and parameter settings. However, this traditional approach often overlooks the impact of real-time network conditions and server load on audio playback quality. When network conditions are poor or the server load is heavy, audio data transmission may be affected by latency or packet loss, leading to issues such as stuttering, degraded sound quality, or even playback failure on the client side. Summary of the Invention

[0003] This invention provides an audio playback method designed to address the issue that in existing audio data transmission processes, when network conditions are poor or the server load is heavy, audio data transmission may be affected by delays or packet loss, leading to stuttering, degraded sound quality, or even playback failure on the client side. By monitoring the data transmission status between the client and the server, and generating corresponding audio data requests based on the current data transmission status and the audio playback request, the client sends an audio data request corresponding to the data transmission status. The server returns the target audio stream data corresponding to the data transmission status, eliminating the need for server-side data transmission status determination, thus improving data transmission efficiency. The client can pre-determine the corresponding decoding parameters based on the audio data request and directly parse the received target audio stream data, thereby improving audio playback efficiency.

[0004] In a first aspect, embodiments of the present invention provide an audio playback method for a client.

[0005] The audio playback method includes the following steps:

[0006] Get the data transmission status between the server and the client;

[0007] Upon receiving a user's audio playback request, an audio data request corresponding to the current data transmission status is generated based on the current data transmission status.

[0008] The audio data request is sent to the server, and the target audio stream data returned by the server based on the audio data request is received.

[0009] The target audio stream data is decoded and played according to the decoding parameters corresponding to the audio data request.

[0010] Optionally, obtaining the data transmission status with the server includes:

[0011] Get the current network status;

[0012] Send a request for the current server status parameters to the server;

[0013] Receive the current server status parameters returned by the server in response to the current server status parameter request;

[0014] Based on the current network status and the current server status parameters, the data transmission status with the server is determined.

[0015] Optionally, generating an audio data request corresponding to the current data transmission state based on the current data transmission state includes:

[0016] If the current data transmission status is normal, then an audio data request under normal transmission status is generated;

[0017] If the current data transmission state is abnormal, an audio data request under abnormal transmission state is generated, and the compression bit rate of the audio data request under abnormal transmission state is lower than the compression bit rate of the audio data request under normal transmission state.

[0018] Optionally, the target audio stream data includes segmented audio stream data, and the step of decoding and playing the target audio stream data according to the decoding parameters corresponding to the audio data request includes:

[0019] Based on the current data transmission status, determine the current decoding parameters;

[0020] Based on the current decoding parameters, the received segmented audio stream data is decoded, and the decoded segmented audio stream data is played.

[0021] During the playback of the decoded segmented audio stream data, the remaining segmented audio stream data of the target audio stream data continues to be received.

[0022] Secondly, embodiments of the present invention also provide an audio transmission method for a server, the audio transmission method comprising:

[0023] Receive audio data requests sent by the client;

[0024] Locate the target audio data corresponding to the audio data request, and process the target audio data based on the audio data request to obtain target audio stream data;

[0025] The target audio stream data is returned to the client.

[0026] Optionally, before receiving the audio data request sent by the client, the method further includes:

[0027] Receive the current server status parameter request sent by the client;

[0028] Based on the current server status parameter request, query the current server status parameters;

[0029] The current server status parameters are returned to the client.

[0030] Optionally, processing the target audio data based on the audio data request to obtain target audio stream data includes:

[0031] If the audio data request is an audio data request under normal transmission conditions, the target audio data is processed by the first compression method to obtain the first audio stream data processed by the first compression method.

[0032] If the audio data request is an audio data request under abnormal transmission conditions, the target audio data is processed by the second compression method to obtain the second audio stream data after the second compression method is processed. The compression bit rate of the second compression method is lower than the compression bit rate of the first compression method.

[0033] The first audio stream data or the second audio stream data is segmented to obtain the target audio stream data.

[0034] Thirdly, embodiments of the present invention also provide an audio playback device for use in the audio playback method described in any one of the embodiments of the present invention, the audio playback device comprising:

[0035] The acquisition module is used to obtain the data transmission status between the server and the client.

[0036] The first receiving module is used to receive the user's audio playback request and, based on the current data transmission status, generate an audio data request corresponding to the current data transmission status.

[0037] The second receiving module is used to send the audio data request to the server and receive the target audio stream data returned by the server based on the audio data request.

[0038] The playback module is used to decode and play the target audio stream data.

[0039] Fourthly, embodiments of the present invention also provide an audio transmission device for use in the audio transmission method described in any one of the embodiments of the present invention, the audio transmission device comprising:

[0040] The third receiving module is used to receive audio data requests sent by the client;

[0041] The processing module is used to find the target audio data corresponding to the audio data request, and process the target audio data based on the audio data request to obtain target audio stream data;

[0042] The first return module is used to return the target audio stream data to the client.

[0043] Thirdly, embodiments of the present invention also provide an audio playback system, including: a client and a server, wherein the client includes the audio playback device described in the embodiments of the present invention, and the server includes the audio transmission device described in the embodiments of the present invention.

[0044] In this embodiment of the invention, the data transmission status between the client and the server is acquired; upon receiving an audio playback request from a user, an audio data request corresponding to the current data transmission status is generated; the audio data request is sent to the server, and the target audio stream data returned by the server based on the audio data request is received; the target audio stream data is decoded and played according to the decoding parameters corresponding to the audio data request. By monitoring the data transmission status between the client and the server, and generating a corresponding audio data request based on the current data transmission status and the audio playback request, the client sends an audio data request corresponding to the data transmission status, and the server returns the target audio stream data corresponding to the data transmission status. This eliminates the need for the server to determine the data transmission status, thereby improving data transmission efficiency. The client can pre-determine the corresponding decoding parameters based on the audio data request and directly parse the received target audio stream data, thus improving audio playback efficiency. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the architecture of an audio playback system provided in an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the architecture of another audio playback system provided in an embodiment of the present invention;

[0048] Figure 3This is a flowchart illustrating an audio playback method provided in an embodiment of the present invention;

[0049] Figure 4 This is a flowchart illustrating another audio playback method provided in an embodiment of the present invention;

[0050] Figure 5 This is a flowchart illustrating an audio transmission method provided in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the structure of an audio playback device provided in an embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram of the structure of an audio transmission device provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] like Figure 1 As shown, Figure 1 This is an architecture diagram of an audio playback system provided by an embodiment of the present invention. The audio playback system includes a client and a server, and the client and the server communicate with each other through a network.

[0055] The network described above is a medium used to provide a communication link between the client and the server. The network can include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0056] Users can use the client to interact with the server over the network to receive or send messages, etc. Various client applications can be installed on the client, such as web browsers, shopping apps, search engines, instant messaging tools, email clients, social media platforms, audio players, and video players.

[0057] The client can be various electronic devices with a display screen and web browsing support, including but not limited to smart learning machines, learning machines (devices that take pictures of objects and identify related knowledge), smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0058] A server can be a server that provides various services, such as a backend server that supports the pages displayed on the client and the player.

[0059] It should be noted that the vehicle body scratch detection method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the vehicle body scratch detection device is generally set in the server / terminal device.

[0060] It should be understood that Figure 1 The number of clients, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of clients, networks, and servers.

[0061] The client obtains the data transmission status between itself and the server; after receiving the user's audio playback request, the client generates an audio data request corresponding to the current data transmission status; the client then sends the audio data request to the server.

[0062] The server receives an audio data request from the client; the server finds the target audio data corresponding to the audio data request; the server processes the target audio data based on the audio data request to obtain the target audio stream data; and the server returns the target audio stream data to the client.

[0063] The client receives the target audio stream data returned by the server based on the audio data request; the client decodes and plays the target audio stream data according to the decoding parameters corresponding to the audio data request.

[0064] In this embodiment of the invention, the client monitors the data transmission status between the client and the server, and generates a corresponding audio data request based on the current data transmission status and the audio playback request. The client sends an audio data request corresponding to the data transmission status, and the server returns the target audio stream data corresponding to the data transmission status. The server does not need to determine the data transmission status, thereby improving data transmission efficiency. The client can predetermine the corresponding decoding parameters based on the audio data request and directly parse the received target audio stream data, thereby improving audio playback efficiency.

[0065] like Figure 2 As shown, Figure 2 This is an architecture diagram of another audio playback system provided in an embodiment of the present invention. The audio playback system includes: a client, a gateway server, and an application server. The client and the gateway server, and the gateway server and the application server communicate with each other through a network.

[0066] The client obtains the data transmission status between itself and the server. After receiving the user's audio playback request, the client generates an audio data request corresponding to the current data transmission status. The client sends the audio data request to the gateway server, which intelligently and dynamically forwards it to the application server based on the load of each application server (connection count > io > memory > CPU).

[0067] The server receives an audio data request; the server finds the target audio data corresponding to the audio data request; the server processes the target audio data based on the audio data request to obtain the target audio stream data; the server sends the target audio stream data to the gateway server, and the gateway server returns the target audio stream data to the client.

[0068] The client receives the target audio stream data returned by the server based on the audio data request; the client decodes and plays the target audio stream data according to the decoding parameters corresponding to the audio data request.

[0069] like Figure 3 As shown, Figure 3 This is a flowchart of an audio playback method provided by an embodiment of the present invention, which includes the following steps:

[0070] 101. Obtain the data transmission status between the server and the client.

[0071] In this embodiment of the invention, the above-described audio playback method can be used on a client device, which can be a device for playing audio, such as a learning machine, a reading pen, a smartphone, a tablet, a computer, or other electronic devices capable of voice playback.

[0072] The above data transmission status is used to indicate the data transmission status between the client and the server. The above data transmission status can be divided into normal and abnormal. Normal data transmission status indicates that the current network status is good and the server performance is good. Abnormal transmission status indicates that the current network status is poor or the server performance is poor.

[0073] The data transmission status between the client and server can be obtained by monitoring the current network status and server status parameters. This data transmission status can be obtained through real-time monitoring or periodic monitoring.

[0074] 102. Upon receiving an audio playback request from a user, generate an audio data request corresponding to the current data transmission status based on the current data transmission status.

[0075] In this embodiment of the invention, the audio playback request can be a playback request generated by the user clicking on an audio object in the playback interface, or it can be a playback request input by the user through other modal commands, such as voice commands, gesture commands, etc.

[0076] It should be noted that in this embodiment of the invention, different data transmission states correspond to different audio request parameters. After receiving the user's audio playback request, the corresponding audio request parameters are found according to the current data transmission state, and an audio data request corresponding to the current data transmission state is generated based on the found audio request parameters.

[0077] The data transmission states described above can be categorized as normal and abnormal. When the data transmission state is normal, an audio data request for normal transmission can be generated. When the data transmission state is abnormal, an audio data request for abnormal transmission can be generated. If the data transmission state is normal, the audio request parameters for normal transmission are retrieved, and an audio data request for normal transmission is generated based on these parameters. If the data transmission state is abnormal, the audio request parameters for abnormal transmission are retrieved, and an audio data request for abnormal transmission is generated based on these parameters.

[0078] The audio request parameters mentioned above may include corresponding parameter requests such as encoder, bit rate, format, and adaptation set.

[0079] 103. Send the audio data request to the server and receive the target audio stream data returned by the server based on the audio data request.

[0080] In this embodiment of the invention, after receiving an audio data request, the client sends the audio data request to the server through the network between the client and the server. The server parses the audio data request and determines the target audio data stream that the user wants to play based on the parsing result.

[0081] After locating the target audio data stream, the server can send the target audio data stream to the client via the network between the client and the server.

[0082] In one possible embodiment, the client and server include a gateway server and an application server. After receiving an audio data request, the client sends the audio data request to the gateway server through the network (external network) between the client and the gateway server. The gateway server forwards the audio data request to the corresponding application server through the network (internal network) between the gateway server and the server. The application server parses the audio data and determines the target audio data stream that the user wants to play based on the parsing result.

[0083] 104. Decode and play the target audio stream data according to the decoding parameters requested by the audio data request.

[0084] In this embodiment of the invention, after generating an audio data request corresponding to the current data transmission state based on the audio request parameters, corresponding decoding parameters can also be generated based on the audio request parameters and pre-stored. After receiving the target audio stream data, the target audio stream data is parsed using the pre-stored corresponding decoding parameters to obtain audio data for playback, and then played through an audio player.

[0085] In this embodiment of the invention, the data transmission status between the client and the server is acquired; upon receiving an audio playback request from a user, an audio data request corresponding to the current data transmission status is generated; the audio data request is sent to the server, and the target audio stream data returned by the server based on the audio data request is received; the target audio stream data is decoded and played according to the decoding parameters corresponding to the audio data request. By monitoring the data transmission status between the client and the server, and generating a corresponding audio data request based on the current data transmission status and the audio playback request, the client sends an audio data request corresponding to the data transmission status, and the server returns the target audio stream data corresponding to the data transmission status. This eliminates the need for the server to determine the data transmission status, thereby improving data transmission efficiency. The client can pre-determine the corresponding decoding parameters based on the audio data request and directly parse the received target audio stream data, thus improving audio playback efficiency.

[0086] It is understood that in the specific implementation of this application, data such as audio data, user data, network data, and server data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use, and processing of related data, as well as the training, deployment, and invocation of algorithm models, must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0087] Optionally, in the step of obtaining the data transmission status with the server, the following steps can be taken: obtaining the current network status; sending a current server status parameter request to the server; receiving the current server status parameters returned by the server in response to the current server status parameter request; and determining the data transmission status with the server based on the current network status and the current server status parameters.

[0088] In this embodiment of the invention, the client can detect key indicators such as network latency and bandwidth. Network latency reflects the time required for data to travel from the sender to the receiver, while bandwidth determines the amount of data that can be transmitted per unit time.

[0089] The client can send a request for current server status parameters to the server to obtain the server's current status parameters. This request can include client identification information, such as a token, current timestamp, and client connection buffer size. Upon receiving the request, the server will return corresponding current server status parameters based on its own status, such as service load and current connection buffer size. These current server status parameters reflect the server's current operational status and processing capacity. The server can dynamically adjust the returned status parameters by monitoring its own performance metrics (such as CPU utilization and memory usage).

[0090] After receiving the current server status parameters returned by the server, the client can combine them with the previously obtained current network status to comprehensively determine the data transmission status between the client and the server. If the network status is good and the server load is light, the data transmission status can be determined as normal; conversely, if the network status is poor or the server load is too heavy, the data transmission status can be determined as abnormal.

[0091] In one possible implementation, when obtaining the current network status, in addition to directly detecting latency and bandwidth, more detailed network information can be obtained through network diagnostic tools or APIs. Simultaneously, when communicating with the server, in addition to using the HTTP protocol, other communication protocols, such as WebSocket, can be used to achieve more efficient bidirectional communication.

[0092] By comprehensively considering the current network and server status parameters, the client can more accurately determine the current data transmission status and generate corresponding request strategies accordingly. This can improve the quality and stability of audio playback and reduce the pressure on the network and server to some extent.

[0093] In one possible implementation, the client obtains current network status data (network latency, bandwidth) and basic parameters of the server (connection buffer, load), which are visible to the client, using a polling mechanism (10 seconds). The specific process is as follows: Figure 4 As shown:

[0094] Step 1: The client sends a request to the server's / health interface:

[0095] Send parameters: (1: client token, 2: current timestamp, 3: client connection buffer size). The client header message body needs to include the login authorization token. The client polls and updates the connection status information.

[0096] Step ②, the server returns parameters: service load, current connection buffer size, and several other key parameters affecting transmission efficiency.

[0097] I. Use tokens for associated storage on the server side: (1: Client buffer size, 2: Server connection buffer size.)

[0098] II. The server dynamically stores basic information such as the current server load (memory, CPU, I / O, number of connections), which are key parameters used by the gateway for intelligent scheduling.

[0099] The client calculates the round-trip network latency by using the (receive time - send time) time based on the packet body time returned by the server.

[0100] Step 3, audio processing request.

[0101] Step 4: The gateway service intelligently and dynamically forwards data to the application server based on the load of each application server (connection count > IO > memory > CPU).

[0102] Step 5: Return the audio data for the corresponding state.

[0103] Optionally, in the step of generating an audio data request corresponding to the current data transmission state, if the current data transmission state is normal, an audio data request under normal transmission state is generated; if the current data transmission state is abnormal, an audio data request under abnormal transmission state is generated, wherein the compression bit rate of the audio data request under abnormal transmission state is lower than that of the audio data request under normal transmission state.

[0104] In this embodiment of the invention, it should be noted that the current data transmission status refers not only to network connectivity, but also includes multiple dimensions such as network bandwidth, latency, server load, and connection buffer size. The current data transmission status determines the efficiency and stability of data transmission. For example, in a high-bandwidth, low-latency network environment, when the server load is light, the data transmission status can be considered normal. Conversely, if network bandwidth is limited, latency is high, or the server load is heavy, then the data transmission status can be considered abnormal.

[0105] When the data transmission status is determined to be normal, the client can generate an audio data request under normal transmission conditions. Specifically, the client can request data according to standard audio quality and bitrate. In this way, due to good network and server conditions, the client can expect to receive a high-quality, uninterrupted audio stream.

[0106] When data transmission is deemed abnormal, the client can generate an audio data request for abnormal transmission status. Specifically, this request can ask the server to transmit audio data at a lower compression bitrate. By reducing the audio data compression bitrate, the client can receive the data faster and play it smoothly.

[0107] In one possible implementation, the client can also utilize adaptive streaming technologies such as DASH (Dynamic Adaptive Streaming over HTTP). DASH allows the client to dynamically select different quality audio streams based on current network conditions. This means that when network conditions improve, the client can seamlessly switch to a higher quality audio stream, and vice versa.

[0108] Optionally, the target audio stream data includes segmented audio stream data. In the step of decoding and playing the target audio stream data according to the decoding parameters corresponding to the audio data request, the current decoding parameters can be determined based on the current data transmission status; based on the current decoding parameters, the received segmented audio stream data can be decoded, and the decoded segmented audio stream data can be played; during the playback of the decoded segmented audio stream data, the remaining segmented audio stream data of the target audio stream data can continue to be received.

[0109] In this embodiment of the invention, the data transmission status is considered to be affected by various factors, including network bandwidth, latency, server load, and client processing capabilities. If the corresponding decoding parameters are looked up only after the target audio stream data is received, it will result in a long waiting time when playback begins. In this embodiment, after determining the data transmission status, the client can then determine the corresponding decoding parameters. This allows the target audio stream data to be decoded by matching the appropriate decoding parameters before receiving the target audio stream data.

[0110] Different data transmission states correspond to different decoding parameters. For example, when network bandwidth is low or server load is heavy, the decoder may choose lower decoding complexity and bit rate to reduce the pressure on data transmission and decoding. Conversely, when network conditions are good and server load is light, higher decoding quality and bit rate can be selected to provide a better audio experience.

[0111] After determining the decoding parameters, you can wait for the target audio stream data to be received. Once the target audio stream data is received, the decoding parameters are used to decode the segmented audio stream data in the target audio stream data and convert it into a format that can be played by the player.

[0112] Once decoding is complete, the player will immediately begin playing the decoded audio data. This streaming playback, where audio is received and played simultaneously, allows playback to begin even before the data has fully downloaded. This significantly improves the user experience, especially when dealing with large audio files or unstable network conditions.

[0113] While playing the decoded audio data, it also continues to receive the remaining segments of the target audio stream data, thereby ensuring the continuity of audio playback and preventing playback interruptions due to insufficient data.

[0114] In one possible implementation, the player might employ buffering technology, which involves pre-downloading and caching several subsequent audio segments while playing the current audio segment. This way, even in the event of network fluctuations or temporary interruptions in data transmission, the player can continue playing audio from the cache, thus maintaining playback continuity.

[0115] like Figure 5 As shown, Figure 5 This is a flowchart of an audio transmission method provided by an embodiment of the present invention, which includes the following steps:

[0116] 201. Receive audio data requests sent by the client.

[0117] In this embodiment of the invention, the audio playback request can be a playback request generated by the user clicking on an audio object in the playback interface, or it can be a playback request input by the user through other modal commands, such as voice commands, gesture commands, etc.

[0118] It should be noted that in this embodiment of the invention, different data transmission states correspond to different audio request parameters. After receiving the user's audio playback request, the corresponding audio request parameters are found according to the current data transmission state, and an audio data request corresponding to the current data transmission state is generated based on the found audio request parameters.

[0119] The data transmission states described above can be categorized as normal and abnormal. When the data transmission state is normal, an audio data request for normal transmission can be generated. When the data transmission state is abnormal, an audio data request for abnormal transmission can be generated. If the data transmission state is normal, the audio request parameters for normal transmission are retrieved, and an audio data request for normal transmission is generated based on these parameters. If the data transmission state is abnormal, the audio request parameters for abnormal transmission are retrieved, and an audio data request for abnormal transmission is generated based on these parameters.

[0120] The audio request parameters mentioned above may include corresponding parameter requests such as encoder, bit rate, format, and adaptation set.

[0121] 202. Locate the target audio data corresponding to the audio data request, and process the target audio data based on the audio data request to obtain the target audio stream data.

[0122] In this embodiment of the invention, after receiving an audio data request, the server parses the audio data request and determines the target audio data stream that the user wants to play based on the parsing result.

[0123] In one possible embodiment, the client and server include a gateway server and an application server. After receiving an audio data request, the client sends the audio data request to the gateway server through the network (external network) between the client and the gateway server. The gateway server forwards the audio data request to the corresponding application server through the network (internal network) between the gateway server and the server. The application server parses the audio data and determines the target audio data stream that the user wants to play based on the parsing result.

[0124] 203. Return the target audio stream data to the client.

[0125] After locating the target audio data stream, the server can send the target audio data stream to the client via the network between the client and the server.

[0126] In this embodiment of the invention, the client monitors the data transmission status between the client and the server, and generates a corresponding audio data request based on the current data transmission status and the audio playback request. The client sends an audio data request corresponding to the data transmission status, and the server returns the target audio stream data corresponding to the data transmission status. The server does not need to determine the data transmission status, thereby improving data transmission efficiency. The client can predetermine the corresponding decoding parameters based on the audio data request and directly parse the received target audio stream data, thereby improving audio playback efficiency.

[0127] Optionally, before receiving the audio data request sent by the client, a current server status parameter request can also be received from the client; based on the current server status parameter request, the current server status parameters can be queried; and the current server status parameters can be returned to the client.

[0128] In this embodiment of the invention, the client can send a current server status parameter request to the server to obtain the server's current status parameters. The current server status parameter request may include the client's identification information, such as a token, current timestamp, and client connection buffer size. Upon receiving the current server status parameter request, the server will return corresponding current server status parameters based on its own status, such as service load and current connection buffer size. The current server status parameters reflect the server's current operating status and processing capacity. The server can dynamically adjust the returned status parameters by monitoring its own performance metrics (such as CPU utilization and memory usage).

[0129] The client sends a request to the server's / health interface:

[0130] Send parameters: (1: client token, 2: current timestamp, 3: client connection buffer size). The client header message body needs to include the login authorization token. The client polls and updates the connection status information.

[0131] The server returns several key parameters that affect transmission efficiency, including service load and the current connection's buffer size.

[0132] I. Use tokens for associated storage on the server side: (1: Client buffer size, 2: Server connection buffer size.)

[0133] II. The server dynamically stores basic information such as the current server load (memory, CPU, I / O, number of connections), which are key parameters used by the gateway for intelligent scheduling.

[0134] The client calculates the round-trip network latency by using the (receive time - send time) time based on the packet body time returned by the server.

[0135] Optionally, in the step of processing the target audio data based on the audio data request to obtain the target audio stream data, if the audio data request is an audio data request under normal transmission conditions, the target audio data can be processed using a first compression method to obtain the first audio stream data processed by the first compression method; if the audio data request is an audio data request under abnormal transmission conditions, the target audio data can be processed using a second compression method to obtain the second audio stream data processed by the second compression method, wherein the compression bit rate of the second compression method is lower than the compression bit rate of the first compression method; the first audio stream data or the second audio stream data can be segmented to obtain the target audio stream data.

[0136] In this embodiment of the invention, the audio stream file can be sliced ​​based on the system component FFmpeg, and the front end uses Dash.js to receive the sliced ​​audio stream file. The specific process is as follows: Figure 4 As shown:

[0137] 1) The service receives audio stream files from TTS input. Based on the network parameters associated with the current token, it obtains the current data transmission status and performs coarse compression when the network latency is high or the buffer is low, and fine compression when the network is normal or the network is good.

[0138] Fine compression: Fine compression typically uses a higher bit rate to maintain better sound quality while appropriately reducing file size.

[0139] Coarse compression: Coarse compression typically uses a lower bit rate to significantly reduce file size, but may sacrifice some audio quality.

[0140] 2) Use the system component ffmpeg to slice the adapted stream, using the following command parameters:

[0141] ffmpeg multimedia processing framework: -iinput.mp3; -c:a aac; -b:a 128k; -f dash; -adaptation_sets"id=0,streams=a";output.mpd.

[0142] -iinput.mp3: Specifies the input audio file.

[0143] -c:a aac: Specifies the audio encoder as AAC.

[0144] -b:a 128k: Sets the audio bitrate to 128kbps (a key parameter for both fine and coarse compression).

[0145] -f dash: Specifies the output format as DASH.

[0146] -adaptation_sets"id=0,streams=a": Specifies the adaptation set, indicating that this is an audio stream.

[0147] output.mpd: The generated MPD file. The MPD file describes the segmentation information of the audio stream.

[0148] 3) The system component ffmpeg performs the slicing operation based on the parameters input in step 1).

[0149] The following three types of files were obtained:

[0150] Ⅰ. output.mpd: MPD file, which describes the segmentation information of the audio stream.

[0151] II. audio / : Contains segmented audio files, usually in .m4s format.

[0152] III. audio / init.mp4: Initialization segment file.

[0153] On the client side, Dash.js is used to receive Dash format audio stream files (the hardware device uses a VLC player to receive the DASH audio stream).

[0154] Ⅰ: Initialize and load the audio segmentation information configured in the output.mpd file, and receive the audio stream for segmented playback;

[0155] II: Pre-received audio segment;

[0156] III: While playing the audio file expected to be received in the previous step, receive the audio stream to improve playback smoothness;

[0157] IV: Finally, all the audio segments are assembled into a complete audio file.

[0158] like Figure 6 As shown, an embodiment of the present invention provides an audio playback device, which includes:

[0159] The acquisition module 301 is used to acquire the data transmission status with the server.

[0160] The first receiving module 302 is used to receive an audio playback request from a user and, based on the current data transmission status, generate an audio data request corresponding to the current data transmission status.

[0161] The second receiving module 303 is used to send the audio data request to the server and receive the target audio stream data returned by the server based on the audio data request.

[0162] The playback module 304 is used to decode and play the target audio stream data according to the decoding parameters corresponding to the audio data request.

[0163] Optionally, the acquisition module 301 is further configured to acquire the current network status; send a current server status parameter request to the server; receive the current server status parameters returned by the server in response to the current server status parameter request; and determine the data transmission status with the server based on the current network status and the current server status parameters.

[0164] Optionally, the first receiving module 302 is further configured to generate an audio data request under normal transmission status if the current data transmission status is normal; and to generate an audio data request under abnormal transmission status if the current data transmission status is abnormal, wherein the compression bit rate of the audio data request under abnormal transmission status is lower than the compression bit rate of the audio data request under normal transmission status.

[0165] Optionally, the target audio stream data includes segmented audio stream data, and the playback module 304 is further configured to determine the current decoding parameters based on the current data transmission status; decode the received segmented audio stream data based on the current decoding parameters, and play the decoded segmented audio stream data; and continue to receive the remaining segmented audio stream data of the target audio stream data during the playback of the decoded segmented audio stream data.

[0166] like Figure 7 As shown, an embodiment of the present invention provides an audio transmission device, which includes:

[0167] The third receiving module 401 is used to receive audio data requests sent by the client;

[0168] The processing module 402 is used to find the target audio data corresponding to the audio data request, and process the target audio data based on the audio data request to obtain target audio stream data;

[0169] The first return module 403 is used to return the target audio stream data to the client.

[0170] Optionally, the device further includes:

[0171] The fourth receiving module is used to receive requests from the client for the current server status parameters.

[0172] The status parameter query module is used to query the current server status parameters based on the current server status parameter request.

[0173] The second return module returns the current server status parameters to the client.

[0174] Optionally, the processing module 402 is further configured to: if the audio data request is an audio data request under normal transmission conditions, process the target audio data using a first compression method to obtain a first audio stream data processed by the first compression method; if the audio data request is an audio data request under abnormal transmission conditions, process the target audio data using a second compression method to obtain a second audio stream data processed by the second compression method, wherein the compression bit rate of the second compression method is lower than the compression bit rate of the first compression method; and segment the first audio stream data or the second audio stream data to obtain target audio stream data.

[0175] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the audio playback method or audio transmission provided in this invention and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0176] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0177] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. An audio playback method, characterized in that, For a client, the audio playback method includes the following steps: Get the data transmission status between the server and the client; Upon receiving a user's audio playback request, an audio data request corresponding to the current data transmission status is generated based on the current data transmission status. The audio data request is sent to the server, and the target audio stream data returned by the server based on the audio data request is received. The target audio stream data is decoded and played according to the decoding parameters corresponding to the audio data request.

2. The audio playback method as described in claim 1, characterized in that, The acquisition of the data transmission status between the server and the client includes: Get the current network status; Send a request for the current server status parameters to the server; Receive the current server status parameters returned by the server in response to the current server status parameter request; Based on the current network status and the current server status parameters, the data transmission status with the server is determined.

3. The audio playback method as described in claim 1, characterized in that, The step of generating an audio data request corresponding to the current data transmission state includes: If the current data transmission status is normal, then an audio data request under normal transmission status is generated; If the current data transmission state is abnormal, an audio data request under abnormal transmission state is generated, and the compression bit rate of the audio data request under abnormal transmission state is lower than the compression bit rate of the audio data request under normal transmission state.

4. The audio playback method as described in claim 1, characterized in that, The target audio stream data includes segmented audio stream data. Decoding and playing the target audio stream data according to the decoding parameters corresponding to the audio data request includes: Based on the current data transmission status, determine the current decoding parameters; Based on the current decoding parameters, the received segmented audio stream data is decoded, and the decoded segmented audio stream data is played. During the playback of the decoded segmented audio stream data, the remaining segmented audio stream data of the target audio stream data continues to be received.

5. An audio transmission method, characterized in that, For use on the server side, the audio transmission method includes: Receive audio data requests sent by the client; Locate the target audio data corresponding to the audio data request, and process the target audio data based on the audio data request to obtain target audio stream data; The target audio stream data is returned to the client.

6. The audio transmission method as described in claim 5, characterized in that, Before receiving the audio data request sent by the client, the method further includes: Receive the current server status parameter request sent by the client; Based on the current server status parameter request, query the current server status parameters; The current server status parameters are returned to the client.

7. The audio transmission method as described in claim 5, characterized in that, The process of processing the target audio data based on the audio data request to obtain target audio stream data includes: If the audio data request is an audio data request under normal transmission conditions, the target audio data is processed by the first compression method to obtain the first audio stream data processed by the first compression method. If the audio data request is an audio data request under abnormal transmission conditions, the target audio data is processed by the second compression method to obtain the second audio stream data after the second compression method is processed. The compression bit rate of the second compression method is lower than the compression bit rate of the first compression method. The first audio stream data or the second audio stream data is segmented to obtain the target audio stream data.

8. An audio playback device, characterized in that, For the audio playback method as described in any one of claims 1 to 4, the audio playback device comprises: The acquisition module is used to obtain the data transmission status between the server and the client. The first receiving module is used to receive the user's audio playback request and, based on the current data transmission status, generate an audio data request corresponding to the current data transmission status. The second receiving module is used to send the audio data request to the server and receive the target audio stream data returned by the server based on the audio data request. The playback module is used to decode and play the target audio stream data.

9. An audio transmission device, characterized in that, For the audio transmission method as described in any one of claims 5 to 7, the audio transmission device comprises: The third receiving module is used to receive audio data requests sent by the client; The processing module is used to find the target audio data corresponding to the audio data request, and process the target audio data based on the audio data request to obtain target audio stream data; The first return module is used to return the target audio stream data to the client.

10. An audio playback system, characterized in that, include: The client includes the audio playback device as described in claim 8, and the server includes the audio transmission device as described in claim 9.

Images