Multimedia data processing method and device and information sending method and device

Abstract

The embodiment of the invention discloses a multimedia data processing method and device and an information sending method and device. The multimedia data processing method is applied to first electronic equipment and comprises the steps that first buffering information corresponding to second electronic equipment is obtained, and the first buffering information is used for indicating buffering delay of a first buffering area in the second electronic equipment; second buffer information corresponding to a second buffer area in the first electronic equipment is determined, the second buffer area is used for buffering multimedia data to be transmitted to the second electronic equipment, and the second buffer information is used for indicating the data volume of the multimedia data stored in the second buffer area; if it is determined that the second electronic equipment has the playing lagging risk according to the first buffer information and the second buffer information, a target operation is executed, and the target operation is used for reducing the data size of the multimedia data stored in the second buffer area. In the embodiment of the invention, the number of times of lagging when the data receiving end plays the multimedia data can be reduced, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method and device for processing multimedia data, and a method and device for sending information. Background Art

[0002] With the rapid development of Internet technology, the transmission of multimedia data (such as audio data, video data, etc.) between different electronic devices has become increasingly important in various application scenarios. During the transmission of multimedia data, since the environment in which the electronic devices are located often causes signal interference, the data receiving end may experience a freeze when playing multimedia data, which seriously affects the user experience. Summary of the invention

[0003] The embodiments of the present application disclose a method and device for processing multimedia data, a method and device for sending information, an electronic device, and a program product, which can reduce the number of freezes that occur at a data receiving end when playing multimedia data and improve user experience.

[0004] The embodiment of the present application discloses a method for processing multimedia data, which is applied to a first electronic device, wherein the first electronic device establishes a communication connection with a second electronic device, and the first electronic device is used to send multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data; the method includes:

[0005] Acquire first buffer information corresponding to the second electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played;

[0006] Determining second buffer information corresponding to a second buffer in the first electronic device, where the second buffer is used to buffer multimedia data to be transmitted to the second electronic device, and the second buffer information is used to indicate a data amount of the multimedia data stored in the second buffer;

[0007] If it is determined according to the first buffer information and the second buffer information that the second electronic device has a risk of playback freeze, a target operation is performed, where the target operation is used to reduce the amount of multimedia data stored in the second buffer.

[0008] The embodiment of the present application discloses an information sending method, which is applied to a second electronic device, wherein the second electronic device establishes a communication connection with a first electronic device, and the first electronic device is used to send multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data; the method includes:

[0009] Sending first buffering information to the first electronic device, where the first buffering information is used to indicate a buffering delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played;

[0010] The first buffer information is used to trigger the first electronic device to determine the second buffer information corresponding to the second buffer, and to perform a target operation when it is determined that the second electronic device is at risk of playback jamming based on the first buffer information and the second buffer information; the second buffer is used to buffer multimedia data in the first electronic device to be transmitted to the second electronic device, the second buffer information is used to indicate the amount of multimedia data stored in the second buffer, and the target operation is used to reduce the amount of multimedia data stored in the second buffer.

[0011] The embodiment of the present application discloses a multimedia data processing device, which is applied to a first electronic device, wherein the first electronic device establishes a communication connection with a second electronic device, and the first electronic device is used to send multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data; the device includes:

[0012] an acquisition module, configured to acquire first buffer information corresponding to the second electronic device, wherein the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, and the first buffer is used to buffer multimedia data to be played;

[0013] a determining module, configured to determine second buffering information corresponding to a second buffer in the first electronic device, the second buffer being configured to buffer multimedia data to be transmitted to the second electronic device, the second buffering information being configured to indicate a data volume of the multimedia data stored in the second buffer;

[0014] An operation module is used to execute a target operation if it is determined that the second electronic device has a risk of playback jamming according to the first buffer information and the second buffer information. The target operation is used to reduce the amount of multimedia data stored in the second buffer.

[0015] The embodiment of the present application discloses an information sending device, which is applied to a second electronic device, wherein the second electronic device establishes a communication connection with a first electronic device, and the first electronic device is used to send multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data; the device includes:

[0016] a sending module, configured to send first buffering information to the first electronic device, wherein the first buffering information is used to indicate a buffering delay of a first buffer in the second electronic device, and the first buffer is used to buffer multimedia data to be played;

[0017] The first buffer information is used to trigger the first electronic device to determine the second buffer information corresponding to the second buffer, and to perform a target operation when it is determined that the second electronic device is at risk of playback jamming based on the first buffer information and the second buffer information; the second buffer is used to buffer multimedia data in the first electronic device to be transmitted to the second electronic device, the second buffer information is used to indicate the amount of multimedia data stored in the second buffer, and the target operation is used to reduce the amount of multimedia data stored in the second buffer.

[0018] An embodiment of the present application discloses an electronic device, including a memory, a processor, and a transceiver unit. The memory stores a computer program. When the computer program is executed by the processor, the electronic device implements the method described in any of the above embodiments.

[0019] An embodiment of the present application discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor in an electronic device, the electronic device implements the method described in any of the above embodiments.

[0020] An embodiment of the present application discloses a computer program product, including a computer program. When the computer program is executed by a processor in an electronic device, the electronic device implements the method described in any of the above embodiments.

[0021] The embodiments of the present application disclose a method and device for processing multimedia data, a method and device for sending information, an electronic device, and a program product. A first electronic device establishes a communication connection with a second electronic device. The first electronic device serves as a data sending end and the second electronic device serves as a data receiving end. The first electronic device sends multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data. The first electronic device obtains first buffer information corresponding to the second electronic device, and the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device. The first buffer is used to buffer multimedia data to be played. Second buffer information corresponding to a second buffer in the first electronic device is determined. The second buffer is used to buffer multimedia data to be transmitted to the second electronic device. The second buffer information is used to indicate the amount of multimedia data stored in the second buffer. If the first electronic device determines that the second electronic device has a risk of playback jamming based on the first buffer information and the second buffer information, a target operation is performed, and the target operation is used to reduce the amount of multimedia data stored in the second buffer. In an embodiment of the present application, the first electronic device can accurately identify whether the second electronic device is at risk of playback jams based on the buffering delay of the first buffer in the second electronic device and the amount of multimedia data stored in its own second buffer, and when it is identified that the second electronic device is at risk of playback jams, perform a target operation to reduce the amount of multimedia data stored in the second buffer, reduce network congestion, and thus improve the transmission stability of the multimedia data, which can reduce the number of times a data receiving end experiences jams when playing multimedia data, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1A A schematic diagram of audio data transmission in the related art;

[0024] Figure 1B A schematic diagram of data transmission between a data sending end and a data receiving end in the related art;

[0025] Figure 2 It is an application scenario diagram of a method for processing multimedia data and a method for sending information in one embodiment;

[0026] Figure 3 is a flowchart of a method for processing multimedia data in one embodiment;

[0027] Figure 4is a flowchart of a method for processing multimedia data in another embodiment;

[0028] Figure 5A is a timing diagram of interaction between a first electronic device and a second electronic device in one embodiment;

[0029] Figure 5B is a timing diagram of interaction between a first electronic device and a second electronic device in another embodiment;

[0030] Figure 6 is a flowchart of a method for processing multimedia data in another embodiment;

[0031] Figure 7 is a flowchart of a method for processing multimedia data in another embodiment;

[0032] Figure 8 is a flow chart of a method for sending information in one embodiment;

[0033] Fig. 9 is a block diagram of a multimedia data processing device in one embodiment;

[0034] Fig.10 is a block diagram of an information sending device in one embodiment;

[0035] Fig.11 FIG. 4 is a structural block diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0038] It is understood that the terms "first", "second", etc. used in this application may be used in this article to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first electronic device may be referred to as a second electronic device, and similarly, a second electronic device may be referred to as a first electronic device. Both the first electronic device and the second electronic device are electronic devices, but they are not the same electronic device. The term "plurality" used in this application refers to two or more. The term "and / or" used in this application refers to one of the schemes, or any combination of multiple schemes.

[0039] During the transmission of multimedia data, the data sending end encodes the multimedia data to obtain a data packet, and then sends the data packet to the data receiving end through the communication connection with the data receiving end. After receiving the data packet sent by the data sending end, the data receiving end decodes the data packet to obtain the multimedia data to be played, and then plays the multimedia data.

[0040] Take the scenario of audio data transmission based on Bluetooth communication protocol as an example. Figure 1A As shown, the data sending end (i.e., the Source end) encodes the original audio data according to the negotiated encoding parameters through the Bluetooth audio encoder, and then encapsulates it into a data packet through the L2CAP (Logical Link Control and Adaptation Protocol) layer of the Bluetooth protocol stack, the Bluetooth Controller, etc., and transmits it to the data receiving end (i.e., the Sink end) through a wireless channel. The Sink end receives the data packet sent by the Source end, and decapsulates the data packet through the Bluetooth Controller, the L2CAP layer, etc., and then the audio decoder decodes the audio data carried in the data packet to obtain the original audio data, and then the original audio data is played through the playback module (such as an audio amplification unit, a speaker, etc.).

[0041] For example, Figure 1B FIG. 1 is a schematic diagram of data transmission between a data transmitting end and a data receiving end in the related art. Figure 1BAs shown, the Source sends a data packet to the Sink. After receiving the data packet, if the Sink parses the data packet normally, it will reply an ACK (Acknowledgment) signal to the Source. The Source can set a timeout. After each data packet sent by the Source, it will wait for the ACK (Acknowledgment) signal from the Sink. If the Source receives an ACK signal within the timeout, it will continue to send the next data packet. If the Source does not receive an ACK signal from the Sink within the timeout or receives a NAK (Negative Acknowledgment) signal, the data packet will be retransmitted.

[0042] Before the Sink starts playing the received multimedia data, it will first cache the multimedia data for a certain length in the playback buffer and then start playing. During the playback process, it will take the cached multimedia data from the playback buffer for playback while parsing the received data packets to obtain the multimedia data and put them into the playback buffer, thereby ensuring that a certain length of multimedia data is stored in the playback buffer, ensuring the smoothness of the playback and improving anti-interference.

[0043] When there is a large signal interference in the network environment where the Source and Sink are located, data packets may be lost. Data packets are prone to long-term and frequent retransmission, resulting in data packets accumulating in the transmission buffer of the Source, such as the Bluetooth Buffer in the Source. The Sink cannot receive data packets. If the signal interference is strong or lasts for a long time, it will cause the playback of the Sink to freeze. The main reasons for the freeze are as follows:

[0044] Reason 1: Due to signal interference in the network environment, the Sink end cannot receive data packets in time, and therefore cannot put multimedia data into the playback buffer in time. When the multimedia data in the playback buffer of the Sink end is completely taken out, there is no buffered multimedia data in the playback buffer, which will cause the playback of the Sink end to be stuck.

[0045] Reason 2: Due to signal interference in the network environment, data packets have been accumulating in the transmission buffer of the Source. As the amount of data in the transmission buffer of the Source increases, a flush operation will be triggered when it exceeds a certain limit. The flush operation will clear the data in the transmission buffer, resulting in discontinuity of multimedia data sent from the Source to the Sink, which causes playback jams on the Sink.

[0046] When the Source performs a flush operation on the data in the transmission buffer, it usually only considers the number of data packets cached in the transmission buffer of the Source, which has a large lag and may cause the situations described in the above reasons 1 and 2 to occur continuously, causing multiple playback freezes on the Sink side, seriously affecting the user experience.

[0047] The multimedia data processing method and device, information sending method and device, electronic device, and program product provided in the embodiments of the present application can reduce the number of freezes that occur at the data receiving end when playing multimedia data, thereby improving user experience.

[0048] Figure 2 FIG. 1 is an application scenario diagram of a method for processing multimedia data and a method for sending information in one embodiment. Figure 2 As shown, the multimedia data processing method can be applied to the first electronic device 210, and the information sending method can be applied to the second electronic device 220.

[0049] Among them, the first electronic device 210 may include but is not limited to mobile phones, wearable devices (such as smart watches, smart glasses, etc.), vehicle-mounted terminals, tablet computers, laptop computers, PC (Personal Computer) devices, and the second electronic device 220 may include but is not limited to mobile phones, wearable devices, vehicle-mounted terminals, tablet computers, laptop computers, PCs, headphones, smart speakers, smart home devices, etc.

[0050] The first electronic device 210 and the second electronic device 220 may establish a communication connection, which may include but is not limited to a Wi-Fi connection, a Bluetooth connection, etc., and data may be transmitted through the communication connection. The first electronic device 210 may be a data transmitter, and the second electronic device 220 may be a data receiver. The first electronic device 210 may send multimedia data to the second electronic device 220 through the communication connection, and the second electronic device 220 may play the received multimedia data.

[0051] When the first electronic device 210 establishes a communication connection with the second electronic device 220, the first electronic device 210 may obtain first buffer information corresponding to the second electronic device, and the first buffer information may be used to indicate the buffer delay of the first buffer in the second electronic device 220, and the first buffer is used to buffer the multimedia data to be played. The first electronic device 210 determines the second buffer information corresponding to its own second buffer, and the second buffer may be used to buffer the multimedia data to be transmitted to the second electronic device, and the second buffer information is used to indicate the amount of multimedia data stored in the second buffer. The first electronic device 210 may determine whether the second electronic device 220 has a risk of playback jamming according to the first buffer information and the second buffer information. If it is determined that the second electronic device 220 has a risk of playback jamming, a target operation may be performed, and the target operation may be used to reduce the amount of multimedia data stored in the second buffer, which may alleviate network congestion and achieve more efficient multimedia data transmission, thereby improving the transmission stability of multimedia data, reducing the number of jams that occur when the data receiving end plays multimedia data, and improving user experience.

[0052] It should be noted that Figure 2 Only one possible application scenario of the embodiment of the present application is shown. In actual applications, the first electronic device 210 can also establish communication connections with multiple second electronic devices 220 respectively, and play multimedia data through the multiple second electronic devices 220.

[0053] like Figure 3 As shown, in one embodiment, a method for processing multimedia data is provided, which can be applied to the first electronic device mentioned above, and the method may include the following steps:

[0054] Step 310: Acquire first buffer information corresponding to the second electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played.

[0055] When a communication connection is established between a first electronic device and a second electronic device, the first electronic device can be used as a data transmitter, and the second electronic device can be used as a data receiver. The first electronic device can send multimedia data to the second electronic device, and the multimedia data can be played through the second electronic device. The multimedia data can include but is not limited to video data, audio data, etc. Exemplarily, the scenario of playing multimedia data can include but is not limited to screen projection, and the scenario of playing audio (such as music, voice, etc.) with an audio output device (such as headphones, speakers, etc.).

[0056] When the first electronic device plays multimedia data through the second electronic device, the first electronic device can obtain first buffer information corresponding to the second electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer zone in the second electronic device.

[0057] The first buffer may be a buffer in the second electronic device for buffering multimedia data to be played. In some embodiments, the second electronic device receives a data packet sent by the first electronic device, and the data packet carries multimedia data. The second electronic device may parse the data packet and decode the multimedia data carried in the data packet to obtain the multimedia data to be played, and then cache the multimedia data to be played in the first buffer. During the process of playing audio data, the second electronic device may read the multimedia data to be played from the first buffer for playback, while parsing the received data packet, decoding the multimedia data, etc., and storing the multimedia data to be played in the first buffer.

[0058] For example, the second electronic device receives an audio data packet sent by the first electronic device, and can decode the audio data carried in the audio data packet to obtain PCM (Pulse Code Modulation) data, and can cache the PCM data in the first buffer; when the second electronic device plays the audio data, it can read the PCM data from the first buffer, and then use DAC (Digital to Analog converter) and AMP (Amplifier for Power) to perform digital-to-analog conversion and power amplification to obtain analog data, and play the analog data through a speaker. It should be noted that the multimedia data to be played stored in the first buffer can also be in other formats, such as the original code stream of audio data, or the image data of a video, etc., which is not limited here.

[0059] After the second electronic device starts to receive the multimedia data sent by the first electronic device, the first buffer needs to accumulate a certain amount of multimedia data to be played before it starts playing. The buffering delay of the first buffer may refer to the delay between the multimedia data to be played entering the first buffer and being taken out of the first buffer. Optionally, the buffering delay may be represented by the buffering delay duration, which may refer to the duration that the multimedia data stays in the first buffer, that is, the duration between the multimedia data entering the first buffer and being taken out.

[0060] Optionally, the buffering delay can also be expressed by the amount of buffered data. The buffered data amount may refer to the amount of data waiting to be retrieved after the multimedia data enters the first buffer. It can also be understood as the amount of multimedia data stored in the first buffer. Exemplarily, the buffered data amount can be expressed in bytes or samples.

[0061] The buffering delay of the first buffer may represent the buffer size of the first buffer. The larger the buffer size of the first buffer is, the larger the buffering delay of the first buffer is; and the smaller the buffer size of the first buffer is, the smaller the buffering delay of the first buffer is.

[0062] In some embodiments, the buffering delay of the first buffer can be a pre-set fixed value, and the second electronic device can pre-set the buffering delay of the first buffer, for example, the buffering delay duration corresponding to the first buffer can be set. When the second electronic device normally receives the multimedia data packet sent by the first electronic device (without signal interference), the multimedia data entering the first buffer needs to wait for the buffering delay duration before being taken out for playback.

[0063] In some embodiments, the buffering delay of the first buffer may also be a dynamically changing value. The second electronic device may dynamically adjust the buffering delay of the first buffer according to the network environment. When there is signal interference in the network environment, a larger buffering delay may be set to cope with the situation of data packet loss in the case of signal interference and reduce playback jams. When there is no signal interference in the network environment or the signal strength between the second electronic device and the first electronic device is strong (indicating a better network environment), a smaller buffering delay may be set to reduce the playback delay of the second electronic device.

[0064] Optionally, the buffering delay of the first buffer may be relatively fixed at a certain stage when the second electronic device plays multimedia data. For example, the buffering delay of the first buffer may be fixed each time the second electronic device plays multimedia data; or, during the process of the second electronic device playing multimedia data, the buffering delay of the first buffer may be fixed in a certain time period.

[0065] In some embodiments, the first electronic device may obtain the first buffer information corresponding to the second electronic device when triggering the second electronic device to play multimedia data. When the first electronic device establishes a communication connection with the second electronic device, the first electronic device may send a start instruction to the second electronic device and obtain the first buffer information corresponding to the second electronic device. The start instruction may be used to notify the second electronic device to start transmitting and playing multimedia data. In other words, the first electronic device may obtain the first buffer information corresponding to the second electronic device before the second electronic device plays the multimedia data.

[0066] In some embodiments, since the buffering delay of the first buffer may change, the first electronic device may obtain the first buffering information corresponding to the second electronic device during the process of the second electronic device playing multimedia data. For example, the first electronic device may obtain the real-time first buffering information of the second electronic device at preset intervals; or, the second electronic device may feedback the first buffering information to the first electronic device when the buffering delay of the first buffer changes significantly, etc., which can improve the accuracy of the first buffering information obtained by the first electronic device, thereby further improving the accuracy of the subsequent execution of the target operation.

[0067] Step 320, determining second buffer information corresponding to a second buffer in the first electronic device, where the second buffer is used to buffer multimedia data to be transmitted to the second electronic device, and the second buffer information is used to indicate the data volume of the multimedia data stored in the second buffer.

[0068] During the process of transmitting multimedia data from the first electronic device to the second electronic device, the first electronic device may monitor the second buffer and obtain second buffer information corresponding to the second buffer. The second buffer is used in the first electronic device to buffer multimedia data to be transmitted to the second electronic device, and the multimedia data to be transmitted to the second electronic device may be a data packet obtained after multimedia encoding.

[0069] Exemplarily, the first electronic device sends multimedia data to the second electronic device via a Bluetooth communication connection, and the second buffer may include a buffer in a Bluetooth Controller and / or a buffer in a Bluetooth Host (Bluetooth host), etc. Further, taking the sending of audio data as an example, after the first electronic device obtains the original audio data from the application, it can encode the original audio data through the Bluetooth audio encoder in the Bluetooth Host to obtain an audio data packet in a format such as SBC (Subband Codec) and AAC (Advanced Audio Coding) suitable for Bluetooth transmission, and store the audio data packet in the buffer of the Bluetooth Host. The Bluetooth Host can communicate with the Bluetooth Controller, and the Bluetooth Host sends the audio data packet in the buffer to the Bluetooth Controller. The Bluetooth Controller can cache the audio data packet sent by the Bluetooth Host in its own buffer, and take out the audio data packet from the buffer for sending.

[0070] The second buffer stores multiple data packets to be sent, and each data packet can carry multimedia data. In the related art, the first electronic device obtains the number of data packets stored in the second buffer, and determines whether there is data accumulation in the second buffer based on the number of data packets. If the number of data packets is small, it means that there is no data accumulation in the second buffer. If the number of data packets is large, it means that there is data accumulation in the second buffer, and the first electronic device will perform rate adaptation or flush operation on the data in the second buffer. Since the amount of multimedia data carried in each data packet may be variable and not fixed, the data storage situation of the second buffer is analyzed using the number of data packets stored in the second buffer, resulting in inaccurate analysis results, which in turn causes the subsequent rate adaptation, flushing operations, etc. performed by the first electronic device to be inaccurate.

[0071] In the embodiment of the present application, the first electronic device obtains the second buffer information of the second buffer in real time, and can judge whether there is data accumulation in the second buffer according to the second buffer information. The second buffer information can be used to characterize the data volume of the multimedia data actually stored in the second buffer, and can also be understood as the sum of the data volume of the multimedia data carried by each data packet stored in the second buffer.

[0072] Optionally, the second buffer information may include the buffer duration of the second buffer, which may refer to the playback duration corresponding to the multimedia data stored in the second buffer, and may be the sum of the playback durations corresponding to the multimedia data carried by each data packet stored in the second buffer. Furthermore, each time the first electronic device stores a data packet in the second buffer, it may add the playback duration of the multimedia data carried by the stored data packet to the current buffer duration of the second buffer, and each time the first electronic device retrieves a data packet from the second buffer, it may deduct the playback duration of the multimedia data carried by the retrieved data packet from the current buffer duration of the second buffer, and update the buffer duration of the second buffer in real time to ensure the accuracy of the second buffer information.

[0073] Optionally, the second buffer information may include the number of bytes or samples corresponding to the multimedia data stored in the second buffer, etc. The second buffer information may be the sum of the number of bytes or the sum of the number of samples corresponding to the multimedia data carried by each data packet stored in the second buffer, etc. Further, each time the first electronic device stores a data packet in the second buffer, the number of bytes or the number of samples of the multimedia data carried by the stored data packet may be added to the current number of bytes or the number of samples in the second buffer, and each time the first electronic device retrieves a data packet from the second buffer, the number of bytes or the number of samples of the multimedia data carried by the retrieved data packet may be deducted from the current number of bytes or the number of samples in the second buffer, and the number of bytes or the number of samples corresponding to the multimedia data stored in the second buffer may be updated in real time to ensure the accuracy of the second buffer information.

[0074] Compared with the related art that directly uses the number of data packets to reflect the storage status of the second buffer, the embodiment of the present application uses the buffering time of the second buffer or the number of bytes or samples corresponding to the multimedia data stored in the second buffer to reflect the storage status of the second buffer, which can more accurately analyze whether there is data accumulation in the second buffer.

[0075] In some other embodiments, the first electronic device may also obtain the second buffer information corresponding to the second buffer according to a preset cycle duration. The cycle duration may be set according to actual needs and is not limited here.

[0076] Step 330: If it is determined according to the first buffer information and the second buffer information that the second electronic device has a risk of playback freeze, a target operation is performed, where the target operation is used to reduce the amount of multimedia data stored in the second buffer.

[0077] The first electronic device can determine whether the second electronic device has a risk of playback jamming based on the first buffer information and the second buffer information. In some embodiments, when there is no signal interference or the signal interference is small in the network environment, the multimedia data can be transmitted to the first electronic device in a timely manner. Therefore, the amount of multimedia data cached in the second buffer is small, and the difference from the buffer delay of the first buffer indicated by the first buffer information is large. At this time, the first buffer in the first electronic device stores enough multimedia data to be played, and there will be no playback jamming. In the case of strong signal interference in the network environment, due to frequent packet loss and retransmission, the multimedia data to be sent will accumulate in the second buffer, so that the amount of multimedia data stored in the second buffer is close to the buffer delay of the first buffer indicated by the first buffer information. At this time, the multimedia data to be played stored in the first buffer of the first electronic device is insufficient, and playback jamming is prone to occur.

[0078] Therefore, it can be determined whether the second buffer information is close to the first buffer information. If the second buffer information is close to the first buffer information, it means that the amount of multimedia data stored in the second buffer in the first electronic device is close to the buffer delay of the first buffer in the second electronic device, which means that the amount of multimedia data buffered in the first buffer in the second electronic device is close to zero (that is, the first buffer is about to be read empty), and the second electronic device is very likely to experience playback jams, and there is a risk of playback jams. Combining the storage situation of the second buffer in the first electronic device and the buffer delay of the first buffer in the second electronic device can accurately identify the risk of playback jams in the second electronic device, thereby improving the accuracy of identifying the risk of playback jams.

[0079] Moreover, the amount of multimedia data stored in the second buffer in the first electronic device is close to the buffering delay of the first buffer in the second electronic device, which may also indicate that there is a certain amount of data accumulation in the second buffer.

[0080] When the first electronic device determines that there is a risk of playback jams in the second electronic device, the first electronic device may execute a target operation, which may be an operation that can reduce the amount of multimedia data stored in the second buffer. By executing the target operation, the amount of multimedia data stored in the second buffer can be reduced, thereby alleviating data accumulation in the second buffer and reducing the amount of multimedia data waiting to be transmitted, thereby alleviating network congestion.

[0081] In some embodiments, the target operation may include a rate reduction operation and / or a data cleanup operation.

[0082] The bit rate reduction operation may refer to an operation of reducing the bit rate corresponding to the multimedia encoder of the first electronic device, and the bit rate may refer to the amount of multimedia data transmitted per unit time, which may be expressed in bits per second (bps) or kilobits per second (kbps), etc. By reducing the bit rate corresponding to the multimedia encoder of the first electronic device, the amount of multimedia data obtained after encoding by the multimedia encoder may be reduced, thereby reducing the amount of multimedia data stored in the second buffer per unit time, and alleviating the data accumulation in the second buffer.

[0083] As an implementation method, multiple bit rate gears can be pre-set, and the bit rates of multimedia encoders corresponding to different bit rate gears are different. When it is detected that the second electronic device has a risk of playback jamming, the first electronic device can switch the multimedia encoder from the first bit rate gear to the second bit rate gear, and the bit rate corresponding to the second bit rate gear is less than the bit rate corresponding to the first bit rate gear. The first bit rate gear is the bit rate gear corresponding to the multimedia encoder before the second electronic device is identified to have a risk of playback jamming. Optionally, the first bit rate gear can be determined according to the application scenario, encoding format, etc. The application scenario may include but is not limited to music playback, video playback, and other scenes with high quality requirements for multimedia data, or games, calls, etc., which have high delay requirements for multimedia data (requiring low delay). By setting multiple bit rate gears to achieve flexible adjustment of the bit rate of the multimedia encoder, it can not only alleviate the data accumulation in the second buffer, but also simplify the adjustment method and reduce unnecessary overhead.

[0084] As another way, the first electronic device may also reduce the bit rate corresponding to the multimedia encoder according to the data accumulation speed of the second buffer. The data accumulation speed and the bit rate corresponding to the multimedia encoder may be negatively correlated. The faster the data accumulation speed, the smaller the bit rate corresponding to the multimedia encoder. The data accumulation speed may refer to the amount of multimedia data newly added to the second buffer per unit time. For example, when the first electronic device identifies that the second electronic device is at risk of playback jamming, the data accumulation speed may be determined based on the difference between the playback duration corresponding to the multimedia data stored in the second buffer per unit time and the playback duration corresponding to the multimedia data taken out per unit time, and the bit rate reduction value may be determined based on the data accumulation speed (the bit rate reduction value may be positively correlated with the data accumulation speed, the faster the data accumulation speed, the larger the bit rate reduction value), the current bit rate of the multimedia encoder may be subtracted from the bit rate reduction value to obtain the bit rate after the multimedia encoder is reduced, and the multimedia encoder may be controlled to encode the multimedia data according to the reduced bit rate. The bit rate of the multimedia encoder is flexibly adjusted according to the data accumulation speed, which improves the accuracy of the bit rate adjustment of the multimedia encoder, further alleviates the data accumulation in the second buffer, and reduces the risk of playback jams on the second electronic device.

[0085] The data clearing operation may refer to an operation of clearing at least part of the multimedia data stored in the second buffer. Further, the data clearing operation may be used to clear all multimedia data stored in the second buffer, or may be used to clear part of the multimedia data stored in the second buffer. The data clearing operation may be a flush operation on the data in the second buffer, deleting all or part of the data stored in the second buffer, thereby alleviating the data accumulation in the second buffer.

[0086] In the related art, when the first electronic device performs a flush operation, it only considers the number of data packets stored in the second buffer, and completely ignores the situation of the first buffer of the second electronic device, which results in a large lag. In addition, since the flush operation of the first electronic device may cause playback jams on the second electronic device, and the data in the first buffer of the second electronic device is cleared, playback jams may also be caused. Therefore, the second electronic device may experience multiple playback jams, seriously affecting the user experience.

[0087] In an embodiment of the present application, when the first electronic device determines that there is a risk of playback jams in the second electronic device, the first electronic device performs a flush operation on the second buffer, and the playback jams caused by the flush operation on the first electronic device and the playback jams caused by insufficient multimedia data cached in the first buffer of the second electronic device are performed at the same stage or even at the same time, which can reduce the number of playback jams felt by the user.

[0088] In an embodiment of the present application, the first electronic device can accurately identify whether the second electronic device is at risk of playback jams based on the buffering delay of the first buffer in the second electronic device and the amount of multimedia data stored in its own second buffer, and when it is identified that the second electronic device is at risk of playback jams, perform a target operation to reduce the amount of multimedia data stored in the second buffer, reduce network congestion, and thus improve the transmission stability of the multimedia data, which can reduce the number of times a data receiving end experiences jams when playing multimedia data, thereby improving the user experience.

[0089] like Figure 4 As shown, in another embodiment, a method for processing multimedia data is provided, and the method may include the following steps:

[0090] Step 402: Acquire first buffer information corresponding to the second electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played.

[0091] In some embodiments, the first buffer information may be sent directly from the second electronic device to the first electronic device. When the first electronic device establishes a communication connection with the second electronic device, the second electronic device may determine the first buffer information corresponding to the first buffer, which may include the buffer delay time of the first buffer in the second electronic device, and send the first buffer information to the first electronic device, and the first electronic device receives the first buffer information sent by the second electronic device.

[0092] Furthermore, the first electronic device and the second electronic device may support a private communication protocol, and the private communication protocol may define a transmission field corresponding to the first buffer information. The second electronic device may send a private data packet to the first electronic device based on the private communication protocol, and the private data packet carries the first buffer information. The first electronic device may parse the received private data packet based on the private communication protocol to obtain the first buffer information. Based on the private communication protocol, the second electronic device may accurately send the first buffer information to the first electronic device, thereby improving the accuracy of the first buffer information obtained by the first electronic device, and further improving the accuracy of identifying whether the second electronic device has a risk of playback jamming.

[0093] For example, Figure 5A FIG. 1 is a timing diagram of interaction between a first electronic device and a second electronic device in an embodiment. Figure 5A As shown, the first electronic device establishes a Bluetooth communication connection with the second electronic device. The second electronic device can send a buffering delay time to the first electronic device based on a private communication protocol, and the first electronic device receives the buffering delay time sent by the second electronic device. The first electronic device can encode multimedia data and send the multimedia data to the second electronic device. The second electronic device receives the multimedia data and decodes and plays the multimedia data. The first electronic device can monitor the buffering time corresponding to the second buffer. If it is determined that the second electronic device has a risk of playback jamming based on the buffering time and the buffering delay time, the target operation is performed.

[0094] In some embodiments, the first buffer information may be calculated by the first electronic device based on the delay value sent by the second electronic device. When the first electronic device plays multimedia data through the second electronic device, the second electronic device may send a delay value to the first electronic device, and the delay value may be used to characterize the delay of the second electronic device in processing the received multimedia data, that is, the delay value may be used to characterize the time between the second electronic device receiving the data packet and playing the multimedia data carried in the data packet.

[0095] After receiving the data packet sent by the first electronic device, the second electronic device parses the data packet to obtain the multimedia data carried by the data packet, decodes the multimedia data, and then caches the decoded multimedia data in the second buffer to wait for being retrieved and played. Therefore, the delay of the second electronic device in processing the received multimedia data may include decoding delay and buffering delay, wherein the decoding delay can be used to characterize the delay caused by the second electronic device decoding the multimedia data through the multimedia decoder. Furthermore, the above-mentioned delay value may include the buffering delay duration and the decoding delay duration.

[0096] The first electronic device receives the delay value sent by the second electronic device, and can determine the first buffer information based on the delay value and the decoding delay. Furthermore, the first electronic device can subtract the decoding delay duration from the delay value to determine the buffer delay duration of the first buffer. Optionally, the decoding delay duration can be a fixed value set based on experience, such as 10ms (milliseconds), 8ms, etc. The decoding delay duration can also be determined based on the codec parameters negotiated between the first electronic device and the second electronic device (such as one or more of the codec format, frame length, sampling rate, and bit depth, etc.), which is not limited here.

[0097] Exemplarily, the first electronic device and the second electronic device transmit multimedia data via a Bluetooth communication connection, and the above-mentioned delay value may be a DelayReport value, which is specified in the Bluetooth standard protocol and is a delay value reported by the data receiving end to the data sending end, and is used to help the data sending end adjust the synchronization of audio and video. In an embodiment of the present application, the first electronic device can calculate the buffering delay duration of the first buffer based on the DelayReport value and the decoding delay duration fed back by the second electronic device, to ensure that the first electronic device can accurately obtain the first buffer information, and can obtain the first buffer information without supporting a private communication protocol, thereby improving the applicability of the solution.

[0098] For example, Figure 5B FIG. 1 is a timing diagram of interaction between a first electronic device and a second electronic device in another embodiment. Figure 5B As shown, the first electronic device establishes a Bluetooth communication connection with the second electronic device, the second electronic device can send a DelayReport value to the first electronic device, and the first electronic device subtracts the decoding delay duration from the received DelayReport value to obtain the buffering delay duration of the first buffer. The first electronic device can encode multimedia data and send the multimedia data to the second electronic device, and the second electronic device receives the multimedia data and decodes and plays the multimedia data. The first electronic device can monitor the buffering duration corresponding to the second buffer, and if it is determined that the second electronic device has a risk of playback jamming based on the buffering duration and the buffering delay duration, the target operation is performed.

[0099] In some embodiments, the first buffer information may be used to indicate the buffer delay of the first buffer when the second electronic device starts to play the multimedia data sent by the first electronic device. After the second electronic device receives the multimedia data sent by the first electronic device, it needs to accumulate a certain amount of multimedia data in the first buffer. The first buffer information may represent the amount of data that the second electronic device needs to accumulate in the first buffer before playing the multimedia data.

[0100] Exemplarily, the first buffer information may include the buffer delay duration of the first buffer when the second electronic device starts to play the multimedia data sent by the first electronic device. The buffer delay duration may be the buffer duration of the first buffer when the second electronic device starts to play the multimedia data, that is, the playback duration corresponding to the multimedia data stored in the first buffer. The first buffer information reflects the buffer delay of the first buffer when the second electronic device starts to play the multimedia data. The first buffer information can be used to more accurately identify whether the second electronic device has a risk of playback jamming, further improving the accuracy of identifying the risk of playback jamming of the second electronic device.

[0101] In some embodiments, the first electronic device may obtain the first buffer information corresponding to the second electronic device when establishing a multimedia data transmission channel with the second electronic device. When the first electronic device establishes a Bluetooth communication connection with the second electronic device, if the first electronic device needs to play multimedia data through the second electronic device, a multimedia data transmission channel may be established with the second electronic device. For example, the multimedia data transmission channel may include but is not limited to any one of a SCO (Synchronous Connection Oriented) channel, an ACL (Asynchronous Connection-Less) channel, a CIS (Connected Isochronous Stream) channel, and the like.

[0102] After the first electronic device establishes a Bluetooth multimedia data transmission channel with the second electronic device, a start instruction may be sent to the second electronic device. After receiving the start instruction, the second electronic device may send first buffer information to the first electronic device based on a private communication protocol. Alternatively, after receiving the start instruction, the second electronic device may send a delay value to the first electronic device based on the Bluetooth standard protocol. The buffer size of the first buffer zone may be preconfigured, so the first buffer information or delay value may be a pre-set fixed value associated with the buffer size of the first buffer zone. The second electronic device may also dynamically allocate the buffer size of the first buffer zone according to the network environment parameters (such as transmission signal strength, signal-to-noise ratio, etc.) when the start instruction is received, and determine the first buffer information or delay value according to the allocated buffer size of the first buffer zone.

[0103] The first electronic device obtains the first buffer information or delay value sent by the second electronic device for the first time after sending the start-play instruction, thereby determining the first buffer information used to indicate the buffer delay of the first buffer when the second electronic device starts playing multimedia data. In the case where the first electronic device and the second electronic device establish a Bluetooth multimedia data transmission channel, the second electronic device can actively report the first buffer information or delay value to the first electronic device, and the first electronic device obtains the first buffer information of the second electronic device when playing multimedia data, and can accurately identify whether the second electronic device has a risk of playback jamming based on the first buffer information and the data storage status of the second buffer.

[0104] As an implementation mode, when the second electronic device adjusts the buffer space of the first buffer, the buffer delay of the first buffer will change. When the buffer delay of the first buffer changes, the second electronic device may send the changed first buffer information to the first electronic device, or may send the delay value to the first electronic device, so that the first electronic device updates the first buffer information in time. Further, when the second electronic device adjusts the buffer space of the first buffer, the second electronic device may determine whether the change amplitude corresponding to the buffer delay of the first buffer is greater than the preset amplitude, for example, the second electronic device may determine whether the change time difference of the buffer delay duration of the first buffer is greater than the preset time difference. If the change amplitude corresponding to the buffer delay of the first buffer is greater than the preset amplitude, indicating that the buffer delay of the first buffer changes greatly, the second electronic device may send the changed first buffer information to the first electronic device, or send the delay value to the first electronic device, so that the first electronic device updates the first buffer information in time. If the change amplitude corresponding to the buffer delay of the first buffer is not greater than the preset amplitude, indicating that the change of the buffer delay of the first buffer is small, the second electronic device may not feedback the first buffer information or the delay value to the first electronic device, and can reduce the number of times the first buffer information is sent while taking into account the accuracy of identifying the risk of playback jamming, and reduce the transmission overhead.

[0105] As another implementation, the second electronic device may also periodically send the first buffer information or delay value to the first electronic device, and the first electronic device may update the recorded first buffer information according to the first buffer information or delay value periodically sent by the second electronic device. Further, the first electronic device may determine the first buffer information reported by the second electronic device this time according to the first buffer information or delay value periodically sent by the second electronic device, and may compare the first buffer information determined this time with the recorded first buffer information, calculate the change range between the two, and determine whether the change range is greater than the preset range. For example, the first electronic device may calculate the time difference between the buffer delay duration of the first buffer determined this time and the buffer delay duration of the recorded first buffer, and determine whether the time difference is greater than the preset time difference. If the change range corresponding to the buffer delay of the first buffer is greater than the preset range, indicating that the buffer delay of the first buffer has changed significantly, the first electronic device may update the recorded first buffer information to the first buffer information determined this time; if the change range corresponding to the buffer delay of the first buffer is not greater than the preset range, indicating that the change of the buffer delay of the first buffer is small, the first electronic device may not update the recorded first buffer information, which can ensure the stability of the recorded first buffer information, thereby improving the accuracy of identifying whether the second electronic device has the risk of playback jamming.

[0106] Step 404, determining second buffer information corresponding to a second buffer in the first electronic device, where the second buffer is used to buffer multimedia data to be transmitted to the second electronic device, and the second buffer information is used to indicate the data volume of the multimedia data stored in the second buffer.

[0107] The description of step 404 may refer to the relevant description in the above embodiment, and will not be repeated here.

[0108] Step 406: If the second buffer information is greater than a target threshold, it is determined that the second electronic device has a risk of playback freeze; the target threshold is determined based on the first buffer information.

[0109] The first electronic device monitors the second buffer information of the second buffer in real time, and can determine whether the second buffer information is greater than a target threshold value, which is determined based on the first buffer information. For example, the target threshold value can be the first buffer information, or the product of the first buffer information and a preset first coefficient, or the difference between the first buffer information and a first preset value, etc. For example, the first buffer information is the buffer delay duration of the first buffer, and the target threshold value can be the buffer delay duration, or the buffer delay duration multiplied by a preset first coefficient (such as 0.9, 0.88, etc.), or the buffer delay duration minus the first preset duration (such as 10ms, 15ms, etc.).

[0110] If the second buffer information is greater than the target threshold, the amount of multimedia data stored in the second buffer in the first electronic device is close to or even greater than the buffer delay in the first buffer in the second electronic device, which means that the amount of multimedia data buffered in the first buffer in the second electronic device is close to zero or has reached zero, and the second electronic device is very likely to experience playback jams, and there is a risk of playback jams. By combining the storage situation of the second buffer in the first electronic device and the buffer delay in the first buffer in the second electronic device, the risk of playback jams in the second electronic device can be accurately identified, thereby improving the accuracy of identifying the risk of playback jams.

[0111] Step 408: Execute a target operation, where the target operation is used to reduce the amount of multimedia data stored in the second buffer.

[0112] In some embodiments, the target operation may include a bit rate reduction operation and / or a data clearing operation, where the data clearing operation may be used to clear all multimedia data stored in the second buffer, or to clear part of the multimedia data stored in the second buffer.

[0113] In some embodiments, clearing part of the multimedia data stored in the second buffer may include but is not limited to at least one of the following:

[0114] (1) Clearing multimedia data stored in the second buffer and matching the first data volume parameter.

[0115] The first data volume parameter may be used to indicate the data volume of the multimedia data cleared from the second buffer, and the first data volume parameter may include the first playback duration corresponding to the multimedia data cleared from the second buffer, the first number of bytes corresponding to the multimedia data cleared from the second buffer, the first number of samples, and the first number of data packets, etc. When the first electronic device determines that the second electronic device has a risk of playback freeze, the multimedia data stored in the second buffer that matches the first data volume parameter may be cleared.

[0116] Optionally, the first data volume parameter may be determined based on at least one of the first buffer information, the second buffer information, and the buffer size of the second buffer area.

[0117] Exemplarily, the first data volume parameter may be determined according to the first buffer information, and the first data volume parameter may be equal to the product of the first buffer information and a preset second coefficient, or equal to the difference between the first buffer information and a second preset value, etc. For example, the first buffer information is the buffer delay duration of the first buffer zone, and the first data volume parameter is equal to the product of the buffer delay duration and a preset second coefficient (such as 0.6, 0.7, etc.), or equal to the buffer delay duration minus the second preset duration (such as 60ms, 70ms), etc.

[0118] Exemplarily, the first data volume parameter may be determined according to the second buffer information, and the first data volume parameter may be equal to the product of the second buffer information and the first ratio, or equal to the difference between the second buffer information and a third preset value, etc. For example, the second buffer information is the current buffering time of the second buffer, and the first data volume parameter is equal to the product of the buffering time and the first ratio (such as 67%, 65%, etc.), or equal to the buffering time minus the third preset time (such as 50ms, 73ms, etc.).

[0119] Exemplarily, the first data volume parameter may be determined according to the buffer size of the second buffer, and the second data volume parameter may be equal to the product of the buffer size of the second buffer and a second ratio (eg, 60%, 70%, etc.).

[0120] Exemplarily, the first data volume parameter can also be determined based on multiple factors such as the first buffer information, the second buffer information, and the buffer size of the second buffer. Different weight coefficients can be assigned to the first buffer information, the second buffer information, and the buffer size of the second buffer, respectively, and weighted averaging and other calculations can be performed based on the weight systems corresponding to the first buffer information, the second buffer information, and the buffer size of the second buffer, respectively, to obtain the first data volume parameter.

[0121] Based on at least one of the first buffer information, the second buffer information and the buffer size of the second buffer, the first data volume parameter is determined, and the data volume of the second buffer to be cleared is dynamically adjusted. This can avoid clearing too much multimedia data, which may lead to a more obvious jump in the played multimedia data, and can also avoid clearing too little multimedia data, which may result in an inability to effectively reduce network congestion, thereby improving the accuracy of the data clearing operation on the second buffer.

[0122] (2) Clearing part of the multimedia data stored in the second buffer so that the multimedia data remaining in the second buffer is less than or equal to the second data volume parameter.

[0123] The second data volume parameter can be used to indicate the maximum amount of multimedia data remaining after a data clearing operation is performed on the second buffer. The second data volume parameter may include a second playback duration, a second number of bytes, a second number of samples, a second number of data packets, etc. When the first electronic device determines that there is a risk of playback jams in the second electronic device, the multimedia data stored in the second buffer can be cleared so that the multimedia data remaining in the second buffer is less than or equal to the second data volume parameter.

[0124] Optionally, the second data volume parameter may be determined based on at least one of the first buffer information, the second buffer information, and a buffer size of the second buffer area.

[0125] It should be noted that the method for determining the second data volume parameter may be similar to the method for determining the first data volume parameter described in the above embodiment, and each corresponds to a different preset coefficient or preset value, which will not be repeated here.

[0126] Based on at least one of the first buffer information, the second buffer information and the buffer size of the second buffer, the second data volume parameter is determined, and the remaining data volume of the second buffer is dynamically adjusted. This can avoid the situation where too little remaining multimedia data results in the failure to transmit the multimedia data to the first electronic device in a timely manner, and can also avoid the situation where too much remaining multimedia data results in the failure to effectively reduce network congestion, thereby improving the accuracy of data clearing operations on the second buffer.

[0127] (3) Clearing discontinuous multimedia data stored in the second buffer.

[0128] When the first electronic device determines that the second electronic device is at risk of playback freeze, it can detect whether there is discontinuous multimedia data in the second buffer. Exemplarily, the data packet numbers corresponding to the data packets stored in the second buffer are continuous, because the multimedia encoder encodes and generates data packets according to the data stream of the multimedia data. Therefore, when the data packet numbers are continuous, the multimedia data carried by the data packets are also continuous.

[0129] It can be detected whether there are discontinuous data packet numbers in the second buffer. If there are discontinuous data packet numbers, it means that there are discontinuous multimedia data, and multiple data packets corresponding to the discontinuous data packet numbers (including but not limited to one of the data packets corresponding to the discontinuous data packet numbers) can be deleted. For example, the second buffer stores data packets numbered 1, 2, 3, 5, and 6, and numbers 3 and 5 are discontinuous. Data packets numbered 1, 2, and 3 can be deleted, and data packets numbered 5 and 6 can also be deleted. Data packets numbered 3 and 5 can also be deleted, but not limited to this. By deleting discontinuous multimedia data in the second buffer, the playback jams caused by the data clearing operation can be combined with the playback jams caused by the discontinuous multimedia data, reducing the number of playback jams on the second electronic device and improving the user experience.

[0130] (4) Clearing part of the multimedia data stored in the second buffer so that the remaining multimedia data in the second buffer is continuous.

[0131] When the first electronic device determines that the second electronic device is at risk of playback jamming, the multimedia data stored in the second buffer can be cleared so that the remaining multimedia data in the second buffer is continuous. Exemplarily, after the data clearing operation is performed on the second buffer, the data packet numbers corresponding to the remaining data packets in the second buffer are continuous. For example, if the second buffer stores data packets numbered 1 to 6, the data packets numbered 1 to 4 can be deleted, or the data packets numbered 3 to 6 can be deleted, etc. The data packet numbers corresponding to the remaining data packets are continuous, which can ensure the continuity of the multimedia data that has not been deleted, and ensure that the multimedia data played by the second electronic device is as continuous as possible, reducing playback jams.

[0132] (5) Clearing part of the multimedia data stored in the second buffer in the order of the storage time of the multimedia data in the second buffer from the earliest to the latest.

[0133] When the first electronic device determines that there is a risk of playback jams in the second electronic device, some multimedia data stored in the second buffer can be cleared in order from the storage time, thereby ensuring the continuity of the remaining multimedia data and the multimedia data subsequently stored in the second buffer, thereby reducing the playback jams caused by the second electronic device.

[0134] In an embodiment of the present application, the first electronic device can clear part of the multimedia data stored in the second buffer, effectively reducing the multimedia data stored in the second buffer to alleviate network congestion, while avoiding deleting too much multimedia data, resulting in long playback jams and skipping times, which leads to a decrease in user experience, thereby improving the user experience.

[0135] In some embodiments, if the first electronic device detects that the multimedia data stored in the second buffer is discontinuous, a data clearing operation may be performed on the discontinuous multimedia data in the second buffer. Exemplarily, it may be detected whether there are discontinuous data packet numbers in the second buffer. If there are discontinuous data packet numbers, it indicates that there are discontinuous multimedia data, and multiple data packets corresponding to the discontinuous data packet numbers (including but not limited to one of the data packets corresponding to the discontinuous data packet numbers) may be deleted.

[0136] If the first electronic device detects that the multimedia data stored in the second buffer is discontinuous, although the multimedia data to be played in the first buffer of the second electronic device is sufficient and there is no data accumulation in the second buffer of the first electronic device, the second electronic device will still experience playback jams due to the discontinuity of the multimedia data, and there is a risk of playback jams. Therefore, a data clearing operation can be performed on the discontinuous multimedia data in the second buffer, and the playback jams caused by the data clearing operation can be merged with the playback jams caused by the discontinuity of the multimedia data in advance, so that the playback jams will not occur first due to the discontinuity of the multimedia data, and then the playback jams will not occur due to the data clearing operation due to the accumulation of data. This can reduce the number of playback jams and improve the user experience.

[0137] In an embodiment of the present application, by combining the storage situation of the second buffer in the first electronic device and the buffering delay of the first buffer in the second electronic device, the risk of playback jamming in the second electronic device can be accurately identified, the accuracy of identifying the risk of playback jamming can be improved, and when it is identified that the risk of playback jamming in the second electronic device exists, the target operation is performed to reduce the amount of multimedia data stored in the second buffer, reduce network congestion, and thus improve the transmission stability of the multimedia data.

[0138] In some embodiments, different target operations may be performed for different risk levels of playback freeze. Figure 6 As shown, in another embodiment, a method for processing multimedia data is provided, and the method may include the following steps:

[0139] Step 602: Acquire first buffer information corresponding to the second electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played.

[0140] Step 604, determining second buffer information corresponding to a second buffer in the first electronic device, where the second buffer is used to buffer multimedia data to be transmitted to the second electronic device, and the second buffer information is used to indicate the data volume of the multimedia data stored in the second buffer.

[0141] The description of steps 602 to 604 may refer to the relevant description in the above embodiments, and will not be repeated here.

[0142] Step 606: If the second buffer information is greater than the first target threshold, it is determined that the second electronic device has a first freeze risk.

[0143] Step 608, executing a data clearing operation.

[0144] The target threshold in the above embodiment may include a first target threshold and a second target threshold. The first target threshold may be greater than the second target threshold. For example, the preset first coefficient corresponding to the first target threshold may be greater than the preset first coefficient corresponding to the second target threshold, or the first preset value corresponding to the first target threshold may be less than the first preset value corresponding to the second target threshold, etc.

[0145] The risk of playback jamming of the second electronic device may include a first jamming risk and a second jamming risk, and the risk level of the first jamming risk may be greater than the risk level of the second jamming risk, that is, the probability of playback jamming of the second electronic device under the first jamming risk is higher than the probability of playback jamming under the second jamming risk. The first jamming risk may correspond to a first target threshold, and the second jamming risk may correspond to a second target threshold.

[0146] The first electronic device can determine whether the second buffer information is greater than the first target threshold. If the second buffer information is greater than the first target threshold, it means that the amount of multimedia data buffered in the first buffer zone in the second electronic device is close to zero or is already zero, and there is an extremely high risk of playback jamming. Therefore, it can be determined that the second electronic device is at the first risk of jamming.

[0147] When the second electronic device is at risk of the first jamming, it can be basically determined that the second electronic device will experience playback jamming. The first electronic device can perform a data clearing operation on the second buffer, so that the playback jamming caused by the data clearing operation and the playback jamming caused by the first buffer being read empty can be synchronously reflected in one playback jamming, so that one playback jamming can eliminate multiple causes of playback jamming, thereby reducing the number of playback jams on the second electronic device and improving user experience.

[0148] In some embodiments, if the second electronic device is at a first risk of freezing, a data clearing operation and a bit rate reduction operation are performed.

[0149] When the second electronic device is at risk of the first freeze, the first electronic device can clear data on the second buffer and reduce the bit rate of the multimedia encoder, thereby further reducing the amount of multimedia data cached in the second buffer and increasing the transmission speed of the multimedia data to the second electronic device, thereby restoring normal playback of the second electronic device as soon as possible.

[0150] Step 610: If the second buffer information is greater than the second target threshold and not greater than the first target threshold, it is determined that the second electronic device has a second freeze risk.

[0151] Step 612, performing a bit rate reduction operation.

[0152] The first electronic device can also determine whether the second buffer information is greater than the second target threshold. If the second buffer information is greater than the second target threshold but not greater than the first target threshold, it means that the data accumulation in the second buffer is relatively light, and the multimedia data cached in the first buffer in the second electronic device can still be played. The second electronic device has a certain risk of playback jamming, but playback jamming will not necessarily occur. Therefore, it can be determined that the second electronic device is at the second risk of jamming.

[0153] When the second electronic device is at risk of the second lag, since the second electronic device may not actually experience playback lag, the first electronic device may not clear the data on the second buffer, but may reduce the bit rate of the multimedia encoder, reduce the amount of multimedia data cached in the second buffer, alleviate data accumulation in the second buffer, and increase the transmission speed of multimedia data to the second electronic device. This further reduces the risk of playback lag on the second electronic device and reduces the number of playback lags caused by the first electronic device clearing the data on the second buffer.

[0154] In an embodiment of the present application, the first electronic device can perform different target operations according to the different risk levels of playback jamming risks of the second electronic device. While reducing the risk of playback jamming of the second electronic device, it also avoids the number of playback jams caused by the first electronic device performing a data clearing operation on the second buffer, thereby improving the user experience.

[0155] As a specific implementation method, Figure 7 As shown, the above-mentioned method for processing multimedia data may include the following steps:

[0156] Step 702, after the Source end (ie, the first electronic device mentioned above) triggers to start playing, the buffering delay time T of the first buffer when the Sink end (ie, the second electronic device mentioned above) starts to play the multimedia data is obtained.

[0157] Step 704, obtaining the current buffering time t of the second buffer at the Source end.

[0158] Step 706 , the Source determines whether the current buffering time t is greater than the first target threshold (eg, buffering delay time T*0.95). If so, execute step 708 ; if not, execute step 710 .

[0159] Step 708: If playback freezes at the Sink, the Source clears the data in the second buffer to a suitable value, and then executes step 712.

[0160] In step 710 , the Source determines whether the current buffering time t is greater than a second target threshold (eg, buffering delay time T*0.7). If so, step 712 is executed; if not, step 714 is executed.

[0161] Step 712: The Source reduces the bit rate of the audio encoder and performs encoding according to the reduced bit rate.

[0162] Step 714: The Source performs encoding according to the current bit rate.

[0163] Step 716: The Source puts the encoded data into the second buffer for transmission.

[0164] Step 718: After the Source waits for a certain period of time, step 704 is executed again.

[0165] In the embodiment of the present application, it is possible to jointly determine whether the Sink has playback jamming based on the buffering delay time of the first buffer in the Sink end and the buffering time t of the second buffer in the Source end, and when the Sink has playback jamming, the Source end clears the data in the second buffer to a suitable value, which not only reduces network congestion, but also synchronizes multiple playback jams into one playback jam, thereby reducing the number of playback jams in the second electronic device. In addition, according to the different risk levels of playback jamming risks in the second electronic device, the first electronic device performs different target operations, further reducing the number of playback jams caused by the first electronic device performing data clearing operations on the second buffer, and improving the user experience.

[0166] like Figure 8 As shown, in one embodiment, a method for sending information is provided, which can be applied to the second electronic device mentioned above, and the method may include the following steps:

[0167] Step 810: Send first buffer information to the first electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played.

[0168] Among them, the first buffer information is used to trigger the first electronic device to determine the second buffer information corresponding to the second buffer, and perform the target operation when it is determined that the second electronic device has a risk of playback jamming based on the first buffer information and the second buffer information; the second buffer is used to buffer multimedia data in the first electronic device to be transmitted to the second electronic device, the second buffer information is used to indicate the data volume of the multimedia data stored in the second buffer, and the target operation is used to reduce the data volume of the multimedia data stored in the second buffer.

[0169] In some embodiments, the first buffering information is used to indicate a buffering delay of the first buffer when the second electronic device starts playing the multimedia data sent by the first electronic device.

[0170] In some embodiments, step 810 includes: sending first buffer information to the first electronic device when a multimedia data transmission channel is established with the first electronic device.

[0171] It should be noted that the description of the information sending method applied to the second electronic device provided in the embodiment of the present application can refer to the relevant description of the multimedia data processing method applied to the first electronic device provided in the above embodiments, and will not be repeated here.

[0172] In an embodiment of the present application, the second electronic device can accurately send the first buffer information to the first electronic device, thereby improving the accuracy of the first buffer information obtained by the first electronic device. The first electronic device can combine the buffer delay of the first buffer zone in the second electronic device and the amount of multimedia data stored in its own second buffer zone based on the first buffer information and the second buffer information, and accurately identify whether the second electronic device is at risk of playback jams, thereby further improving the accuracy of identifying whether the second electronic device is at risk of playback jams.

[0173] like Fig. 9 As shown, in one embodiment, a multimedia data processing device 900 is provided, which can be applied to the above-mentioned first electronic device. The multimedia data processing device 900 may include an acquisition module 910, a determination module 920 and an operation module 930.

[0174] The acquisition module 910 is used to acquire first buffer information corresponding to the second electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played.

[0175] The determination module 920 is used to determine the second buffer information corresponding to the second buffer in the first electronic device, the second buffer is used to buffer the multimedia data to be transmitted to the second electronic device, and the second buffer information is used to indicate the data volume of the multimedia data stored in the second buffer.

[0176] The operation module 930 is used to execute a target operation if it is determined that the second electronic device has a risk of playback freeze according to the first buffer information and the second buffer information. The target operation is used to reduce the amount of multimedia data stored in the second buffer.

[0177] In one embodiment, the first buffering information is used to indicate a buffering delay of the first buffer zone when the second electronic device starts to play the multimedia data sent by the first electronic device.

[0178] In one embodiment, the acquisition module 910 is further configured to acquire first buffer information corresponding to the second electronic device when a multimedia data transmission channel is established with the second electronic device.

[0179] In one embodiment, the target operation includes a bit rate reduction operation and / or a data clearing operation. The bit rate reduction operation refers to an operation of reducing the bit rate corresponding to the multimedia encoder of the first electronic device, and the data clearing operation refers to an operation of clearing at least part of the multimedia data stored in the second buffer.

[0180] In one embodiment, the target operation includes a data clearing operation, and the data clearing operation is used to clear all multimedia data stored in the second buffer, or to clear part of the multimedia data stored in the second buffer. Wherein, clearing part of the multimedia data stored in the second buffer includes at least one of the following:

[0181] Clearing multimedia data stored in the second buffer and matching the first data volume parameter;

[0182] clearing part of the multimedia data stored in the second buffer so that the multimedia data remaining in the second buffer is less than or equal to the second data volume parameter;

[0183] Clearing discontinuous multimedia data stored in the second buffer;

[0184] clearing part of the multimedia data stored in the second buffer so that the remaining multimedia data in the second buffer is continuous;

[0185] Part of the multimedia data stored in the second buffer is cleared in a descending order of the storage times of the multimedia data in the second buffer.

[0186] In one embodiment, the first data volume parameter is determined based on at least one of the first buffer information, the second buffer information, and a buffer size of the second buffer;

[0187] The second data volume parameter is determined according to at least one of the first buffer information, the second buffer information and the buffer size of the second buffer area.

[0188] In one embodiment, the multimedia data processing device 900 further includes a risk identification module.

[0189] The risk identification module is used to determine that the second electronic device has a risk of playback jamming if the second buffer information is greater than a target threshold; the target threshold is determined based on the first buffer information.

[0190] In one embodiment, the target threshold includes a first target threshold and a second target threshold, the first target threshold is greater than the second target threshold, and the playback freeze risk includes a first freeze risk and a second freeze risk.

[0191] The risk identification module is also used to determine that the second electronic device has a first risk of freezing if the second buffer information is greater than the first target threshold; and to determine that the second electronic device has a second risk of freezing if the second buffer information is greater than the second target threshold and not greater than the first target threshold.

[0192] The operation module 930 is further used to perform a data clearing operation if the second electronic device has a first risk of freezing; and to perform a bit rate reduction operation if the second electronic device has a second risk of freezing.

[0193] In one embodiment, the operation module 930 is further used to execute a data clearing operation and a bit rate reduction operation if the second electronic device has a first risk of freezing.

[0194] In one embodiment, the operation module 930 is further configured to perform a data clearing operation on the discontinuous multimedia data in the second buffer if it is detected that the multimedia data stored in the second buffer is discontinuous.

[0195] In one embodiment, the acquisition module 910 is further configured to receive first buffer information sent by a second electronic device; or,

[0196] It is also used to receive a delay value sent by a second electronic device, the delay value is used to characterize the delay of the second electronic device in processing the received multimedia data, the delay of the second electronic device in processing the received multimedia data includes buffering delay and decoding delay; and determine the first buffering information based on the delay value and the decoding delay.

[0197] In one embodiment, the first buffering information includes the buffering delay duration of the first buffer, and the buffering delay duration refers to the duration that the multimedia data stays in the first buffer; the second buffering information includes the buffering duration of the second buffer, and the buffering duration of the second buffer refers to the playback duration corresponding to the multimedia data stored in the second buffer.

[0198] In an embodiment of the present application, the first electronic device can accurately identify whether the second electronic device is at risk of playback jams based on the buffering delay of the first buffer in the second electronic device and the amount of multimedia data stored in its own second buffer, and when it is identified that the second electronic device is at risk of playback jams, perform a target operation to reduce the amount of multimedia data stored in the second buffer, reduce network congestion, and thus improve the transmission stability of the multimedia data, which can reduce the number of times a data receiving end experiences jams when playing multimedia data, thereby improving the user experience.

[0199] like Fig.10 As shown, in one embodiment, an information sending device 1000 is provided, which can be applied to the above-mentioned second electronic device. The information sending device 1000 may include a sending module 1010.

[0200] The sending module 1010 is used to send first buffer information to the first electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played.

[0201] Among them, the first buffer information is used to trigger the first electronic device to determine the second buffer information corresponding to the second buffer, and perform the target operation when it is determined that the second electronic device has a risk of playback jamming based on the first buffer information and the second buffer information; the second buffer is used to buffer multimedia data in the first electronic device to be transmitted to the second electronic device, the second buffer information is used to indicate the data volume of the multimedia data stored in the second buffer, and the target operation is used to reduce the data volume of the multimedia data stored in the second buffer.

[0202] In one embodiment, the first buffering information is used to indicate a buffering delay of the first buffer zone when the second electronic device starts to play the multimedia data sent by the first electronic device.

[0203] In one embodiment, the sending module 1010 is further configured to send the first buffer information to the first electronic device when a multimedia data transmission channel is established with the first electronic device.

[0204] In an embodiment of the present application, the second electronic device can accurately send the first buffer information to the first electronic device, thereby improving the accuracy of the first buffer information obtained by the first electronic device. The first electronic device can combine the buffer delay of the first buffer zone in the second electronic device and the amount of multimedia data stored in its own second buffer zone based on the first buffer information and the second buffer information, and accurately identify whether the second electronic device is at risk of playback jams, thereby further improving the accuracy of identifying whether the second electronic device is at risk of playback jams.

[0205] Fig.11 FIG. 1 is a structural block diagram of an electronic device in an embodiment. Fig.11 As shown, the electronic device 1100 may include one or more of the following components: a processor 1110, and a memory 1120 coupled to the processor 1110, wherein the memory 1120 may store one or more computer programs, and the one or more computer programs may be configured to be executed by one or more processors 1110, so that the electronic device 1100 implements the method for processing multimedia data applied to the first electronic device as described in the above embodiments.

[0206] The processor 1110 may include one or more processing cores. The processor 1110 uses various interfaces and lines to connect various parts of the entire electronic device 1100, and executes various functions and processes data of the electronic device 1100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1120, and calling data stored in the memory 1120. Optionally, the processor 1110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1110, but may be implemented separately through a communication chip.

[0207] The memory 1120 may include a random access memory (RAM) or a read-only memory (ROM). The memory 1120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the electronic device 1100 during use, etc.

[0208] The electronic device 1100 may further include a transceiver unit, which may include but is not limited to a Bluetooth module, a Wi-Fi module, etc., and may be used to provide a communication function.

[0209] An embodiment of the present application discloses an electronic device, including a memory, a processor, and a transceiver unit. The memory stores a computer program. When the computer program is executed by the processor, the electronic device implements the information sending method applied to a second electronic device as described in the above embodiments.

[0210] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor in an electronic device, the electronic device implements a method for processing multimedia data applied to a first electronic device as described in the above embodiments.

[0211] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor in an electronic device, the electronic device implements the information sending method applied to a second electronic device as described in the above embodiments.

[0212] An embodiment of the present application discloses a computer program product, which includes a computer program. When the computer program can be executed by a processor in an electronic device, the electronic device implements a method for processing multimedia data applied to a first electronic device as described in the above embodiments.

[0213] An embodiment of the present application discloses a computer program product, which includes a computer program. When the computer program can be executed by a processor in an electronic device, the electronic device implements the information sending method applied to a second electronic device as described in the above embodiments.

[0214] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a ROM, etc.

[0215] As used herein, any reference to memory, storage, database, or other medium may include nonvolatile and / or volatile memory. Suitable nonvolatile memory may include ROM, programmable ROM (Programmable ROM, PROM), erasable PROM (Erasable PROM, EPROM), electrically erasable PROM (Electrically Erasable PROM, EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache memory. By way of illustration and not limitation, RAM may be in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM) and direct memory bus dynamic RAM (DRDRAM).

[0216] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application. It should be noted that "multiple" in the present application includes "two or more".

[0217] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0218] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0219] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0220] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0221] The above is a detailed introduction to a multimedia data processing method and device, and an information sending method and device disclosed in the embodiments of the present application. The principles and implementation methods of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those of ordinary skill in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for processing multimedia data, characterized in that: Applied to a first electronic device, the first electronic device establishes a communication connection with a second electronic device, the first electronic device is used to send multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data; the method includes: Acquire first buffer information corresponding to the second electronic device, where the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played; Determining second buffer information corresponding to a second buffer in the first electronic device, where the second buffer is used to buffer multimedia data to be transmitted to the second electronic device, and the second buffer information is used to indicate a data amount of the multimedia data stored in the second buffer; If it is determined according to the first buffer information and the second buffer information that the second electronic device has a risk of playback freeze, a target operation is performed, where the target operation is used to reduce the amount of multimedia data stored in the second buffer.

2. The method according to claim 1, characterized in that The first buffering information is used to indicate a buffering delay of the first buffer zone when the second electronic device starts to play the multimedia data sent by the first electronic device.

3. The method according to claim 2, characterized in that The acquiring first buffer information corresponding to the second electronic device includes: When a multimedia data transmission channel is established with the second electronic device, first buffer information corresponding to the second electronic device is obtained.

4. The method according to any one of claims 1 to 3, characterized in that: The target operation includes a bit rate reduction operation and / or a data clearing operation. The bit rate reduction operation refers to an operation of reducing the bit rate corresponding to the multimedia encoder of the first electronic device, and the data clearing operation refers to an operation of clearing at least part of the multimedia data stored in the second buffer.

5. The method according to claim 4, characterized in that The target operation includes a data clearing operation, and the data clearing operation is used to clear all multimedia data stored in the second buffer, or to clear part of the multimedia data stored in the second buffer; The clearing of part of the multimedia data stored in the second buffer includes at least one of the following: clearing multimedia data stored in the second buffer and matching the first data volume parameter; clearing part of the multimedia data stored in the second buffer so that the multimedia data remaining in the second buffer is less than or equal to a second data volume parameter; clearing discontinuous multimedia data stored in the second buffer; clearing part of the multimedia data stored in the second buffer so that the remaining multimedia data in the second buffer is continuous; Part of the multimedia data stored in the second buffer is cleared in order of storage times of the multimedia data in the second buffer from earliest to latest.

6. The method according to claim 5, characterized in that The first data volume parameter is determined according to at least one of the first buffer information, the second buffer information and a buffer size of the second buffer; The second data volume parameter is determined according to at least one of the first buffer information, the second buffer information and the buffer size of the second buffer area.

7. The method according to claim 4, characterized in that Before executing the target operation if it is determined according to the first buffer information and the second buffer information that the second electronic device has a risk of playback freeze, the method further includes: If the second buffer information is greater than a target threshold, it is determined that the second electronic device has a risk of playback freeze; the target threshold is determined based on the first buffer information.

8. The method according to claim 7, characterized in that The target threshold includes a first target threshold and a second target threshold, the first target threshold is greater than the second target threshold, the playback jam risk includes a first jam risk and a second jam risk; if the second buffer information is greater than the target threshold, determining that the second electronic device has a playback jam risk includes: If the second buffer information is greater than the first target threshold, determining that the second electronic device has the first freeze risk; If the second buffer information is greater than the second target threshold and not greater than the first target threshold, it is determined that the second electronic device has the second freeze risk; If it is determined according to the first buffer information and the second buffer information that the second electronic device has a risk of playback freeze, executing a target operation includes: If the second electronic device has the first risk of freezing, executing the data clearing operation; If the second electronic device has the second risk of freezing, the bit rate reduction operation is performed.

9. The method according to claim 8, characterized in that If the second electronic device has the first freezing risk, performing a data clearing operation includes: If the second electronic device has the first risk of freezing, the data clearing operation and the bit rate reduction operation are performed.

10. The method according to claim 1, characterized in that The method further comprises: If it is detected that the multimedia data stored in the second buffer is discontinuous, a data clearing operation is performed on the discontinuous multimedia data in the second buffer.

11. The method according to any one of claims 1 to 3, 5 to 10, characterized in that: The acquiring first buffer information corresponding to the second electronic device includes: receiving first buffer information sent by the second electronic device; or, receiving a delay value sent by the second electronic device, where the delay value is used to represent a delay in the second electronic device processing the received multimedia data, where the delay in the second electronic device processing the received multimedia data includes the buffering delay and the decoding delay; First buffering information is determined according to the delay value and the decoding delay.

12. The method according to any one of claims 1 to 3, 5 to 10, characterized in that: The first buffering information includes the buffering delay duration of the first buffer, and the buffering delay duration refers to the duration that the multimedia data stays in the first buffer; the second buffering information includes the buffering duration of the second buffer, and the buffering duration of the second buffer refers to the playback duration corresponding to the multimedia data stored in the second buffer.

13. A method for sending information, characterized in that: Applied to a second electronic device, the second electronic device establishes a communication connection with a first electronic device, the first electronic device is used to send multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data; the method includes: Sending first buffering information to the first electronic device, where the first buffering information is used to indicate a buffering delay of a first buffer in the second electronic device, where the first buffer is used to buffer multimedia data to be played; The first buffer information is used to trigger the first electronic device to determine the second buffer information corresponding to the second buffer, and to perform a target operation when it is determined that the second electronic device is at risk of playback jamming based on the first buffer information and the second buffer information; the second buffer is used to buffer multimedia data in the first electronic device to be transmitted to the second electronic device, the second buffer information is used to indicate the amount of multimedia data stored in the second buffer, and the target operation is used to reduce the amount of multimedia data stored in the second buffer.

14. The method according to claim 13, characterized in that The first buffering information is used to indicate a buffering delay of the first buffer zone when the second electronic device starts to play the multimedia data sent by the first electronic device.

15. The method according to claim 14, characterized in that The sending the first buffer information to the first electronic device includes: When a multimedia data transmission channel is established with the first electronic device, first buffer information is sent to the first electronic device.

16. A multimedia data processing device, characterized in that: Applied to a first electronic device, the first electronic device establishes a communication connection with a second electronic device, the first electronic device is used to send multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data; the device includes: an acquisition module, configured to acquire first buffer information corresponding to the second electronic device, wherein the first buffer information is used to indicate a buffer delay of a first buffer in the second electronic device, and the first buffer is used to buffer multimedia data to be played; a determining module, configured to determine second buffering information corresponding to a second buffer in the first electronic device, the second buffer being configured to buffer multimedia data to be transmitted to the second electronic device, the second buffering information being configured to indicate a data volume of the multimedia data stored in the second buffer; An operation module is used to execute a target operation if it is determined that the second electronic device has a risk of playback jamming according to the first buffer information and the second buffer information. The target operation is used to reduce the amount of multimedia data stored in the second buffer.

17. An information sending device, characterized in that: Applied to a second electronic device, the second electronic device establishes a communication connection with a first electronic device, the first electronic device is used to send multimedia data to the second electronic device through the communication connection, and the second electronic device plays the multimedia data; the device includes: a sending module, configured to send first buffering information to the first electronic device, wherein the first buffering information is used to indicate a buffering delay of a first buffer in the second electronic device, and the first buffer is used to buffer multimedia data to be played; The first buffer information is used to trigger the first electronic device to determine the second buffer information corresponding to the second buffer, and to perform a target operation when it is determined that the second electronic device is at risk of playback jamming based on the first buffer information and the second buffer information; the second buffer is used to buffer multimedia data in the first electronic device to be transmitted to the second electronic device, the second buffer information is used to indicate the amount of multimedia data stored in the second buffer, and the target operation is used to reduce the amount of multimedia data stored in the second buffer.

18. An electronic device, characterized in that: The electronic device comprises a memory, a processor and a transceiver unit. The memory stores a computer program. When the computer program is executed by the processor, the electronic device implements the method according to any one of claims 1 to 12 or 13 to 15.

19. A computer program product, characterized in that The method comprises a computer program, and when the computer program is executed by a processor in an electronic device, the electronic device implements the method according to any one of claims 1 to 12 or 13 to 15.

Images