Audio transmission method, terminal, electronic device and storage medium

Abstract

The present disclosure relates to an audio transmission method, terminal, electronic device and storage medium. The audio transmission method includes: a first terminal sends multiple audio packages to a second terminal, wherein each audio package in the multiple audio packages includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; the first terminal receives feedback from the second terminal regarding the received audio package; the first terminal determines the audio package received by the second terminal based on the feedback, and determines the audio package to be retransmitted based on the redundant audio frames in the audio package received by the second terminal; and the first terminal sends the audio package to be retransmitted to the second terminal.

Description

Technical Field

[0001] The present disclosure relates to the field of signal processing, and in particular to an audio transmission method, terminal, electronic device, and storage medium. Background Art

[0002] During real-time communication, audio data loss is inevitable due to network conditions. When packet loss occurs, the sound quality heard by the receiver becomes poor, or even inaudible. Therefore, recovering lost audio data during real-time communication is essential.

[0003] The main methods for recovering lost audio data are forward error correction (FEC) technology and retransmission technology. The main method of FEC technology is that the sender adds a part of redundant information when sending audio packets, so that the receiver can recover the lost data through the redundant information when network packet loss occurs. Retransmission technology means that after the receiver finds that the received audio packet is lost, it notifies the sender to resend the lost audio packet. However, FEC technology may be difficult to effectively recover lost audio data in the case of high packet loss rate, and retransmission technology will increase the bit rate due to excessive retransmission operations in the case of high packet loss rate, thereby causing audio delay. In other words, both technologies are difficult to efficiently recover lost audio data in the case of high packet loss rate. Summary of the Invention

[0004] The present disclosure provides an audio transmission method, an apparatus, an electronic device, and a storage medium to at least solve the problem in the related art that it is difficult to efficiently recover lost audio data in the case of a high packet loss rate.

[0005] According to a first aspect of an embodiment of the present disclosure, an audio transmission method is provided, which includes: a first terminal sending multiple audio packages to a second terminal, wherein each audio package in the multiple audio packages includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; the first terminal receives feedback from the second terminal regarding the received audio package; the first terminal determines the audio package received by the second terminal based on the feedback, and determines the audio package to be retransmitted based on the redundant audio frames in the audio package received by the second terminal; and the first terminal sends the audio package to be retransmitted to the second terminal.

[0006] Optionally, determining the audio packets that need to be retransmitted based on the redundant audio frames in the audio packets received by the second terminal includes: the first terminal obtaining a first number of audio frames that have not been continuously received by the second terminal based on the redundant audio frames in the audio packets received by the second terminal; obtaining a second number of redundant audio frames contained in each audio packet; and determining the audio packets that need to be retransmitted based on the first number and the second number.

[0007] Optionally, the second number is N; the step of determining the audio package to be retransmitted based on the first number and the second number includes: judging the first number; when the first number is less than or equal to N+1, determining the audio package to be retransmitted is: taking the last audio frame in the first number of audio frames as the audio package of the audio frame at the current moment; when the first number is greater than N, determining the audio package to be retransmitted is: taking the N+1th audio frame in the first number of audio frames as the audio package of the audio frame at the current moment, and obtaining the number of remaining continuous unreceived audio frames starting from the next audio frame of the N+1th audio frame, and updating the first number to the number of remaining continuous unconnected audio frames, and returning to the step of judging the first number until it is determined that the audio package to be retransmitted is: taking the last audio frame in the remaining continuous unreceived audio frames as the audio package of the audio frame at the current moment.

[0008] Optionally, the first terminal obtains the first number of audio frames that the second terminal has not received continuously based on the redundant audio frames in the audio package received by the second terminal, including: the first terminal starts counting when it is determined that the second terminal has not received a certain audio frame based on the redundant audio frames in the audio package received by the second terminal, and stops counting when it is determined that the second terminal has received the audio frame again, thereby obtaining the first number of audio frames that the second terminal has not received continuously.

[0009] Optionally, the audio transmission method further includes: the first terminal acquiring a packet loss rate of audio packets sent to the second terminal; and adjusting the preset number according to the packet loss rate.

[0010] Optionally, adjusting the preset number according to the packet loss rate includes: increasing the preset number when the packet loss rate increases; and decreasing the preset number when the packet loss rate decreases.

[0011] Optionally, the feedback includes a packet identifier of a received audio packet, wherein the packet identifier includes index information of the audio packet received by the second terminal.

[0012] According to a second aspect of an embodiment of the present disclosure, an audio transmission method is provided, which includes: receiving multiple audio packages from a first terminal, wherein each of the multiple audio packages includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; sending feedback for the received audio package to the first terminal; receiving an audio package to be retransmitted sent by the first terminal, wherein the audio package to be retransmitted is determined by the first terminal based on the feedback to be received by the second terminal, and is determined based on the redundant audio frames in the audio package received by the second terminal.

[0013] Optionally, the audio transmission method further includes: the second terminal restoring, based on the redundant audio frame included in the received audio packet, an audio frame in the audio packet not received by the second terminal that corresponds to the redundant audio frame.

[0014] Optionally, the audio transmission method further includes: the second terminal sending feedback on a packet loss rate of audio packets sent by the first terminal to the first terminal.

[0015] Optionally, the feedback includes a packet identifier of a received audio packet, wherein the packet identifier includes index information of the audio packet received by the second terminal.

[0016] According to a third aspect of an embodiment of the present disclosure, a first terminal for audio transmission is provided, the first terminal comprising: an audio sending unit configured to send multiple audio packages to a second terminal, wherein each of the multiple audio packages includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; a feedback receiving unit configured to receive feedback from the second terminal regarding the received audio package; a retransmission determination unit configured to: determine the audio package received by the second terminal based on the feedback, and determine the audio package to be retransmitted based on the redundant audio frames in the audio package received by the second terminal; wherein the audio sending unit is further configured to send the audio package to be retransmitted to the second terminal.

[0017] Optionally, determining the audio packets that need to be retransmitted based on the redundant audio frames in the audio packets received by the second terminal includes: obtaining a first number of audio frames that have not been continuously received by the second terminal based on the redundant audio frames in the audio packets received by the second terminal; obtaining a second number of redundant audio frames contained in each audio packet; and determining the audio packets that need to be retransmitted based on the first number and the second number.

[0018] Optionally, the second number is N; the step of determining the audio package to be retransmitted based on the first number and the second number includes: judging the first number; when the first number is less than or equal to N+1, determining the audio package to be retransmitted is: taking the last audio frame in the first number of audio frames as the audio package of the audio frame at the current moment; when the first number is greater than N, determining the audio package to be retransmitted is: taking the N+1th audio frame in the first number of audio frames as the audio package of the audio frame at the current moment, and obtaining the number of remaining continuous unreceived audio frames starting from the next audio frame of the N+1th audio frame, and updating the first number to the number of remaining continuous unconnected audio frames, and returning to the step of judging the first number until it is determined that the audio package to be retransmitted is: taking the last audio frame in the remaining continuous unreceived audio frames as the audio package of the current audio frame.

[0019] Optionally, the method of obtaining the first number of audio frames that the second terminal has not received continuously based on the redundant audio frames in the audio package received by the second terminal includes: starting counting when it is determined that the second terminal has not received a certain audio frame based on the redundant audio frames in the audio package received by the second terminal, and stopping counting when it is determined that the audio frame is received again based on the redundant audio frames in the received audio package, thereby obtaining the first number of audio frames that the second terminal has not received continuously.

[0020] Optionally, the first terminal further includes: an adjusting unit configured to: obtain a packet loss rate of audio packets sent to the second terminal; and adjust the preset number according to the packet loss rate.

[0021] Optionally, adjusting the preset number according to the packet loss rate includes: increasing the preset number when the packet loss rate increases; and decreasing the preset number when the packet loss rate decreases.

[0022] Optionally, the feedback includes a packet identifier of a received audio packet, wherein the packet identifier includes index information of the audio packet received by the second terminal.

[0023] According to a fourth aspect of an embodiment of the present disclosure, a second terminal for audio transmission is provided, the second terminal comprising: an audio receiving unit configured to receive multiple audio packets from a first terminal, wherein each of the multiple audio packets includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; a feedback sending unit configured to send feedback for the received audio packet to the first terminal; wherein the audio receiving unit is further configured to receive an audio packet to be retransmitted sent by the first terminal, the audio packet to be retransmitted is an audio packet determined by the first terminal to be received by the second terminal based on the feedback, and is determined based on the redundant audio frames in the audio packet received by the second terminal.

[0024] Optionally, the second terminal further includes: a recovery unit configured to recover, based on the redundant audio frame included in the received audio package, an audio frame in the audio package not received by the second terminal that corresponds to the redundant audio frame.

[0025] Optionally, the feedback sending unit is further configured to: send feedback on a packet loss rate of audio packets sent by the first terminal to the first terminal.

[0026] Optionally, the feedback includes a packet identifier of a received audio packet, wherein the packet identifier includes index information of the audio packet received by the second terminal.

[0027] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, characterized in that it includes: at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, prompt the at least one processor to execute the audio transmission method as described above.

[0028] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium storing instructions is provided, characterized in that when the instructions are executed by at least one processor, the at least one processor is prompted to execute the audio transmission method as described above.

[0029] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, wherein the computer instructions implement the audio transmission method as described above when executed by a processor.

[0030] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: According to the audio transmission method performed by the first terminal in the embodiments of the present disclosure, since each audio packet not only includes the audio frame at the current moment and a preset number of redundant audio frames before the current moment, but also determines the audio packet received by the second terminal based on the feedback of the second terminal for the received audio packet, and determines the audio packet to be retransmitted based on the redundant audio frames in the audio packet received by the second terminal, therefore, compared with the traditional FEC scheme, in the case of high packet loss rate, the above-mentioned audio transmission method of the present disclosure can also further recover the lost audio data through active retransmission, and compared with the traditional retransmission scheme, due to the selective retransmission, the bit rate can be effectively saved under high packet loss rate. Therefore, the audio transmission method of the present disclosure that combines FEC technology and retransmission technology makes it possible to efficiently recover lost audio data under high packet loss rate.

[0031] According to the audio transmission method performed by the second terminal in an embodiment of the present disclosure, since the audio packet received from the first terminal includes not only the audio frame at the current moment but also a preset number of redundant audio frames before the current moment, and the audio packet to be retransmitted sent by the first terminal can be received (the audio packet to be retransmitted is the audio packet received by the second terminal determined by the first terminal based on the feedback, and is determined based on the redundant audio frames in the audio packet received by the second terminal), therefore, in the case of a high packet loss rate, the selectively retransmitted audio packet can be received, thereby efficiently recovering the lost audio data.

[0032] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate exemplary embodiments consistent with the present disclosure, and together with the description, are used to explain the principles of the present disclosure, and do not constitute an undue limitation of the present disclosure.

[0034] Figure 1 is an exemplary system architecture in which exemplary embodiments of the present disclosure may be applied;

[0035] Figure 2 is a flowchart of an audio transmission method performed by a first terminal according to an exemplary embodiment of the present disclosure;

[0036] Figure 3 is an example of the format of an audio packet carrying redundant audio frames according to an exemplary embodiment of the present disclosure.

[0037] Figure 4is a schematic diagram of determining an audio packet that needs to be retransmitted in an audio transmission method according to an exemplary embodiment of the present disclosure;

[0038] Figure 5 is a flowchart of an audio transmission method performed by a second terminal according to an exemplary embodiment of the present disclosure;

[0039] Figure 6 is a block diagram illustrating a first terminal for audio transmission according to an exemplary embodiment of the present disclosure;

[0040] Figure 7 is a block diagram illustrating a second terminal for audio transmission according to another exemplary embodiment of the present disclosure;

[0041] Figure 8 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0043] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0044] It should be noted that the phrase "at least one of the items" in this disclosure includes three types of parallel situations: "any one of the items", "a combination of any multiple items of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. For another example, "performing at least one of step 1 and step 2" includes the following three parallel situations: (1) performing step 1; (2) performing step 2; and (3) performing steps 1 and 2.

[0045] Figure 1 An exemplary system architecture 100 is shown in which exemplary embodiments of the present disclosure may be applied.

[0046] like Figure 1As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc. Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages (such as video data upload requests, video data download requests, etc.). Various communication client applications may be installed on the terminal devices 101, 102, 103, such as audio and video communication software, audio and video recording software, instant communication software, conference software, email clients, social platform software, etc. The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with display screens and capable of playing, recording, and editing audio and video, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When the terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above, and can be implemented as multiple software or software modules (for example, to provide distributed services) or as a single software or software module. No specific limitations are given here.

[0047] The terminal devices 101, 102, and 103 may be equipped with image acquisition devices (such as cameras) to collect video data. In practice, the smallest visual unit that makes up a video is a frame. Each frame is a static image. Synthesizing a temporally continuous sequence of frames together forms a dynamic video. In addition, the terminal devices 101, 102, and 103 may also be equipped with components for converting electrical signals into sounds (such as speakers) to play sounds, and may also be equipped with devices for converting analog audio signals into digital audio signals (such as microphones) to collect sounds. In addition, the terminal devices 101, 102, and 103 can communicate with each other by voice or video, for example, in real-time voice or video.

[0048] The server 105 may be a server that provides various services, such as a background server that supports multimedia applications installed on the terminal devices 101, 102, and 103. The background server may parse, store, and process received audio and video data upload requests and other data, and may also receive audio and video data download requests sent by the terminal devices 101, 102, and 103, and feed the audio and video data indicated by the audio and video data download requests back to the terminal devices 101, 102, and 103.

[0049] It should be noted that the server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitations are given here.

[0050] It should be noted that the audio transmission method provided in the embodiments of the present disclosure is generally performed by a terminal device, or can also be performed collaboratively by a terminal device and a server. Accordingly, the first terminal and the second terminal mentioned in the embodiments of the present disclosure can be set in the terminal device, or in both the terminal device and the server.

[0051] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the embodiment is merely illustrative. Any number of terminal devices, networks, and servers may be provided as required, and the present disclosure does not limit this.

[0052] Figure 2 1 is a flow chart of an audio transmission method performed by a first terminal according to an exemplary embodiment of the present disclosure. Figure 2 In step S210, the first terminal sends multiple audio packets to the second terminal. According to an exemplary embodiment, each of the multiple audio packets includes an audio frame at the current moment and a preset number of redundant audio frames preceding the current moment. For example, in addition to the audio frame preceding the current moment, each audio packet may also carry redundant audio frames from the n previous moments, where n may be, for example, 3, but is not limited thereto. As an example, the multiple audio packets may be sent sequentially in chronological order, and each audio packet may have a packet sequence number. For example, a packet identifier for the audio packet may be included in the packet header of the audio packet. For example, the packet identifier may include index information for the audio packet. The index information may be, for example, a packet sequence number, but is not limited thereto. Because the audio packet includes the preset number of redundant audio frames preceding the current moment, if packet loss occurs during transmission and an audio packet received by the second terminal after the lost packet contains redundant audio frames corresponding to the lost packet, the receiving terminal can use these redundant audio frames to recover the audio frames corresponding to the redundant audio frames in the lost packet. This allows faster recovery of lost audio data than retransmitting the lost packet, thereby reducing audio latency.

[0053] In step S220, the first terminal receives feedback from the second terminal regarding the received audio packet. As an example, the feedback may include the packet identifier of the received audio packet, but is not limited thereto. For example, the packet identifier may include index information of the audio packet received by the second terminal, and the index information may be, for example, a packet sequence number. According to an exemplary embodiment, if the second terminal successfully receives the audio packet, the second terminal may send confirmation feedback to the first terminal, such as an acknowledgment character ACK. If the second terminal fails to successfully receive the audio packet (i.e., there is packet loss), the first terminal will not receive confirmation feedback for the audio packet. For example, the confirmation feedback may include the packet sequence number of the received audio packet. In this way, the first terminal can know which audio packets have been received by the second terminal based on the confirmation feedback. In the present disclosure, if the first terminal receives feedback from the second terminal regarding the received audio packet, it is considered that the first terminal has received feedback for all audio frames in the audio packet. For example, if an audio packet is an audio packet with ACK, all audio frames included in the audio packet are audio frames with ACK.

[0054] In step S230, the first terminal determines the audio packets received by the second terminal based on the feedback, and determines the audio packets to be retransmitted based on the redundant audio frames in the audio packets received by the second terminal. Figure 3As shown, if audio packets 7, 8, and 9 are lost during transmission, but the first terminal receives feedback for packet 10, the first terminal can determine that the second terminal has received audio packet 10 based on the feedback. Since packet 10 contains redundant audio frames 7, 8, and 9 corresponding to audio packets 7, 8, and 9, the second terminal can use the redundant audio frames 7, 8, and 9 contained in packet 10 to respectively restore the audio frames corresponding to the redundant audio frames in the lost audio packets 7, 8, and 9 (that is, the audio frames at the current moment in audio packets 7, 8, and 9). In this case, the first terminal will believe that the audio frame corresponding to the redundant audio frame is also an audio frame with confirmation feedback, that is, the first terminal will believe that the audio frame corresponding to the redundant audio frame has been received by the second terminal, and therefore, the audio packet corresponding to the audio frame will not be retransmitted. In other words, when the audio packet received by the second terminal after the lost audio packet contains the redundant audio frame corresponding to the lost audio packet, the first terminal does not need to retransmit the lost audio packet. Conversely, when the audio packet received by the second terminal after the lost audio packet does not contain the redundant audio frame corresponding to the lost audio packet, the first terminal needs to determine the audio packet to be retransmitted based on the redundant audio frames in the audio packet received by the second terminal. However, since each audio packet also carries redundant audio frames corresponding to other audio packets, in the present invention, not every unreceived audio packet is retransmitted, but audio packets are selectively retransmitted.

[0055] Alternatively, according to an exemplary embodiment, although Figure 2 Not shown, but Figure 2The method shown may also include: the first terminal obtaining a packet loss rate of audio packets sent to the second terminal; and adjusting the preset number based on the packet loss rate. For example, the first terminal may calculate the packet loss rate based on the feedback from the second terminal regarding the received audio packets, or the first terminal may directly receive feedback on the packet loss rate from the second terminal. For example, the first terminal may calculate how many audio packets are received by the second terminal using the packet identifier (e.g., packet sequence number) included in the feedback from the second terminal regarding the received audio packets, thereby calculating the packet loss rate. Alternatively, the second terminal may know how many audio packets are received and how many audio packets are lost based on the packet identifier (e.g., packet sequence number) in the received audio packets, thereby calculating the packet loss rate, and then feeding the packet loss rate back to the first terminal. As an example, the first terminal may periodically receive feedback on the packet loss rate from the second terminal, for example, the first terminal may receive feedback on the packet loss rate from the second terminal at every moment, or may receive feedback on the packet loss rate from the second terminal every few moments. The first terminal may then adjust the number of redundant audio frames added to the audio packet accordingly based on the packet loss rate fed back by the second terminal. According to an exemplary embodiment, when the packet loss rate increases, the preset number is increased; when the packet loss rate decreases, the preset number is decreased. For example, the greater the packet loss rate, the more redundant audio frames from the previous time point can be included in each audio packet. Conversely, the smaller the packet loss rate, the fewer redundant audio frames from the previous time point can be included in each audio packet.

[0056] Figure 3 is an example of the format of an audio packet carrying redundant audio frames according to an exemplary embodiment of the present disclosure. Figure 3 In the example shown, each audio packet carries not only the audio frame at the current moment but also redundant audio frames from three moments before the current moment. However, as described above, the number of redundant audio frames can be adaptively adjusted, for example, based on the packet loss rate, to facilitate more efficient packet loss recovery for the second terminal.

[0057] Figure 4 Schematic diagram of determining the audio packets to be retransmitted in the audio transmission method of the exemplary embodiment of the present disclosure. Figure 4 , a method for determining audio packets that need to be retransmitted is introduced.

[0058] According to an exemplary embodiment, specifically, in order to determine an audio packet that needs to be retransmitted, first, the first terminal obtains a first number of audio frames that the second terminal has not received continuously based on redundant audio frames in the audio packet received by the second terminal; second, obtains a second number of redundant audio frames contained in each audio packet; and finally, determines the audio packet that needs to be retransmitted based on the first number and the second number. For example, the first number can be obtained in the following manner: the first terminal starts counting when it determines that the second terminal has not received a certain audio frame based on redundant audio frames in the audio packet received by the second terminal, and stops counting when it determines that the second terminal has received an audio frame again, thereby obtaining the first number of audio frames that the second terminal has not received continuously.

[0059] For example, in Figure 4 In the example of , it is assumed that each audio packet includes not only the audio frame at the current moment, but also the redundant audio frames of the 6 moments before the current moment, that is, each audio packet carries 6 redundant audio frames. Figure 4 In the example shown in (a), if the audio frame before audio frame 8 (for example, audio frame 7) is an audio frame with confirmation feedback, it is considered that the audio frame before audio frame 8 is received by the second terminal. When it is determined that the second terminal has not received audio frame 8, counting begins. At this time, the first number is equal to 1. If it is then determined that the second terminal has not received audio frame 9, counting continues. At this time, the first number is equal to 2. Subsequently, if it is determined that the second terminal has not received audio frame 10, counting continues. At this time, the first number is equal to 3. If it is determined that the second terminal has not received audio frame 11, counting continues. At this time, the first number is equal to 4. Until counting is stopped when it is determined that the second terminal has received an audio frame again, for example, counting is stopped when it is determined that the second terminal has received audio frame 12. At this time, the first number = 4 can be obtained.

[0060] Here, it should be noted that the second terminal did not receive audio frames 8, 9 or 10, indicating that the second terminal not only did not receive the audio package with audio frames 8, 9 or 10 as the audio frames at the current moment, but also that other audio packets received by the second terminal did not contain redundant audio frames corresponding to audio frames 8, 9 or 10. This is because, as mentioned above, if other audio packets received by the second terminal contain redundant audio frames corresponding to audio frames 8, 9 or 10, it will be considered that audio frames 8, 9 or 10 have been received by the second terminal, that is, audio frames 8, 9 or 10 are also considered to be audio frames with ACK.

[0061] As mentioned above, since each audio packet can carry redundant audio frames (even a retransmitted audio packet can carry a preset number of redundant audio frames), when determining the audio packets that need to be retransmitted, the present invention not only considers the first number, but also considers the second number of redundant audio frames contained in each audio packet (assuming the second number is N), and determines the audio packets that need to be retransmitted based on the first number and the second number.

[0062] Specifically, for example, the audio packets to be retransmitted may be determined by the following steps:

[0063] Step 1: Determine the first quantity.

[0064] Step 2: When the first number is less than or equal to N+1, the audio packet to be retransmitted is determined as follows: the last audio frame in the first number of audio frames is used as the audio packet of the audio frame at the current moment. Here, the last audio frame in the first number of audio frames is used as the audio packet of the audio frame at the current moment, in addition to including the last audio frame in the first number of audio frames, it also includes a preset number of audio frames before the last audio frame, that is, it also includes a preset number of redundant audio frames before the current moment (that is, the moment corresponding to the last audio frame).

[0065] For example, Figure 4 As shown in (a), if the number of redundant audio frames included in each audio packet is 6 (i.e., N is 6), and audio frames 8 to 11 are determined to be the number of consecutive unreceived audio frames (i.e., the first number is 4), then since the first number is 4 and N is 6, the first number is less than N+1. Therefore, the audio packet to be retransmitted is determined to be the audio packet with the last audio frame (i.e., audio frame 11) of the four consecutive unreceived audio frames as the audio frame at the current moment. That is, the audio packet to be retransmitted includes, in addition to audio frame 11, the six redundant audio frames preceding audio frame 11, that is, the redundant audio frames 5 to 10 corresponding to audio frame 11 at the six moments preceding the current moment. Since the audio packet to be retransmitted includes redundant audio frames 8 to 10, after receiving the retransmitted audio packet, the second terminal can use the redundant audio frames received in the retransmitted audio packet to recover the audio frames corresponding to these redundant audio frames in the audio packet that it has not received.

[0066] Step 3: When the first number is greater than N, the audio packet to be retransmitted is determined as follows: the N+1th audio frame in the first number of audio frames is used as the audio packet of the audio frame at the current moment, and the number of the remaining continuous unreceived audio frames starting from the next audio frame of the N+1th audio frame is obtained, and the first number is updated to the number of the remaining continuous unconnected audio frames, and the step of judging the first number is returned until it is determined that the audio packet to be retransmitted is: the last audio frame in the remaining continuous unreceived audio frames is used as the audio packet of the audio frame at the current moment. Here, the N+1th audio frame in the first number of audio frames is used as the audio packet of the audio frame at the current moment, in addition to the audio frame before the N+1th audio frame, it also includes a preset number of audio frames before the N+1th audio frame, that is, it also includes a preset number of redundant audio frames before the current moment (that is, the moment corresponding to the N+1th audio frame). Similarly, the last audio frame among the remaining consecutive unreceived audio frames is used as the audio package of the audio frame at the current moment. In addition to including the last audio frame among the remaining consecutive unreceived audio frames, it also includes a preset number of audio frames before the last audio frame, that is, it also includes a preset number of redundant audio frames before the current moment (that is, the moment corresponding to the last audio frame).

[0067] For example, Figure 4 As shown in (b), if the number of redundant audio frames included in each audio packet is 6 (i.e., N is 6), audio frames 8 to 16 are determined to be 9 consecutive unreceived audio frames (i.e., the first number is 9), then since the first number is 9 and N is 6, the first number is greater than N. Therefore, it is first determined that the audio packet to be retransmitted is the audio packet with the N+1th audio frame (i.e., the 7th audio frame, that is, audio frame 14) as the audio frame at the current moment. In addition to audio frame 14, the audio packet also includes the 6 audio frames before audio frame 14, that is, audio frames 8 to audio frames 13.

[0068] Next, the number of remaining consecutive unreceived audio frames starting from the next audio frame of the 7th audio frame is obtained, that is, the number of consecutive unreceived audio frames is recalculated starting from the 8th audio frame (i.e., audio frame 15). Since there are only two consecutive unreceived audio frames left, the number of remaining consecutive unreceived audio frames is 2. At this time, the first number is updated to 2, and then the process returns to step 1. Since the first number = 2 and N = 6 at this time, the first number is less than N+1. Therefore, the audio packet to be retransmitted is determined to be: the last audio frame (i.e., audio frame 16) among the remaining consecutive unreceived audio frames is used as the audio frame at the current moment. In addition to audio frame 16, this audio packet also contains a preset number of audio frames before audio frame 16.

[0069] Finally, for Figure 4 In the example shown in (b), the audio packets that need to be retransmitted are only the audio packet with audio frame 14 as the audio frame at the current moment and the audio packet with audio frame 16 as the audio frame at the current moment. After the second terminal receives the two retransmitted audio packets, since the first retransmitted audio packet includes redundant audio frames 8 to 13 in addition to audio frame 14, and the second retransmitted audio packet includes at least redundant audio frame 15 in addition to audio frame 16, the second terminal can use the redundant audio frames in the received retransmitted audio packet to restore the audio frames corresponding to these redundant audio frames in the audio packet that it has not received, for example, restore audio frames 8 to 13 and audio frame 15.

[0070] Finally, in step S240, the first terminal sends the audio packet to be retransmitted to the second terminal.

[0071] Since the first terminal does not directly retransmit the audio packet when the second terminal does not receive the audio packet, but retransmits it selectively, the bit rate can be effectively saved in the case of a high packet loss rate. This is because each audio packet also has a header, and the header itself consumes a lot of bit rate, so combining several audio frames into an audio packet for retransmission will save bit rate compared to retransmitting each lost audio packet separately.

[0072] The above has been combined Figures 2 to 4 An audio transmission method performed by a first terminal according to an exemplary embodiment of the present disclosure is described. Since each audio packet includes not only the audio frame at the current moment and a preset number of redundant audio frames before the current moment, but also the audio packet received by the second terminal is determined based on the feedback of the second terminal for the received audio packet, and the audio packet to be retransmitted is determined based on the redundant audio frames in the audio packet received by the second terminal, therefore, compared with the traditional FEC scheme, in the case of high packet loss rate, the above-mentioned audio transmission method of the present disclosure can further recover the lost audio data through active retransmission. Compared with the traditional retransmission scheme, due to the selective retransmission, the bit rate can be effectively saved under high packet loss rate. Therefore, the audio transmission method of the present disclosure that combines FEC technology and retransmission technology makes it possible to efficiently recover lost audio data under high packet loss rate.

[0073] Figure 5 It is a flowchart of the audio transmission method executed by the second terminal according to an exemplary embodiment of the present disclosure. It should be noted that the contents related to the operations executed by the second terminal mentioned above in the process of describing the audio transmission method executed by the first terminal are applicable to Figure 5 The audio transmission method shown is performed by the second terminal, so the relevant content is not repeated here.

[0074] Reference Figure 5 In step S510, multiple audio packets are received from the first terminal. Each of the multiple audio packets includes an audio frame at the current moment and a preset number of redundant audio frames before the current moment. Because the audio packet includes the preset number of redundant audio frames before the current moment, if packet loss occurs and an audio packet received by the receiving end after the lost packet contains redundant audio frames corresponding to the audio frames in the lost packet, the second terminal can use these redundant audio frames to recover the audio frames in the lost packet that correspond to the redundant audio frames.

[0075] In step S520, feedback regarding the received audio packet is sent to the first terminal. For example, when an audio packet is received, confirmation feedback regarding the audio packet is sent to the transmitting end. For example, the confirmation feedback may include the packet identifier of the received audio packet. For example, the packet identifier includes index information of the audio packet received by the second terminal. The index information may be, for example, the packet sequence number of the received audio packet. For example, based on the packet sequence number of the received audio packet, the second terminal can determine how many audio packets have been received and how many audio packets have been lost, thereby calculating the packet loss rate and then feeding the packet loss rate back to the transmitting end.

[0076] Therefore, optionally, although Figure 5 Not shown, but Figure 5 The illustrated method may further include: the second terminal sending feedback on the packet loss rate of the audio packets sent by the first terminal to the first terminal. As an example, the second terminal may periodically send feedback on the packet loss rate to the first terminal, for example, at every moment or every few moments. The first terminal may then adjust a preset number of redundant audio frames included in the audio packets based on the packet loss rate feedback from the receiving terminal.

[0077] In addition, in step S530, the audio packet to be retransmitted sent by the first terminal is received. Here, as mentioned in the description of step 2 above, the audio packet to be retransmitted is the audio packet received by the second terminal determined by the first terminal according to the feedback, and is determined according to the redundant audio frames in the audio packet received by the second terminal. Figures 2 to 4 The description of how the first terminal determines the audio packet to be retransmitted is mentioned, so it is not repeated here.

[0078] Since the audio packet received by the second terminal includes both the audio frame at the current moment and a preset number of redundant audio frames before the current moment, the second terminal can also restore the audio packet corresponding to the redundant audio frame in the audio packet not received by the second terminal based on the redundant audio frame included in the received audio packet. Figure 5 The audio transmission method shown may further include: the second terminal restoring an audio frame corresponding to the redundant audio frame in the audio package not received by the second terminal based on the redundant audio frame included in the received audio package.

[0079] because Figure 5 The audio transmission method shown includes, in addition to the audio frames at the current moment, a preset number of redundant audio frames before the current moment in the audio packets received from the first terminal, and can receive the audio packets that need to be retransmitted sent by the first terminal (the audio packets that need to be retransmitted are the audio packets received by the second terminal determined by the first terminal based on the feedback, and are determined based on the redundant audio frames in the audio packets received by the second terminal). Therefore, in the case of a high packet loss rate, selectively retransmitted audio packets can be received, thereby efficiently recovering the lost audio data.

[0080] Figure 6 is a block diagram illustrating a first terminal for audio transmission according to an exemplary embodiment of the present disclosure.

[0081] Reference Figure 6 , the first terminal 600 may include an audio sending unit 610, a feedback receiving unit 620 and a retransmission determination unit 630. Specifically, the audio sending unit 610 may be configured to send multiple audio packets to the second terminal, wherein each of the multiple audio packets includes an audio frame at the current moment and a preset number of redundant audio frames before the current moment. The feedback receiving unit 620 may be configured to receive feedback from the second terminal regarding the received audio packet. The retransmission determination unit 630 may be configured to: determine the audio packet received by the second terminal based on the feedback, and determine the audio packet to be retransmitted based on the redundant audio frames in the audio packet received by the second terminal. In addition, the audio sending unit 610 may also be configured to send the audio packet to be retransmitted to the second terminal.

[0082] because Figure 2 The audio transmission method shown can be Figure 6 The first terminal 600 shown is executed, and the above has been Figure 2 and Figure 4 The details related to the operations performed by the audio transmitting unit 610, the feedback receiving unit 620 and the retransmission determining unit 630 are described in the description of Figure 6Any details about the operations performed by the units in Figures 2 to 4 The corresponding descriptions are not repeated here.

[0083] Furthermore, it should be noted that although the first terminal 600 is described above as being divided into units for performing corresponding processes, it is clear to those skilled in the art that the processes performed by the aforementioned units may also be performed without any specific unit division or clear demarcation between the units in the first terminal 600. Furthermore, the first terminal 600 may further include other units, such as a storage unit, a data processing unit, and the like.

[0084] Figure 7 is a block diagram illustrating a second terminal for audio transmission according to another exemplary embodiment of the present disclosure.

[0085] Reference Figure 7 , the second terminal 700 may include an audio receiving unit 710 and a feedback sending unit 720. Specifically, the audio receiving unit 710 may be configured to receive multiple audio packets from the first terminal, wherein each of the multiple audio packets includes an audio frame at the current moment and a preset number of redundant audio frames before the current moment. The feedback sending unit 720 may be configured to send feedback for the received audio packet to the first terminal. In addition, the audio receiving unit 710 may also be configured to: receive an audio packet that needs to be retransmitted sent by the first terminal, the audio packet that needs to be retransmitted is the audio packet received by the second terminal determined by the first terminal based on the feedback, and is determined based on the redundant audio frames in the audio packet received by the second terminal. Optionally, the second terminal 700 may further include a recovery unit (not shown), which may restore the audio frame corresponding to the redundant audio frame in the audio packet not received by the second terminal based on the redundant audio frames included in the received audio packet.

[0086] because Figure 5 The audio transmission method shown can be Figure 7 The second terminal 700 shown is executed, and the above has been Figure 5 The details of the operations performed by the audio receiving unit 710 and the feedback sending unit 720 are described in the description of FIG. Figure 7 Any details about the operations performed by the units in Figure 5 The corresponding descriptions are not repeated here.

[0087] Furthermore, it should be noted that although the second terminal 700 is described above as being divided into units for performing corresponding processes, it is clear to those skilled in the art that the processes performed by the aforementioned units may also be performed without any specific unit division or clear demarcation between the units in the second terminal 700. Furthermore, the second terminal 700 may further include other units, such as a storage unit, a data processing unit, and the like.

[0088] Figure 8 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

[0089] Reference Figure 8 The electronic device 800 may include at least one memory 801 and at least one processor 802, wherein the at least one memory stores computer-executable instructions. When the computer-executable instructions are executed by the at least one processor, the at least one processor 802 is prompted to perform the audio transmission method according to the embodiment of the present disclosure ( Figure 2 The audio transmission method performed by the first terminal or Figure 5 The audio transmission method performed by the second terminal is shown).

[0090] As an example, the electronic device can be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device is not necessarily a single electronic device, but can also be any collection of devices or circuits that can execute the above-mentioned instructions (or instruction sets) individually or in combination. The electronic device can also be part of an integrated control system or system manager, or can be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0091] In electronic devices, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0092] The processor can execute instructions or codes stored in the memory, wherein the memory can also store data. Instructions and data can also be sent and received over the network via the network interface device, wherein the network interface device can use any known transmission protocol.

[0093] The memory may be integrated with the processor, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. Furthermore, the memory may comprise a separate device, such as an external disk drive, a storage array, or any other storage device usable by a database system. The memory and processor may be operatively coupled or may be in communication with each other, for example, via an I / O port, a network connection, or the like, such that the processor can access files stored in the memory.

[0094] In addition, the electronic device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0095] According to an embodiment of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein, when the instructions are executed by at least one processor, the at least one processor is prompted to perform the audio transmission method according to the exemplary embodiment of the present disclosure ( Figure 2 The audio transmission method performed by the first terminal or Figure 5Examples of computer-readable storage media include read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device configured to store computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The instructions or computer program in the above-mentioned computer-readable storage medium can be executed in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0096] According to an embodiment of the present disclosure, a computer program product may also be provided, the computer program including computer instructions, which, when executed by a processor, implement the audio transmission method according to the exemplary embodiment of the present disclosure ( Figure 2 The audio transmission method performed by the first terminal or Figure 5 The audio transmission method performed by the second terminal is shown).

[0097] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are to be defined by the claims.

Claims

1. An audio transmission method, comprising: The first terminal sends a plurality of audio packets to the second terminal, wherein each of the plurality of audio packets includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; receiving, by the first terminal, feedback from the second terminal regarding the received audio packet; The first terminal determines, based on the feedback, an audio packet received by the second terminal, obtains a first number of audio frames not continuously received by the second terminal based on redundant audio frames in the audio packet received by the second terminal, obtains a second number of redundant audio frames included in each audio packet, and determines an audio packet to be retransmitted based on the first number and the second number; The first terminal sends the audio packet to be retransmitted to the second terminal, The second number is N; and determining the audio packets to be retransmitted according to the first number and the second number includes: determining the first quantity; When the first number is less than or equal to N+1, determining that the audio packet to be retransmitted is: taking the last audio frame of the first number of audio frames as the audio packet of the audio frame at the current moment; When the first number is greater than N, the audio package that needs to be retransmitted is determined as follows: the N+1th audio frame in the first number of audio frames is used as the audio package of the audio frame at the current moment, and the number of the remaining consecutive unreceived audio frames starting from the next audio frame of the N+1th audio frame is obtained, and the first number is updated to the number of the remaining consecutive unreceived audio frames, and the step of judging the first number is returned until it is determined that the audio package that needs to be retransmitted is: the last audio frame in the remaining consecutive unreceived audio frames is used as the audio package of the audio frame at the current moment.

2. The audio transmission method according to claim 1, wherein: The first terminal obtains, based on redundant audio frames in an audio packet received by the second terminal, a first number of audio frames not continuously received by the second terminal, including: The first terminal starts counting based on the redundant audio frames in the audio packet received by the second terminal when it is determined that the second terminal has not received a certain audio frame, and stops counting when it is determined that the second terminal has received the audio frame again, thereby obtaining a first number of audio frames that the second terminal has not received continuously.

3. The audio transmission method according to claim 1, further comprising: Acquiring, by the first terminal, a packet loss rate of audio packets sent to the second terminal; The preset number is adjusted according to the packet loss rate.

4. The audio transmission method according to claim 3, wherein: Adjusting the preset number according to the packet loss rate includes: When the packet loss rate increases, increasing the preset number; When the packet loss rate decreases, the preset number is reduced.

5. The audio transmission method according to claim 1, wherein: The feedback includes a packet identifier of the received audio packet, wherein the packet identifier includes index information of the audio packet received by the second terminal.

6. An audio transmission method, comprising: Receiving a plurality of audio packets from a first terminal, wherein each of the plurality of audio packets includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; Sending feedback regarding the received audio packet to the first terminal; Receive an audio packet that needs to be retransmitted and is sent by a first terminal, where the audio packet that needs to be retransmitted is determined by the first terminal through the following operations: The first terminal determines, based on the feedback, an audio packet received by the second terminal, obtains a first number of audio frames not continuously received by the second terminal based on redundant audio frames in the audio packet received by the second terminal, obtains a second number of redundant audio frames contained in each audio packet, and determines the audio packet to be retransmitted based on the first number and the second number. The second number is N; and determining the audio packets to be retransmitted according to the first number and the second number includes: determining the first quantity; When the first number is less than or equal to N+1, determining that the audio packet to be retransmitted is: taking the last audio frame of the first number of audio frames as the audio packet of the audio frame at the current moment; When the first number is greater than N, the audio package that needs to be retransmitted is determined as follows: the N+1th audio frame in the first number of audio frames is used as the audio package of the audio frame at the current moment, and the number of the remaining consecutive unreceived audio frames starting from the next audio frame of the N+1th audio frame is obtained, and the first number is updated to the number of the remaining consecutive unreceived audio frames, and the step of judging the first number is returned until it is determined that the audio package that needs to be retransmitted is: the last audio frame in the remaining consecutive unreceived audio frames is used as the audio package of the audio frame at the current moment.

7. The audio transmission method according to claim 6, wherein: The audio transmission method further includes: the second terminal restoring, based on the redundant audio frame included in the received audio packet, an audio frame corresponding to the redundant audio frame in the audio packet not received by the second terminal.

8. The audio transmission method according to claim 6, further comprising: The second terminal sends feedback on a packet loss rate of audio packets sent by the first terminal to the first terminal.

9. The audio transmission method according to claim 6, wherein: The feedback includes a packet identifier of the received audio packet, wherein the packet identifier includes index information of the audio packet received by the second terminal.

10. A first terminal for audio transmission, comprising: an audio sending unit configured to send a plurality of audio packets to the second terminal, wherein each of the plurality of audio packets includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; a feedback receiving unit, configured to receive feedback from the second terminal regarding the received audio packet; a retransmission determination unit, configured to: determine, based on the feedback, an audio packet received by the second terminal, obtain a first number of audio frames not continuously received by the second terminal based on redundant audio frames in the audio packet received by the second terminal, obtain a second number of redundant audio frames contained in each audio packet, and determine an audio packet to be retransmitted based on the first number and the second number; The audio sending unit is further configured to send the audio packet to be retransmitted to the second terminal. The second number is N; and determining the audio packets to be retransmitted according to the first number and the second number includes: determining the first quantity; When the first number is less than or equal to N+1, determining that the audio packet to be retransmitted is: taking the last audio frame of the first number of audio frames as the audio packet of the audio frame at the current moment; When the first number is greater than N, the audio package that needs to be retransmitted is determined as follows: the N+1th audio frame in the first number of audio frames is used as the audio package of the audio frame at the current moment, and the number of the remaining continuous unreceived audio frames starting from the next audio frame of the N+1th audio frame is obtained, and the first number is updated to the number of the remaining continuous unconnected audio frames, and the step of judging the first number is returned until it is determined that the audio package that needs to be retransmitted is: the last audio frame in the remaining continuous unreceived audio frames is used as the audio package of the current audio frame. The first terminal according to claim 10 , wherein: The obtaining, based on redundant audio frames in an audio packet received by the second terminal, a first number of audio frames not continuously received by the second terminal includes: According to the redundant audio frames in the audio package received by the second terminal, counting is started when it is determined that the second terminal has not received a certain audio frame, and counting is stopped when it is determined that the audio frame is received again according to the redundant audio frames in the received audio package, and a first number of audio frames that the second terminal has not received continuously is obtained.

12. The first terminal according to claim 10, further comprising: The adjustment unit is configured to: Obtaining a packet loss rate of audio packets sent to the second terminal; The preset number is adjusted according to the packet loss rate.

13. The first terminal according to claim 12, wherein: The adjusting the preset number according to the packet loss rate includes: When the packet loss rate increases, increasing the preset number; When the packet loss rate decreases, the preset number is reduced.

14. The first terminal according to claim 10, wherein: The feedback includes a packet identifier of the received audio packet, wherein the packet identifier includes index information of the audio packet received by the second terminal.

15. A second terminal for audio transmission, comprising: An audio receiving unit configured to receive a plurality of audio packets from a first terminal, wherein each of the plurality of audio packets includes an audio frame at a current moment and a preset number of redundant audio frames before the current moment; a feedback sending unit, configured to send feedback regarding the received audio packet to the first terminal; The audio receiving unit is further configured to receive an audio packet that needs to be retransmitted and is sent by the first terminal, where the audio packet that needs to be retransmitted is determined by the first terminal through the following operations: The first terminal determines, based on the feedback, an audio packet received by the second terminal, obtains a first number of audio frames not continuously received by the second terminal based on redundant audio frames in the audio packet received by the second terminal, obtains a second number of redundant audio frames contained in each audio packet, and determines the audio packet to be retransmitted based on the first number and the second number. The second number is N; and determining the audio packets to be retransmitted according to the first number and the second number includes: determining the first quantity; When the first number is less than or equal to N+1, determining that the audio packet to be retransmitted is: taking the last audio frame of the first number of audio frames as the audio packet of the audio frame at the current moment; When the first number is greater than N, the audio package that needs to be retransmitted is determined as follows: the N+1th audio frame in the first number of audio frames is used as the audio package of the audio frame at the current moment, and the number of the remaining continuous unreceived audio frames starting from the next audio frame of the N+1th audio frame is obtained, and the first number is updated to the number of the remaining continuous unconnected audio frames, and the step of judging the first number is returned until it is determined that the audio package that needs to be retransmitted is: the last audio frame in the remaining continuous unreceived audio frames is used as the audio package of the current audio frame.

16. The second terminal according to claim 15, further comprising: The restoring unit is configured to restore, based on the redundant audio frame included in the received audio packet, an audio frame corresponding to the redundant audio frame in the audio packet not received by the second terminal.

17. The second terminal according to claim 15, wherein: The feedback sending unit is further configured to send feedback on a packet loss rate of audio packets sent by the first terminal to the first terminal.

18. The second terminal according to claim 15, wherein: The feedback includes a packet identifier of the received audio packet, wherein the packet identifier includes index information of the audio packet received by the second terminal.

19. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer-executable instructions, Wherein, when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to perform the audio transmission method according to any one of claims 1 to 9.

20. A computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by at least one processor, the at least one processor is prompted to perform the audio transmission method according to any one of claims 1 to 9.

21. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the audio transmission method according to any one of claims 1 to 9 is implemented.

Images