Data transmission method and device and electronic equipment

Abstract

The invention relates to the technical field of computers, in particular to a data transmission method and device and electronic equipment, and is used for effectively reducing the packet loss rate of a network. According to the embodiment of the invention, multimedia data is transmitted to a receiving end; on the basis of feedback information of the receiving end, when it is determined that the transmission quality of the multimedia data does not meet a set standard, triggering each data packet to be transmitted to be repeatedly transmitted; wherein after the repeated transmission is triggered, for each data packet to be transmitted, the following operations are executed: obtaining at least one target retransmission parameter corresponding to one data packet; determining a corresponding quantization strategy according to the at least one target retransmission parameter, and performing quantization processing on one data packet by adopting the obtained at least one quantization strategy to obtain at least one retransmission data packet after quantization processing; and based on the data size of the at least one retransmission data packet, sending the data packet and the at least one retransmission data packet to a receiving end by adopting the corresponding bandwidth occupation amount.

Description

Background Art

[0002] During data transmission, the instability of the transmission network can easily lead to packet loss, causing the receiving end to be unable to receive data normally. For example, in audio and video calls and live broadcast services, voice quality is mainly affected by network packet loss, and the instability of the transmission network causes the receiving end to have stuck and discontinuous sound. There are many causes of network packet loss, such as network data congestion to a certain extent triggering the network device to actively discard packets, network equipment or transmission line failures causing data packet loss, wireless network interference causing data anomalies and unable to pass verification and discard packets, etc. Network packet loss is highly random and sudden, so it poses a great challenge to the data transmission process.

[0003] In related technologies, a repetitive multiple-send strategy is usually used to solve the problem of network packet loss. The so-called repetitive multiple-send strategy refers to copying the original data packet multiple times and sending them to the receiving end at different time intervals. However, although the repetitive multiple-send strategy can solve the problem that the receiving end cannot receive the corresponding data packet due to network packet loss, the repetitive multiple-send strategy requires additional bandwidth consumption. Although the higher the repetitive multiple, the stronger the anti-packet loss capability, it also brings more serious bandwidth consumption problems. For bandwidth-constrained networks, the increase in the repetitive transmission multiple leads to increased network congestion, which will further trigger more packet loss, and then cause the receiving end to be unable to receive the transmitted data normally, seriously affecting the data transmission process between the sender and the receiver.

[0004] On the other hand, in related technologies, the repetition and multiple transmission strategies are usually pre-defined, and the control rules are limited. In the ever-changing network status, the repetition and multiple transmission strategies cannot be adjusted in time according to the network status. Therefore, when the network environment changes, the fixed repetition and multiple transmission strategies are still used, which aggravates the packet loss problem.

[0005] Therefore, using the repeated multiple transmission method cannot effectively solve the problem of network packet loss. Summary of the invention

[0006] The purpose of this application is to provide a data transmission method, device and electronic device to effectively reduce the packet loss rate of the network.

[0007] In a first aspect, the present application provides a data transmission method, applied to a sending end, the method comprising:

[0008] Transmit multimedia data to a receiving end;

[0009] Based on the feedback information of the receiving end, when it is determined that the transmission quality of the multimedia data does not meet the set standard, triggering repeated transmission of each data packet to be transmitted;

[0010] After the repeated transmission is triggered, the following operations are performed for each data packet to be transmitted:

[0011] Obtaining at least one target retransmission parameter corresponding to a data packet;

[0012] Determine the corresponding quantization strategies according to the at least one target retransmission parameter, and respectively use the obtained at least one quantization strategy to quantize the one data packet to obtain at least one retransmission data packet after quantization, wherein the data sizes of the retransmission data packets obtained by using different quantization strategies are different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy;

[0013] Based on the data size of the at least one retransmission data packet, the one data packet and the at least one retransmission data packet are sent to the receiving end using corresponding bandwidth occupancy amounts.

[0014] In a second aspect, the present application provides a data transmission method, applied to a receiving end, the method comprising:

[0015] Determining, according to the multimedia data transmitted by the sending end, feedback information used to indicate the transmission quality of the multimedia data;

[0016] Sending the feedback information to the sending end, so that when the sending end determines based on the feedback information that the transmission quality of the multimedia data does not meet the set standard, triggering repeated transmission of each data packet to be transmitted;

[0017] Receive each data packet sent by the sending end and at least one retransmission data packet corresponding to each data packet; wherein, the at least one retransmission data packet corresponding to the one data packet is obtained by the sending end after obtaining at least one target retransmission parameter corresponding to the one data packet, and then using the quantization strategy corresponding to the at least one target retransmission parameter to quantize the one data packet; wherein, the data sizes of the retransmission data packets obtained by using different quantization strategies are different, and the data size of the retransmission data packets is positively correlated with the bandwidth occupancy.

[0018] In a third aspect, the present application provides a data transmission device, including:

[0019] A transmission module, used for transmitting multimedia data to a receiving end;

[0020] A trigger module, configured to trigger repeated transmission of each data packet to be transmitted when it is determined that the transmission quality of the multimedia data does not meet a set standard based on the feedback information of the receiving end;

[0021] After the repeated transmission is triggered, the following operations are performed for each data packet to be transmitted:

[0022] An acquisition module, used for acquiring at least one target retransmission parameter corresponding to a data packet;

[0023] A processing module, configured to determine corresponding quantization strategies according to the at least one target retransmission parameter, and respectively use the obtained at least one quantization strategy to perform quantization processing on the one data packet to obtain at least one retransmission data packet after quantization processing, wherein the data sizes of the retransmission data packets obtained by using different quantization strategies are different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy;

[0024] The first sending module is used to send the one data packet and the at least one retransmission data packet to the receiving end by using corresponding bandwidth occupancy amounts based on the data size of the at least one retransmission data packet.

[0025] In a possible embodiment, the acquisition module is specifically used to:

[0026] Obtaining a current first network transmission state based on the most recently received feedback information from the receiving end;

[0027] Selecting, from the initial retransmission parameters, a candidate retransmission parameter set corresponding to the first network transmission state according to a mapping relationship between the network transmission state and the initial retransmission parameters;

[0028] Based on the obtained candidate retransmission parameter set, the at least one target retransmission parameter corresponding to the one data packet is obtained.

[0029] In a possible embodiment, the number of candidate retransmission parameters included in the candidate retransmission parameter set is the same as the number of times the data packet is repeatedly transmitted; and the acquisition module is specifically configured to:

[0030] If the feedback information includes at least one reference retransmission parameter detected by the receiving end, smoothing the at least one reference retransmission parameter detected by the receiving end according to each candidate retransmission parameter in the candidate retransmission parameter set to obtain the at least one target retransmission parameter corresponding to the one data packet;

[0031] If the feedback information does not include the at least one reference retransmission parameter detected by the receiving end, each candidate retransmission parameter in the candidate retransmission parameter set is used as the at least one target retransmission parameter corresponding to the one data packet.

[0032] In a possible embodiment, the acquisition module is specifically used to:

[0033] Receiving at least one target retransmission parameter corresponding to the data packet sent by a target device; wherein the target device is the receiving end, or a third-party device other than the sending end and the receiving end;

[0034] The target retransmission parameter used for each repeated transmission of a data packet is determined by the target device according to the most recently received feedback information from the receiving end.

[0035] In a possible embodiment, the processing module is specifically used for:

[0036] For at least one repeated transmission of the one data packet, perform the following operations:

[0037] Determine a target quantization strategy corresponding to a target retransmission parameter of a repeated transmission according to a correspondence between the retransmission parameter and the quantization strategy; wherein different quantization strategies correspond to different quantization accuracies, and the coded bit sizes of the retransmission data packets obtained based on the different quantization accuracies are different;

[0038] According to the target quantization accuracy of the target quantization strategy, the one data packet is encoded to obtain a retransmission data packet corresponding to the one repeated transmission.

[0039] In a fourth aspect, the present application provides a data transmission device, including:

[0040] A determination module, used to determine feedback information indicating the transmission quality of the multimedia data according to the multimedia data transmitted by the sending end;

[0041] A second sending module, configured to send the feedback information to the sending end, so that when the sending end determines based on the feedback information that the transmission quality of the multimedia data does not meet the set standard, the sending end triggers repeated transmission of each data packet to be transmitted;

[0042] A receiving module is used to receive each data packet sent by the sending end and at least one retransmission data packet corresponding to each data packet; wherein, the at least one retransmission data packet corresponding to the one data packet is obtained by the sending end after obtaining at least one target retransmission parameter corresponding to the one data packet, and then using the quantization strategy corresponding to the at least one target retransmission parameter to quantize the one data packet; wherein the data size of the retransmission data packet obtained by using different quantization strategies is different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy.

[0043] In a possible embodiment, the determining module is further used for:

[0044] Obtaining a current first network transmission state according to the latest determined feedback information;

[0045] Selecting, from the initial retransmission parameters, a candidate retransmission parameter set corresponding to the first network transmission state according to a mapping relationship between the network transmission state and the initial retransmission parameters;

[0046] Based on the obtained candidate retransmission parameter set, the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end is obtained, and the at least one target retransmission parameter corresponding to each data packet obtained is sent to the transmitting end.

[0047] In a possible embodiment, the number of candidate retransmission parameters included in the candidate retransmission parameter set is the same as the number of times the data packet is repeatedly transmitted; and the determining module is specifically configured to:

[0048] If the feedback information includes at least one reference retransmission parameter detected by the receiving end, then, according to each candidate retransmission parameter in the candidate retransmission parameter set, respectively smoothing the at least one reference retransmission parameter detected by the receiving end, to obtain the at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end;

[0049] If the feedback information does not include at least one reference retransmission parameter detected by the receiving end, each candidate retransmission parameter in the candidate retransmission parameter set is used as the at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end.

[0050] In a possible embodiment, the determining module is further used for:

[0051] Feedback information including at least one reference retransmission parameter detected by the receiving end is sent to a third-party device, so that the third-party device determines the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end according to the at least one reference retransmission parameter detected by the receiving end in the most recently received feedback information, and sends the at least one target retransmission parameter corresponding to each data packet to be transmitted to the transmitting end.

[0052] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores a program code, and when the program code is executed by the processor, the processor executes the steps of any method described in the first aspect, or the steps of any method described in the second aspect.

[0053] In a sixth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes the steps of any method described in the first aspect or the steps of any method described in the second aspect.

[0054] In the seventh aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium; when the processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of any method described in the first aspect, or the steps of any method described in the second aspect.

[0055] The embodiment of the present application adopts the above technical solution, which has at least the following technical effects:

[0056] In the embodiment of the present application, based on the feedback information of the receiving end, when it is determined that the transmission quality of the multimedia data does not meet the set standard, the transmitting end is triggered to repeat the transmission of each data packet to be transmitted; wherein, for each data packet, at least one target retransmission parameter corresponding to a data packet is obtained, the corresponding quantization strategy is determined respectively, and at least one quantization strategy obtained is used to quantize a data packet respectively, and at least one retransmission data packet after quantization is obtained; that is, the transmitting end determines the corresponding quantization strategy according to the at least one target retransmission parameter obtained, and quantizes the data packet to be transmitted according to the quantization strategy corresponding to each retransmission parameter, and can obtain at least one retransmission data packet corresponding to the data packet after quantization. Since the data size of the retransmission data packet obtained by adopting different quantization strategies is different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy, different quantization strategies can be used to adjust the bandwidth occupancy of the retransmission data packet sent, thereby reducing the occupancy of network transmission resources and avoiding network congestion; for example, the bandwidth occupancy of the retransmission data obtained by quantization according to the quantization strategy corresponding to the retransmission parameter is reduced compared with the bandwidth occupancy of the original data packet, so that while solving the network packet loss problem, it can also effectively solve the network congestion problem caused by excessive transmission data. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0058] Figure 1 A schematic diagram of repeated transmission of multimedia data in the prior art;

[0059] Figure 2 A schematic diagram of an application architecture of a data transmission method according to an embodiment of the present application;

[0060] Figure 3 This is a flow chart of a data transmission method according to an embodiment of the present application;

[0061] Figure 4This is a schematic diagram of transmitting multimedia data to a receiving end according to an embodiment of the present application;

[0062] Figure 5 A schematic diagram of a transmitter triggering repeated transmission according to an embodiment of the present application;

[0063] Figure 6 This is a flow chart of a transmitting end determining a target retransmission parameter according to an embodiment of the present application;

[0064] Figure 7 This is a flow chart of a receiving end determining a target retransmission parameter according to an embodiment of the present application;

[0065] Figure 8 This is a flow chart of a third-party device determining a target retransmission parameter according to an embodiment of the present application;

[0066] Fig. 9 A flowchart of at least one repeated transmission of a data packet according to an embodiment of the present application;

[0067] Fig.10 This is a schematic diagram of an encoder framework according to an embodiment of the present application;

[0068] Fig.11 A schematic diagram of repeated transmission according to an embodiment of the present application;

[0069] Fig.12 This is a schematic diagram of the overall process of multimedia data transmission according to an embodiment of the present application;

[0070] Fig.13 This is a schematic diagram of the structure of a data transmission device in an embodiment of the present application;

[0071] Fig.14 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application;

[0072] Fig.15 This is a schematic diagram of the structure of a data transmission device in an embodiment of the present application;

[0073] Fig.16 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0075] In order to facilitate those skilled in the art to better understand the technical solution of the present application, some of the terms involved in the present application are introduced below:

[0076] Packet loss rate: It indicates the ratio of the number of lost data packets to the number of sent data packets. The packet loss rate is related to the length of the data packet and the frequency of packet sending. Repeated transmission can effectively avoid the problem that the receiving end cannot receive the corresponding data packet due to network packet loss, but repeated transmission requires additional bandwidth consumption.

[0077] Repeated transmission: means that the data packet is copied multiple times and sent to the receiving end at different time intervals.

[0078] In practical applications, when multimedia data is transmitted, packet loss may occur during the transmission process due to the instability of the transmission network, causing the receiving end to be unable to receive data normally. The repeated multiple transmission strategy is usually used to solve the problem of network packet loss. However, the repeated multiple transmission strategy requires additional bandwidth consumption, and the increase in the number of repeated transmissions leads to increased network congestion, which will further trigger more packet loss, causing the receiving end to be unable to receive the transmitted data normally, seriously affecting the data transmission process between the sender and the receiver.

[0079] For ease of understanding, the following briefly introduces the repeated multiple-send strategy solution in the related technology.

[0080] See also Figure 1 As shown, it is a schematic diagram of the repetitive multiple transmission strategy in the related art. The repetitive multiple transmission strategy is to copy a data packet multiple times and send them to the receiving end at different time intervals. Taking the data packet to be transmitted as a data packet obtained after audio data encoding processing as an example, the transmitting end configures the repetitive multiple transmission multiple according to the real-time packet loss status counted by the receiving end, determines the number of multiple transmission data packets to be transmitted to the receiving end according to the repetitive multiple transmission multiple, and transmits the data packet and a certain number of multiple transmission data packets corresponding to the data packet to the receiving end.

[0081] The higher the statistical packet loss rate, the larger the value of the configured multiple transmission multiple is. For example, when the packet loss rate reaches 20% to 30%, the multiple transmission multiple is configured to be 100%; when the packet loss rate reaches 30% to 50%, the multiple transmission multiple is configured to be 200%. In addition, the increase of the multiple transmission multiple can ensure that the receiving end receives multimedia data. Therefore, in theory, increasing the multiple transmission multiple can solve the problem of network packet loss.

[0082] However, repetition requires additional bandwidth consumption. The higher the repetition multiple, the higher the bandwidth usage. In other words, the increase in the repetition multiple will lead to increased network congestion, further triggering more packet loss. In related technologies, the repetition strategy is usually pre-made and has limited control rules. In the ever-changing network state, the repetition strategy cannot be adjusted in time according to the network state. Therefore, the use of the repetition strategy cannot effectively solve the problem of network packet loss.

[0083] Therefore, in order to solve the above problems, a data transmission method is proposed in an embodiment of the present application, which is mainly aimed at how to determine the corresponding quantization strategy according to at least one target retransmission parameter corresponding to the current transmission when repeatedly transmitting a data packet in a network packet loss scenario, and respectively use the at least one obtained quantization strategy to quantize the data packet to obtain at least one retransmission data packet after quantization, wherein the data size of the retransmission data packet obtained by using different quantization strategies is different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy; in this way, the bandwidth occupancy of each retransmission data packet can be effectively controlled; based on the data size of at least one retransmission data packet, the corresponding bandwidth occupancy is respectively used to send the data packet and at least one retransmission data packet to the receiving end; that is, by controlling the bandwidth occupancy of each retransmission data packet, the network congestion problem caused by excessive bandwidth occupancy can be effectively avoided, and the packet loss rate of the network can be effectively reduced.

[0084] The system architecture of the data transmission method of the embodiment of the present application may include at least one transmitting end and at least one receiving end. The transmitting end is used to transmit multimedia data to the receiving end.

[0085] See also Figure 2 As shown, taking the system architecture including multiple terminals 20 and a server 21 as an example, the embodiment of the present application does not limit the number of terminals and servers included in the system architecture. The terminals in the system architecture can communicate with each other or with the server based on the communication network. Each terminal in the system architecture can be used as both a sender and a receiver of multimedia data; and the server in the system architecture can also be used as both a sender and a receiver of multimedia data.

[0086] The terminal 20 may be any smart device such as a smart phone, a tablet computer, a portable personal computer, etc. Various applications may be installed on the terminal 20, and the object sends multimedia data to other terminals 20 or the server 21 through the application client on the terminal 20. Alternatively, the server 21 receives a request sent by the application on the terminal 20, and the server sends multimedia data to the terminal 20.

[0087] The server 21 is used as a sending end and the terminal 20 is used as a receiving end. The server 21 can provide various network services for the terminal 20. For different applications, the server 21 can be considered as a corresponding background server, through which the multimedia data corresponding to the request can be transmitted to the corresponding receiving end according to the request sent by the terminal.

[0088] The server 21 transmits multimedia data to the terminal 20; based on the feedback information of the terminal 20, when it is determined that the transmission quality of the multimedia data does not meet the set standard, the server 21 triggers repeated transmission of each data packet to be transmitted; wherein, the server 21 performs the following operations for each data packet: obtaining at least one target retransmission parameter corresponding to a data packet; determining a corresponding quantization strategy according to at least one target retransmission parameter, and quantizing a data packet using at least one obtained quantization strategy to obtain at least one retransmitted data packet after quantization, and based on the data size of at least one retransmitted data packet, using corresponding bandwidth occupancy to send a data packet and at least one retransmitted data packet to the terminal 20.

[0089] The server 21 may be a single server, a server cluster consisting of several servers, or a cloud computing center.

[0090] The terminal 20 and the server 21 can be connected through a communication network to achieve mutual communication. Optionally, the above-mentioned communication network uses standard communication technology, protocol, or a combination of the two. The communication network is usually the Internet, but it can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or a virtual private network. In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0091] It is worth noting that the application architecture diagram in the embodiment of the present application is to more clearly illustrate the technical solution in the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. For other application architectures and business applications, the technical solution provided in the embodiment of the present application is also applicable to similar problems. In each embodiment of the present application, the data transmission method is applied to Figure 2 The application architecture shown is used as an example for schematic description.

[0092] Based on the above embodiments, see Figure 3 As shown, it is a flow chart of a data transmission method in an embodiment of the present application, which is applied to the sending end. Specifically, the method includes:

[0093] Step S301: The sending end transmits multimedia data to the receiving end.

[0094] Optionally, the embodiments of the present application may include but are not limited to audio data, video data, text data, and picture data.

[0095] For example, Figure 4 As shown, objects can be triggered by Figure 4The control shown in the middle left figure inputs audio and sends the input audio data to the receiving end.

[0096] In the embodiment of the present application, the sending end inputs the multimedia data to be sent into the encoder, obtains the data packet corresponding to the multimedia data, and sends the corresponding data packet to the receiving end.

[0097] The multimedia data to be sent may be multiple frames, and one data packet may include data of some frames among the multiple frames.

[0098] For example, if the multimedia data is audio data, the audio data includes multiple frames, and each frame of data can be input into an audio encoder to obtain a data packet corresponding to each frame, wherein each data packet corresponds to the audio data of a frame in the multiple frames of audio data.

[0099] Step S302: The receiving end determines feedback information for indicating the transmission quality of the multimedia data according to the multimedia data transmitted by the sending end.

[0100] In the embodiment of the present application, the receiving end periodically detects the number of data packets of the received multimedia data, determines feedback information, and sends the periodically detected feedback information to the sending end.

[0101] The feedback information includes the packet loss rate and the maximum number of consecutive packet losses.

[0102] In the embodiment of the present application, the packet loss rate can be obtained by performing packet loss statistics on the data packets at the receiving end within a statistical period. Based on the timestamp information (timestamp) recorded in each data packet, it is determined whether the received adjacent data packets meet the normal spacing between timestamps. If not, it indicates that there is a packet loss phenomenon. The number of lost data packets within a statistical period is counted, and the packet loss rate is determined based on the number of lost data packets.

[0103] Among them, normal adjacent timestamps should increase linearly.

[0104] It should be noted that the time interval for sending each data packet by the sender is pre-set, and the receiving end can know the number of data packets sent by the sender within the statistical period. Therefore, the receiving end can determine the packet loss rate corresponding to a statistical period based on the number of data packets lost within the statistical period and the number of data packets sent by the sender.

[0105] The maximum number of consecutive packet losses may be the maximum number of consecutively lost data packets counted by the receiving end within a statistical period according to the timestamp.

[0106] Step S303: The receiving end sends feedback information to the sending end.

[0107] Step S304: When the transmitting end determines that the transmission quality of the multimedia data does not meet the set standard based on the feedback information from the receiving end, the transmitting end triggers repeated transmission of each data packet to be transmitted.

[0108] For example, Figure 5 As shown in FIG. 1 , a schematic diagram of a transmitter triggering repeated transmission in an embodiment of the present application is shown. Figure 5 As shown, the sending end transmits multimedia data to the receiving end, and receives feedback information determined by the receiving end based on the transmitted multimedia data. When the sending end determines that the set standard is not met based on the received feedback information, the sending end triggers repeated transmission, and the sending end begins to repeatedly transmit each data packet to be transmitted to the receiving end.

[0109] In an optional implementation, when the transmitting end determines that the transmission quality of the multimedia data does not meet the set standard based on the feedback information from the receiving end, the transmitting end can judge that at least one of the packet loss rate and the maximum number of consecutive packet losses included in the feedback information exceeds a threshold, and determine that the transmission quality of the multimedia data does not meet the set standard.

[0110] Exemplarily, taking the packet loss rate as an example, if the threshold corresponding to the packet loss rate is 40%, when the received packet loss rate is 50%, it is determined that the transmission quality of the multimedia data does not meet the set standard.

[0111] Based on the above process, when it is determined that the transmission quality of the multimedia data does not meet the set standard, repeated transmission of each data packet to be transmitted can be triggered.

[0112] Step S305: The sending end determines each data packet to be sent and the corresponding at least one retransmission data packet.

[0113] In the embodiment of the present application, after the transmitting end triggers retransmission, the receiving end periodically detects the received data packets to determine feedback information, and sends the periodically detected feedback information to the transmitting end.

[0114] The feedback information includes reference retransmission parameters, packet loss rate and maximum number of consecutive packet losses.

[0115] The following description is for a data packet to be sent:

[0116] Optionally, in an embodiment of the present application, the transmitting end obtains at least one target retransmission parameter corresponding to a data packet, determines a corresponding quantization strategy according to the target retransmission parameter, performs quantization processing on each data packet using the determined quantization strategy, and obtains at least one retransmission data packet after the quantization processing of each data packet;

[0117] In implementation, the sending end may obtain at least one target retransmission parameter corresponding to a data packet in a variety of different ways.

[0118] Method 1: The sending end determines at least one target retransmission parameter corresponding to a data packet;

[0119] Optional, see Figure 6 , a flowchart of a transmitting end determining a target retransmission parameter in an embodiment of the present application, the specific steps are as follows:

[0120] Step S601: The transmitting end obtains the current first network transmission state based on the feedback information of the receiving end received most recently;

[0121] Optionally, an embodiment of the present application determines the current first network transmission state based on the packet loss rate and the maximum number of consecutive packet losses in the latest received feedback information.

[0122] Among them, for the process of determining the current first network transmission state according to the packet loss rate and the maximum number of consecutive packet losses in the latest received feedback information, the embodiments of the present application have the following implementation methods:

[0123] Implementation method 1 determines the current first network transmission state according to the latest received feedback information and the mapping relationship between the feedback information and the network transmission state.

[0124] In the embodiment of the present application, when determining the network transmission status according to the packet loss rate and the maximum number of consecutive packet losses, the packet loss rate can be divided into different ranges.

[0125] Exemplarily, the range of packet loss rate classification may be:

[0126] lossrate<3%; 3%≤lossrate<5%; 5%≤lossrate<10%; 10%≤lossrate<15%;

[0127] 15%≤lossrate<20%; 20%≤lossrate<25%; 25%≤lossrate<30%; 30%≤lossrate<50%; 50%≤lossrate. Lossrate indicates the packet loss rate.

[0128] Furthermore, when determining the network transmission status according to the packet loss rate and the maximum number of consecutive packet losses, the embodiment of the present application may also divide the maximum number of consecutive packet losses into different ranges.

[0129] Exemplarily, the range of the maximum number of consecutive packet losses may be:

[0130] contlost<2; 2≤contlost<3; 3≤contlost<4; 4≤contlost<5; 5≤contlost<6;

[0131] 6≤contlost<8; 8≤contlost<10; 10≤contlost<15; 15≤contlost<20; 20≤contlost. contlost indicates the maximum number of consecutive packet losses.

[0132] Based on the above-mentioned range of packet loss rate, 9 ranges corresponding to the packet loss rate can be determined, and by dividing the range of the maximum continuous packet loss number, 10 ranges corresponding to the maximum continuous packet loss number can be determined. Based on the 9 ranges corresponding to the determined packet loss rate and the 10 ranges corresponding to the determined maximum continuous packet loss number, the embodiment of the present application can determine 90 corresponding network transmission states.

[0133] Exemplarily, Table 1 illustrates the mapping relationship between feedback information and network transmission status.

[0134] In Table 1, the embodiment of the present application takes the case of contlost<2, 2≤contlost<3 included in the maximum number of consecutive packet losses as an example to illustrate the mapping relationship between feedback information and network transmission status.

[0135] It should be noted that the selection of specific values ​​in Table 1 is only for example and does not limit the present application.

[0136]

[0137]

[0138] Table 1

[0139] It should be noted that lossrate in Table 1 of the embodiment of the present application represents the determined packet loss rate, and contlost represents the determined maximum number of consecutive packet losses.

[0140] If the most recently received packet loss rate is less than 3% and the maximum number of consecutive packet losses is less than 2, the current network transmission state is determined to be 0.

[0141] Implementation method 2, based on the latest received feedback information, determines the packet loss rate level corresponding to the packet loss rate in the feedback information, and determines the maximum continuous packet loss number level corresponding to the maximum continuous packet loss number in the feedback information, and determines the current first network transmission state based on the packet loss rate level and the maximum continuous packet loss number level.

[0142] During implementation, the embodiment of the present application can determine the corresponding packet loss rate level based on the packet loss rate in the latest received feedback information and the corresponding relationship between the preset packet loss rate and the packet loss rate level; and can determine the corresponding maximum continuous packet loss level based on the maximum continuous packet loss number in the latest received feedback information and the corresponding relationship between the preset maximum continuous packet loss number and the maximum continuous packet loss level.

[0143] Among them, in the embodiment of the present application, packet loss rate levels corresponding to different packet loss rates can be preset.

[0144] Optionally, the correspondence between the preset packet loss rate and the packet loss rate level in the embodiment of the present application, and the correspondence between the maximum continuous packet loss number and the maximum continuous packet loss number level can be displayed in a table.

[0145] Exemplarily, the corresponding relationship between different preset packet loss rates and packet loss rate levels is shown in Table 2:

[0146]

[0147]

[0148] Table 2

[0149] It should be noted that the lossrate in Table 2 of the embodiment of the present application represents the determined packet loss rate, and State_lossrate represents the packet loss rate level.

[0150] Exemplarily, if the packet loss rate is less than 3%, it is determined that the packet loss rate level corresponding to the packet loss rate is 0.

[0151] Optionally, in the embodiment of the present application, different maximum consecutive packet loss numbers corresponding to different maximum consecutive packet loss numbers may be preset.

[0152] Exemplarily, the corresponding relationship between different preset maximum continuous packet loss numbers and maximum continuous packet loss levels is shown in Table 3:

[0153]

[0154]

[0155] Table 3

[0156] It should be noted that contlost in Table 3 of the embodiment of the present application represents the determined maximum number of consecutive packet losses, and State_contlost represents the level of the maximum number of consecutive packet losses.

[0157] Exemplarily, if the maximum number of consecutive packet losses is less than 2, the maximum number of consecutive packet losses level corresponding to the maximum number of consecutive packet losses is determined to be 0.

[0158] Based on Table 3 of the above example, the maximum continuous packet loss number level corresponding to the maximum continuous packet loss number can be determined according to the maximum continuous packet loss number determined in a statistical period.

[0159] Based on Table 2 and Table 3, the embodiment of the present application can determine that the current packet loss rate level includes 9 levels and the maximum continuous packet loss number level includes 10 levels.

[0160] Optionally, different network transmission states may be preset based on different packet loss rate levels and different maximum continuous packet loss number levels.

[0161] Among them, based on 9 levels of packet loss rate and 10 levels of maximum continuous packet loss, 90 network transmission states can be preset.

[0162] Exemplarily, some preset network transmission states are shown in Table 4. It should be noted that Table 4 is only an example and does not limit the present application.

[0163]

[0164]

[0165] Table 4

[0166] Optionally, based on the determined packet loss rate level and the maximum continuous packet loss number level, in the embodiment of the present application, the corresponding network transmission state can be determined according to the packet loss rate level and the maximum continuous packet loss number level. Among them, different packet loss rates and different maximum continuous packet loss number levels are combined to determine the corresponding network transmission state.

[0167] Exemplarily, when the packet loss rate level is 0 and the maximum continuous packet loss level is 0, the corresponding network transmission state 0 is the case where the packet loss rate level is 0 and the maximum continuous packet loss level is 0, then the current first network transmission state is the case where the packet loss rate level is 0 and the maximum continuous packet loss level is 0.

[0168] Step S602: The transmitting end selects a candidate retransmission parameter set corresponding to the first network transmission state from the initial retransmission parameters according to the mapping relationship between the network transmission state and the initial retransmission parameters;

[0169] It should be noted that the number of candidate retransmission parameters included in the candidate retransmission parameter set is the same as the number of times the data packet is repeatedly transmitted.

[0170] Exemplarily, the embodiment of the present application uses Table 5 to represent the mapping relationship between the network transmission status and each initial retransmission parameter, taking the number of initial retransmission parameters as 3 as an example.

[0171]

[0172]

[0173] Table 5

[0174] Exemplarily, if the current first network transmission state is 0, according to the mapping relationship between the network transmission state and each initial retransmission parameter, the corresponding candidate retransmission parameter set is determined to be a1, a2, and a3.

[0175] In the embodiment of the present application, the candidate retransmission parameters in the candidate retransmission parameter set include candidate retransmission parameter sets corresponding to different sequence numbers. For example, the sequence numbers of the candidate retransmission parameters a1, a2, and a3 in the candidate retransmission parameter set are 1, 2, and 3.

[0176] It should be noted that the initial retransmission parameter may be a specific value, for example, a1 may be 70%.

[0177] Optionally, an embodiment of the present application may provide several retransmission strategies. Based on the mapping relationship between the preset retransmission strategy and each initial retransmission parameter, after determining the preset network transmission state, the corresponding candidate retransmission parameter set may be determined according to the determined retransmission strategy.

[0178] In the embodiment of the present application, the number of retransmitted data packets and the time interval between each retransmitted data packet when sending can be adjusted by adjusting different retransmission strategies.

[0179] It should be noted that only one retransmission strategy can be used in each transmission.

[0180] For example, if the embodiment of the present application provides five retransmission strategies, each network transmission state corresponds to at least one initial retransmission parameter for each of the five retransmission strategies. After determining the network transmission state, at least one corresponding candidate retransmission parameter is determined by determining the retransmission strategy to be used.

[0181] For example, if the determined network transmission state is 0, and the determined retransmission strategy used is to repeatedly send 3 retransmission data packets, and each retransmission data packet is sent at an interval of 20ms, then the sender will send the retransmission data packets determined according to a1, a2, and a3 every 20ms.

[0182] Optionally, based on the process of determining at least one corresponding candidate retransmission parameter according to the network transmission state and the retransmission strategy, the embodiments of the present application provide the following methods:

[0183] Mode 1: Determine at least one corresponding candidate retransmission parameter based on a preset retransmission strategy.

[0184] In the embodiment of the present application, after determining the network transmission status, at least one corresponding candidate retransmission parameter can be determined based on a pre-set retransmission strategy.

[0185] Mode 2: Based on a preset retransmission strategy rule, it is determined that a retransmission strategy needs to be used, and at least one corresponding candidate retransmission parameter is determined based on the determined retransmission strategy and the determined network transmission status.

[0186] In the embodiment of the present application, it can be determined whether a retransmission strategy needs to be used based on the current network status and pre-set retransmission strategy rules.

[0187] Exemplarily, when the network state is judged to be at a peak period, the embodiment of the present application determines the corresponding retransmission strategy to be used according to the preset retransmission strategy rule. The present application does not limit this.

[0188] Step S603: The transmitting end obtains at least one target retransmission parameter corresponding to a data packet based on the obtained candidate retransmission parameter set.

[0189] Optionally, in the embodiment of the present application, the process of obtaining at least one target retransmission parameter corresponding to a data packet can be divided into the following cases:

[0190] Case 1: if the feedback information includes at least one reference retransmission parameter detected by the receiving end, then according to each candidate retransmission parameter in the candidate retransmission parameter set, at least one reference retransmission parameter detected by the receiving end is smoothed to obtain at least one target retransmission parameter corresponding to a data packet;

[0191] It should be noted that the candidate retransmission parameters and corresponding feedback information in the embodiment of the present application are determined by the receiving end within the same statistical period.

[0192] Based on the determined candidate retransmission parameters, the embodiment of the present application can determine at least one target retransmission parameter by smoothing at least one reference retransmission parameter detected by the receiving end and each candidate retransmission parameter.

[0193] Optionally, in an embodiment of the present application, the reference retransmission parameters detected by the receiving end include at least one reference retransmission parameter, and the candidate retransmission parameter set includes at least one candidate retransmission parameter. The at least one reference retransmission parameter detected by the receiving end is a reference retransmission parameter corresponding to a different sequence number, and the candidate retransmission parameter set also includes candidate retransmission parameters corresponding to the same sequence number type.

[0194] Optionally, an embodiment of the present application performs smoothing processing on a candidate retransmission parameter corresponding to the sequence number in at least one candidate retransmission parameter set and a reference retransmission parameter corresponding to the sequence number based on the sequence number to determine a target retransmission parameter corresponding to the sequence number.

[0195] Optionally, for one of the at least one sequence numbers, in the embodiment of the present application, the candidate retransmission parameter of the sequence number received and the reference retransmission parameter of the sequence number can be smoothed according to formula 1. Success_rate(i,k)=a*Success_rate(i-1,k)+(1-a)*Success_rate_cur(k)

[0196] —Formula 1

[0197] Among them, Success_rate(i,k) represents the target retransmission parameter; Success_rate(i-1,k) represents the candidate retransmission parameter; Success_rate_cur(k) represents the reference retransmission parameter detected by the receiving end, and a represents the weight.

[0198] Based on formula 1, after detecting at least one reference retransmission parameter, the target retransmission parameter corresponding to any sequence number can be determined based on the reference retransmission parameter corresponding to the sequence number and the candidate retransmission parameter corresponding to the sequence number.

[0199] For reference retransmission parameters corresponding to other sequence numbers, refer to Formula 1 for smoothing processing to determine the target retransmission parameters corresponding to the sequence numbers.

[0200] Optionally, the embodiment of the present application may use the determined target retransmission parameter as a candidate retransmission parameter to update the stored candidate retransmission parameter.

[0201] According to the sequence number corresponding to each target retransmission parameter in at least one target retransmission parameter, the candidate retransmission parameter corresponding to the sequence number is replaced to obtain an updated candidate retransmission parameter.

[0202] Exemplarily, the reference retransmission parameter is used as an example of the reception success rate of the receiving end detecting the receiving retransmission data packet. For the candidate retransmission parameter corresponding to a sequence number in each candidate retransmission parameter set, an example is given. If the candidate retransmission parameter corresponding to the sequence number is 70%, and the reference retransmission parameter corresponding to the sequence number detected is 80%, if the target retransmission parameter obtained after smoothing the candidate retransmission parameter and the reference retransmission parameter is 75%, then the mapping relationship between the network transmission state and the initial retransmission parameter corresponding to the sequence number will be modified, and the network transmission state will be modified to a mapping relationship with 75% of the initial retransmission parameter. Then, when the candidate retransmission parameter corresponding to the sequence number is determined according to the network transmission state next time, the determined candidate retransmission parameter is 75%.

[0203] Case 2: if the feedback information does not include at least one reference retransmission parameter detected by the receiving end, each candidate retransmission parameter in the candidate retransmission parameter set is used as at least one target retransmission parameter corresponding to a data packet.

[0204] It should be noted that after the sending end triggers retransmission, the receiving end has not yet determined the feedback information of the first periodic detection after the retransmission is triggered. The reference retransmission parameter in the feedback information is periodically detected by the receiving end after the sending end triggers retransmission. Therefore, at this time, the feedback information of the sending end does not include at least one reference retransmission parameter detected by the receiving end.

[0205] Mode 2: The sending end receives at least one target retransmission parameter corresponding to a data packet sent by the receiving end.

[0206] In this manner, the receiving end determines at least one target retransmission parameter corresponding to each data packet, and sends the determined at least one target retransmission parameter corresponding to each data packet to the sending end.

[0207] Optional, see Figure 7 , a flowchart of a receiving end determining a target retransmission parameter in an embodiment of the present application, the specific steps are as follows:

[0208] Step S701: The receiving end obtains the current first network transmission state according to the latest determined feedback information;

[0209] Step S702: The receiving end selects a candidate retransmission parameter set corresponding to the first network transmission state from the initial retransmission parameters according to the mapping relationship between the network transmission state and the initial retransmission parameters;

[0210] The number of candidate retransmission parameters included in the candidate retransmission parameter set is the same as the number of times the data packet is repeatedly transmitted.

[0211] Step S703: The receiving end obtains at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end based on the obtained candidate retransmission parameter set.

[0212] It should be noted that in the embodiment of the present application, the process in step S701-step S703 can refer to the process in step S601-step S603, and the present application will not repeat them here.

[0213] Step S704: The receiving end sends at least one target retransmission parameter corresponding to each obtained data packet to the sending end.

[0214] Optionally, the target retransmission parameter is determined in the following ways in the embodiment of the present application:

[0215] Mode 1: If the feedback information includes at least one reference retransmission parameter detected by the receiving end, then according to each candidate retransmission parameter in the candidate retransmission parameter set, the at least one reference retransmission parameter detected by the receiving end is smoothed to obtain the at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end;

[0216] It should be noted that the process of the receiving end obtaining the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end according to the at least one reference retransmission parameter detected by the receiving end included in the feedback information can refer to the process of the transmitting end obtaining the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end according to the at least one reference retransmission parameter detected by the receiving end included in the feedback information. This application will not be repeated here.

[0217] Mode 2: If the feedback information does not include at least one reference retransmission parameter detected by the receiving end, each candidate retransmission parameter in the candidate retransmission parameter set is used as at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end.

[0218] It should be noted that after the transmitting end triggers retransmission, the receiving end has not yet determined the feedback information of the first periodic detection after the retransmission is triggered. The reference retransmission parameter in the feedback information is periodically detected by the receiving end after the transmitting end triggers retransmission. Therefore, at this time, the feedback information of the receiving end does not include at least one reference retransmission parameter detected by the receiving end.

[0219] Mode 3: The sending end receives at least one target retransmission parameter corresponding to a data packet sent by a third-party device.

[0220] In this manner, the third-party device determines at least one target retransmission parameter corresponding to each data packet, and sends the determined at least one target retransmission parameter corresponding to each data packet to the transmitting end.

[0221] Optional, see Figure 8 , a flowchart of a third-party device determining a target retransmission parameter in an embodiment of the present application, the specific steps are as follows:

[0222] Step S801: The receiving end sends feedback information including at least one reference retransmission parameter detected by the receiving end to a third-party device.

[0223] Step S802: The third-party device determines at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end according to at least one reference retransmission parameter detected by the receiving end in the latest received feedback information.

[0224] It should be noted that in the embodiment of the present application, the process of the third-party device determining at least one target retransmission parameter corresponding to each data packet to be transmitted can refer to the process of the sending end obtaining at least one target retransmission parameter corresponding to each data packet to be transmitted, and this application will not repeat it here.

[0225] Step S803: The third-party device sends at least one target retransmission parameter corresponding to each data packet to be transmitted to the transmitting end.

[0226] After determining at least one target retransmission parameter corresponding to a data packet, the transmitting end of the embodiment of the present application can quantize the data packet according to the target retransmission parameter to obtain at least one retransmission data packet after quantization.

[0227] The following is a detailed description of the process of performing quantization processing on a data packet to obtain at least one retransmitted data packet after quantization processing:

[0228] In the embodiment of the present application, a corresponding quantization strategy is determined according to at least one target retransmission parameter, and at least one obtained quantization strategy is used to quantize a data packet to obtain at least one retransmitted data packet after quantization.

[0229] The data sizes of the retransmitted data packets obtained by using different quantization strategies are different, and the data size of the retransmitted data packets is positively correlated with the bandwidth occupancy.

[0230] Optionally, in the embodiment of the present application, for at least one repeated transmission of a data packet, the following is provided: Fig. 9 A flowchart of at least one repeated transmission of a data packet is shown, and the specific steps are as follows:

[0231] Step S901: Determine a target quantization strategy corresponding to a target retransmission parameter for a repeated transmission according to a correspondence between the retransmission parameter and the quantization strategy;

[0232] Among them, different quantization strategies correspond to different quantization accuracies, and the encoding bit sizes of the retransmitted data packets obtained based on the different quantization accuracies are different;

[0233] Step S902: Encode a data packet according to the target quantization accuracy of the target quantization strategy to obtain a retransmission data packet corresponding to a repeated transmission.

[0234] It should be noted that, in the embodiment of the present application, a correspondence between a retransmission parameter and a quantization strategy may be preset, and a corresponding target quantization strategy may be determined according to the target retransmission parameter.

[0235] Optionally, the correspondence between the preset retransmission parameters and the quantization strategy in the embodiment of the present application is shown in Table 6:

[0236]

[0237]

[0238] Table 6

[0239] Among them, in the embodiment of the present application, Success_rate(i,k) represents the determined target retransmission parameter.

[0240] Exemplarily, for a retransmitted data packet whose quantization strategy is "no quantization processing", the retransmitted data packet is sent to the receiving end using a corresponding bandwidth occupancy based on the size of the original data packet.

[0241] In the embodiment of the present application, the data packets are quantized and encoded based on different quantization strategies in the encoder, and stored in the sending buffer.

[0242] Optionally, taking the multimedia data as audio data as an example, the embodiment of the present application may process the data packet in the following ways:

[0243] Method 1: A traditional audio encoder based on an acoustic excitation model processes data packets.

[0244] Among them, the traditional encoder can be an opus encoder.

[0245] In the embodiment of the present application, the encoder can control the quantization accuracy by adjusting the prediction gain of the prediction filter. When the prediction gain value decreases, the entropy value of the quantization index is increased, thereby increasing the number of bits for encoding, that is, the quantization accuracy is improved. On the contrary, if the prediction gain of the prediction filter increases, the entropy value of the quantization index is reduced, thereby reducing the number of bits for encoding, that is, the quantization accuracy is reduced.

[0246] Method 2: Processing data packets based on deep learning audio encoder.

[0247] Among them, the mainstream of deep learning audio encoder is based on encoder-rvq-decoder framework, where rvq (residual vector quantizers) represents the quantization precision control unit. Fig.10 As shown, Fig.10 The left figure in the figure shows the training process of the deep learning audio encoder, in which the audio data is input into the encoder-rvq-decoder framework, and the audio data is converted into a coded signal through the encoder in the framework. After the coded signal is denoised, the quantization precision control unit is used to compress the coded signal, and finally the decoder is used to convert it into audio. Among them, the converted audio and the original audio are input into the discriminator, and the discriminator makes the converted audio as close to the original audio as possible.

[0248] like Fig.10 As shown in the right figure in the inference deployment of the deep learning audio encoder, the encoder and quantization precision control unit are at the sending end, sending the compressed bit stream to the receiving end, and the decoder is applied at the receiving end to decode the audio signal to form high-quality audio.

[0249] like Fig.10As shown, the audio data is input into the encoder-rvq-decoder framework to obtain the corresponding retransmitted data packets after quantization processing.

[0250] Step S306: The transmitting end uses corresponding bandwidth occupancy amounts based on the data size of at least one retransmitted data packet to send a data packet and at least one retransmitted data packet to the receiving end.

[0251] For example, Fig.11 When the transmitting end repeatedly transmits each data packet to be transmitted, the embodiment of the present application sends each data packet and at least one retransmission data packet corresponding to each data packet to the receiving end based on the bandwidth occupancy.

[0252] like Fig.12 As shown, a schematic diagram of the overall process of multimedia data transmission in an embodiment of the present application. Among them, when triggering repeated transmission of each data packet to be transmitted, the data packet to be transmitted is processed according to different quantization strategies to obtain the corresponding retransmission data packet obtained by different quantization processing, and the processed retransmission data packet is stored in the sending buffer area, and multiple transmission operations are performed to the receiving end according to the retransmission data packet. After the receiving end receives the data packet and the retransmission data packet, the corresponding multimedia data is obtained by decoding the received data packet and the retransmission data packet respectively. In addition, when the receiving end receives the data packet, the packet loss rate detection and the maximum continuous packet loss number detection are performed on the received data packet within the statistical period, and when the receiving end receives the retransmission data packet, the receiving end performs reception success rate statistics on the retransmission data packets of each sequence number in the received retransmission data packet within the statistical period. In the multi-transmission packet quantization control module, according to the determined packet loss rate, the maximum continuous packet loss number and the reception success rate, the retransmission data packets processed by different quantization strategies stored in the sending buffer area of ​​the sending end are selected, and the retransmission data packet processed by the quantization strategy that needs to be sent is selected, so that the sending end sends the selected retransmission data packet to the receiving end.

[0253] For example, take the retransmission strategy of 400% multiple transmission as an example, that is, four corresponding retransmission data packets are sent based on one data packet. If the statistical period is two seconds, then within two seconds after the repeated transmission is triggered, the receiving end will count the reception success rates of the retransmission data packets with sequence numbers 1 to 4 respectively, and determine the reception success rates corresponding to different sequence numbers. For example, if the number of retransmission data packets with sequence numbers 1 to 4 received by the receiving end within two seconds is 10, 70, 85, and 5 respectively, and the number of retransmission data packets with sequence numbers 1 to 4 sent by the sending end within these two seconds is 100, 100, 100, and 20 respectively, then the reception success rates of the retransmission data packets with sequence numbers 1 to 4 are 10 / 100=10%, 70 / 100=70%, 85 / 100=85%, and 5 / 20=25% respectively.

[0254] It should be noted that in the embodiment of the present application, the work of selecting the retransmission data packets to be sent in the multi-packet quantization control module can be performed by a third-party device, or the work of selecting the retransmission data packets to be sent in the multi-packet quantization control module can be performed by the transmitting end. The present application does not limit this.

[0255] Step S307: The receiving end receives each data packet sent by the sending end and at least one retransmission data packet corresponding to each data packet.

[0256] Based on the above process, the data transmission method of the embodiment of the present application can be applied to the sound transmission scenarios in voip (voice transmission based on IP) calls, the transmission scenarios of pictures and sounds during games, the transmission scenarios of pictures and sounds during live broadcasts, and the transmission scenarios of sounds during audio broadcasts.

[0257] In the embodiment of the present application, with respect to the transmission of images and sounds, the sounds and images may be transmitted separately.

[0258] For the convenience of description, the above parts are divided into modules according to their functions and described separately. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software or hardware.

[0259] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0260] Regarding the device in the above embodiment, the specific execution mode of each module therein has been described in detail in the embodiment of the method, and will not be elaborated here.

[0261] Based on the same inventive concept, an embodiment of the present application provides a data transmission device, the principle of solving the problem of the device is similar to the method of the above embodiment, so the implementation of the device can refer to the implementation of the above method, and the repeated parts will not be repeated.

[0262] like Fig.13 As shown, a data transmission device 1300 provided in an embodiment of the present application includes a transmission module 1301, a trigger module 1302, an acquisition module 1303, a processing module 1304, and a first sending module 1305.

[0263] Transmission module 1301, used for transmitting multimedia data to a receiving end;

[0264] A trigger module 1302, configured to trigger repeated transmission of each data packet to be transmitted when it is determined that the transmission quality of the multimedia data does not meet a set standard based on the feedback information of the receiving end;

[0265] After the repeated transmission is triggered, the following operations are performed for each data packet to be transmitted:

[0266] An acquisition module 1303, configured to acquire at least one target retransmission parameter corresponding to a data packet;

[0267] The processing module 1304 is used to determine the corresponding quantization strategy according to the at least one target retransmission parameter, and respectively use the obtained at least one quantization strategy to perform quantization processing on the one data packet to obtain at least one retransmission data packet after quantization processing, wherein the data size of the retransmission data packet obtained by using different quantization strategies is different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy;

[0268] The first sending module 1305 is configured to send the one data packet and the at least one retransmission data packet to the receiving end by using corresponding bandwidth occupancy amounts based on the data size of the at least one retransmission data packet.

[0269] In a possible embodiment, the acquisition module 1303 is specifically used for:

[0270] Obtaining a current first network transmission state based on the most recently received feedback information from the receiving end;

[0271] Selecting, from the initial retransmission parameters, a candidate retransmission parameter set corresponding to the first network transmission state according to a mapping relationship between the network transmission state and the initial retransmission parameters;

[0272] Based on the obtained candidate retransmission parameter set, the at least one target retransmission parameter corresponding to the one data packet is obtained.

[0273] In a possible embodiment, the number of candidate retransmission parameters included in the candidate retransmission parameter set is the same as the number of times the data packet is repeatedly transmitted; and the acquisition module 1303 is specifically configured to:

[0274] If the feedback information includes at least one reference retransmission parameter detected by the receiving end, smoothing the at least one reference retransmission parameter detected by the receiving end according to each candidate retransmission parameter in the candidate retransmission parameter set to obtain the at least one target retransmission parameter corresponding to the one data packet;

[0275] If the feedback information does not include the at least one reference retransmission parameter detected by the receiving end, each candidate retransmission parameter in the candidate retransmission parameter set is used as the at least one target retransmission parameter corresponding to the one data packet.

[0276] In a possible embodiment, the acquisition module 1303 is specifically used for:

[0277] Receiving at least one target retransmission parameter corresponding to the data packet sent by a target device; wherein the target device is the receiving end, or a third-party device other than the sending end and the receiving end;

[0278] The target retransmission parameter used for each repeated transmission of a data packet is determined by the target device according to the most recently received feedback information from the receiving end.

[0279] In a possible embodiment, the processing module 1304 is specifically used for:

[0280] For at least one repeated transmission of the one data packet, perform the following operations:

[0281] Determine a target quantization strategy corresponding to a target retransmission parameter of a repeated transmission according to a correspondence between the retransmission parameter and the quantization strategy; wherein different quantization strategies correspond to different quantization accuracies, and the coded bit sizes of the retransmission data packets obtained based on the different quantization accuracies are different;

[0282] According to the target quantization accuracy of the target quantization strategy, the one data packet is encoded to obtain a retransmission data packet corresponding to the one repeated transmission.

[0283] Based on the same inventive concept as the above-mentioned method embodiment, an electronic device is also provided in the embodiment of the present application. The principle of solving the problem by the electronic device is similar to the method of the above-mentioned embodiment. Therefore, the implementation of the electronic device can refer to the implementation of the above-mentioned method, and the repeated parts will not be repeated.

[0284] See also Fig.14 As shown, the electronic device 140 may include at least a processor 141 and a memory 142. The memory 142 stores a program code, and when the program code is executed by the processor 141, the processor 141 executes the steps executed by the client in the data transmission method in the above embodiment of the present application.

[0285] like Fig.15 As shown, a data transmission device 1500 provided in an embodiment of the present application includes a determination module 1501 , a second sending module 1502 , and a receiving module 1503 .

[0286] A determination module 1501, configured to determine feedback information indicating transmission quality of the multimedia data according to the multimedia data transmitted by the transmitting end;

[0287] The second sending module 1502 is used to send the feedback information to the sending end, so that when the sending end determines that the transmission quality of the multimedia data does not meet the set standard based on the feedback information, it triggers repeated transmission of each data packet to be transmitted;

[0288] The receiving module 1503 is used to receive each data packet sent by the sending end and at least one retransmission data packet corresponding to each data packet; wherein, the at least one retransmission data packet corresponding to the one data packet is obtained by the sending end obtaining at least one target retransmission parameter corresponding to the one data packet and then using the quantization strategy corresponding to the at least one target retransmission parameter to quantize the one data packet; wherein the data size of the retransmission data packet obtained by using different quantization strategies is different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy.

[0289] In a possible embodiment, the determining module 1501 is further configured to:

[0290] Obtaining a current first network transmission state according to the latest determined feedback information;

[0291] Selecting, from the initial retransmission parameters, a candidate retransmission parameter set corresponding to the first network transmission state according to a mapping relationship between the network transmission state and the initial retransmission parameters;

[0292] Based on the obtained candidate retransmission parameter set, the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end is obtained, and the at least one target retransmission parameter corresponding to each data packet obtained is sent to the transmitting end.

[0293] In a possible embodiment, the number of candidate retransmission parameters included in the candidate retransmission parameter set is the same as the number of times the data packet is repeatedly transmitted; and the determining module 1501 is specifically configured to:

[0294] If the feedback information includes at least one reference retransmission parameter detected by the receiving end, then, according to each candidate retransmission parameter in the candidate retransmission parameter set, respectively smoothing the at least one reference retransmission parameter detected by the receiving end, to obtain the at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end;

[0295] If the feedback information does not include at least one reference retransmission parameter detected by the receiving end, each candidate retransmission parameter in the candidate retransmission parameter set is used as the at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end.

[0296] In a possible embodiment, the determining module 1501 is further configured to:

[0297] Feedback information including at least one reference retransmission parameter detected by the receiving end is sent to a third-party device, so that the third-party device determines the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end according to the at least one reference retransmission parameter detected by the receiving end in the most recently received feedback information, and sends the at least one target retransmission parameter corresponding to each data packet to be transmitted to the transmitting end.

[0298] Refer to the following Fig.16 The electronic device 160 according to this embodiment of the present application is described. Fig.16 The electronic device 160 is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0299] like Fig.16 The electronic device 160 is in the form of a general electronic device. The components of the electronic device 160 may include but are not limited to: at least one processing unit 161, at least one storage unit 162, and a bus 163 connecting different system components (including the storage unit 162 and the processing unit 161).

[0300] Bus 163 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0301] The storage unit 162 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1621 and / or a cache memory 1622 , and may further include a read-only memory (ROM) 1623 .

[0302] The storage unit 162 may also include a program / utility 1616 having a set (at least one) of program modules 1624, such program modules 1624 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0303] The electronic device 160 may also communicate with one or more external devices 164 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable objects to interact with the electronic device 160, and / or communicate with any device that enables the electronic device 160 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 165. Furthermore, the electronic device 160 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 166. As shown, the network adapter 166 communicates with other modules for the electronic device 160 via a bus 163. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 160, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0304] Based on the same inventive concept as the above-mentioned method embodiments, various aspects of the interface presentation method provided in the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps executed by the server in the interface presentation method according to various exemplary embodiments of the present application described above in this specification.

[0305] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0306] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0307] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data transmission method, characterized in that: Applied to the sending end, the method includes: Transmit multimedia data to a receiving end; Based on the feedback information of the receiving end, when it is determined that the transmission quality of the multimedia data does not meet the set standard, triggering repeated transmission of each data packet to be transmitted; After the repeated transmission is triggered, the following operations are performed for each data packet to be transmitted: Obtaining at least one target retransmission parameter corresponding to a data packet; Determine the corresponding quantization strategies according to the at least one target retransmission parameter, and respectively use the obtained at least one quantization strategy to quantize the one data packet to obtain at least one retransmission data packet after quantization, wherein the data sizes of the retransmission data packets obtained by using different quantization strategies are different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy; Based on the data size of the at least one retransmission data packet, the one data packet and the at least one retransmission data packet are sent to the receiving end using corresponding bandwidth occupancy amounts.

2. The method according to claim 1, characterized in that The obtaining of at least one target retransmission parameter corresponding to a data packet includes: Obtaining a current first network transmission state based on the most recently received feedback information from the receiving end; Selecting, from the initial retransmission parameters, a candidate retransmission parameter set corresponding to the first network transmission state according to a mapping relationship between the network transmission state and the initial retransmission parameters; Based on the obtained candidate retransmission parameter set, the at least one target retransmission parameter corresponding to the one data packet is obtained.

3. The method according to claim 2, characterized in that The number of candidate retransmission parameters included in the candidate retransmission parameter set is the same as the number of times the data packet is repeatedly transmitted; The obtaining, based on the obtained candidate retransmission parameter set, the at least one target retransmission parameter corresponding to the one data packet comprises: If the feedback information includes at least one reference retransmission parameter detected by the receiving end, smoothing the at least one reference retransmission parameter detected by the receiving end according to each candidate retransmission parameter in the candidate retransmission parameter set to obtain the at least one target retransmission parameter corresponding to the one data packet; If the feedback information does not include the at least one reference retransmission parameter detected by the receiving end, each candidate retransmission parameter in the candidate retransmission parameter set is used as the at least one target retransmission parameter corresponding to the one data packet.

4. The method according to claim 1, characterized in that The obtaining of at least one target retransmission parameter corresponding to a data packet includes: Receiving at least one target retransmission parameter corresponding to the data packet sent by a target device; wherein the target device is the receiving end, or a third-party device other than the sending end and the receiving end; The target retransmission parameter used for each repeated transmission of a data packet is determined by the target device according to the most recently received feedback information from the receiving end.

5. The method according to any one of claims 1 to 4, characterized in that: The step of respectively determining corresponding quantization strategies according to the at least one target retransmission parameter, and respectively using the obtained at least one quantization strategy to quantize the one data packet to obtain at least one retransmission data packet after quantization, comprises: For at least one repeated transmission of the one data packet, perform the following operations: Determine a target quantization strategy corresponding to a target retransmission parameter of a repeated transmission according to a correspondence between the retransmission parameter and the quantization strategy; wherein different quantization strategies correspond to different quantization accuracies, and the coded bit sizes of the retransmission data packets obtained based on the different quantization accuracies are different; According to the target quantization accuracy of the target quantization strategy, the one data packet is encoded to obtain a retransmission data packet corresponding to the one repeated transmission.

6. A data transmission method, characterized in that: Applied to the receiving end, the method includes: Determining, according to the multimedia data transmitted by the sending end, feedback information used to indicate the transmission quality of the multimedia data; Sending the feedback information to the sending end, so that when the sending end determines based on the feedback information that the transmission quality of the multimedia data does not meet the set standard, triggering repeated transmission of each data packet to be transmitted; Receive each data packet sent by the sending end and at least one retransmission data packet corresponding to each data packet; wherein, the at least one retransmission data packet corresponding to the one data packet is obtained by the sending end after obtaining at least one target retransmission parameter corresponding to the one data packet, and then using the quantization strategy corresponding to the at least one target retransmission parameter to quantize the one data packet; wherein, the data sizes of the retransmission data packets obtained by using different quantization strategies are different, and the data size of the retransmission data packets is positively correlated with the bandwidth occupancy.

7. The method according to claim 6, characterized in that The method further comprises: Obtaining a current first network transmission state according to the latest determined feedback information; Selecting, from the initial retransmission parameters, a candidate retransmission parameter set corresponding to the first network transmission state according to a mapping relationship between the network transmission state and the initial retransmission parameters; Based on the obtained candidate retransmission parameter set, the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end is obtained, and the at least one target retransmission parameter corresponding to each data packet obtained is sent to the transmitting end.

8. The method according to claim 7, characterized in that The number of candidate retransmission parameters included in the candidate retransmission parameter set is the same as the number of times the data packet is repeatedly transmitted; The obtaining, based on the obtained candidate retransmission parameter set, the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end comprises: If the feedback information includes at least one reference retransmission parameter detected by the receiving end, then, according to each candidate retransmission parameter in the candidate retransmission parameter set, respectively smoothing the at least one reference retransmission parameter detected by the receiving end, to obtain the at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end; If the feedback information does not include at least one reference retransmission parameter detected by the receiving end, each candidate retransmission parameter in the candidate retransmission parameter set is used as the at least one target retransmission parameter corresponding to each data packet to be transmitted by the sending end.

9. The method according to claim 6, characterized in that After determining feedback information for indicating transmission quality of the multimedia data according to the multimedia data transmitted by the transmitting end, the method further includes: Feedback information including at least one reference retransmission parameter detected by the receiving end is sent to a third-party device, so that the third-party device determines the at least one target retransmission parameter corresponding to each data packet to be transmitted by the transmitting end according to the at least one reference retransmission parameter detected by the receiving end in the most recently received feedback information, and sends the at least one target retransmission parameter corresponding to each data packet to be transmitted to the transmitting end.

10. A data transmission device, characterized in that: include: A transmission module, used for transmitting multimedia data to a receiving end; A trigger module, configured to trigger repeated transmission of each data packet to be transmitted when it is determined that the transmission quality of the multimedia data does not meet a set standard based on the feedback information of the receiving end; After the repeated transmission is triggered, the following operations are performed for each data packet to be transmitted: An acquisition module, used for acquiring at least one target retransmission parameter corresponding to a data packet; A processing module, configured to determine corresponding quantization strategies according to the at least one target retransmission parameter, and respectively use the obtained at least one quantization strategy to perform quantization processing on the one data packet to obtain at least one retransmission data packet after quantization processing, wherein the data sizes of the retransmission data packets obtained by using different quantization strategies are different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy; The first sending module is used to send the one data packet and the at least one retransmission data packet to the receiving end by using corresponding bandwidth occupancy amounts based on the data size of the at least one retransmission data packet.

11. A data transmission device, characterized in that: include: A determination module, used to determine feedback information indicating the transmission quality of the multimedia data according to the multimedia data transmitted by the sending end; A second sending module, configured to send the feedback information to the sending end, so that when the sending end determines based on the feedback information that the transmission quality of the multimedia data does not meet the set standard, the sending end triggers repeated transmission of each data packet to be transmitted; A receiving module is used to receive each data packet sent by the sending end and at least one retransmission data packet corresponding to each data packet; wherein, the at least one retransmission data packet corresponding to the one data packet is obtained by the sending end after obtaining at least one target retransmission parameter corresponding to the one data packet, and then using the quantization strategy corresponding to the at least one target retransmission parameter to quantize the one data packet; wherein the data size of the retransmission data packet obtained by using different quantization strategies is different, and the data size of the retransmission data packet is positively correlated with the bandwidth occupancy.

12. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor performs the steps of any method in claims 1 to 5, or the steps of any method in claims 6 to 9.

13. A computer-readable storage medium, characterized in that: It includes a computer program. When the computer program is run on an electronic device, the computer program is used to make the electronic device execute the steps of any method in claims 1 to 5 or the steps of any method in claims 6 to 9.

14. A computer program product, characterized in that It includes a computer program, which is stored in a computer-readable storage medium. The processor of the electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device performs the steps of any method in claims 1 to 5, or the steps of any method in claims 6 to 9.

Images