Data Transmission Method, Device and Computer Storage Medium

Abstract

An embodiment of the present application provides a data transmission method, device, and computer storage medium. The data transmission method includes: obtaining original data to be transmitted; determining a hierarchical coding SVC extension header for transmitting the original data and an actual sequence number corresponding to the original data; performing forward error correction code (FEC) processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data. The technical solution provided by the present application performs FEC encoding processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data, and then the target transmission data can be sent to the server. Since the server has different operating modes, it effectively realizes that data transmission operations can be performed based on multiple modes. This not only reduces the data transmission cost and bandwidth cost, but also takes into account the effect of anti-data packet loss, and at the same time makes the method applicable to various application scenarios.

Description

Technical Field

[0001] This application relates to the field of network technologies, and in particular, to a data transmission method, device, and computer storage medium. Background Art

[0002] With the rapid development of network technologies, real-time streaming media transmission has entered the era of real-time communication (RTC) with full-link instant messaging. There are more and more low-latency scenarios. Among them, forward error correction (FEC), as one of the quality of service (QoS) optimization solutions for the full link, can implement the general functions of FEC encoding and decoding to reduce link packet loss, stuttering, and latency problems.

[0003] Currently, when a network server transmits real-time streaming media data to a receiving end, the network server often directly sends the real-time streaming media data to the receiving end. In this way, the quality and effect of data transmission are not guaranteed, and it will also increase the data transmission cost and bandwidth cost corresponding to the network server. Summary of the Invention

[0004] Embodiments of this application provide a data transmission method, device, and computer storage medium. Since the server has different operating modes, it is possible to reduce the data transmission cost and bandwidth cost corresponding to the server while being compatible with the packet loss resistance effect.

[0005] In a first aspect, embodiments of this application provide a data transmission method, including:

[0006] Obtain the original data to be transmitted;

[0007] Determine a hierarchical coding SVC extension header for transmitting the original data and an actual sequence number corresponding to the original data;

[0008] Perform forward error correction code (FEC) processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data.

[0009] In a second aspect, embodiments of this application provide a data transmission device, including:

[0010] A first acquisition module, configured to obtain the original data to be transmitted;

[0011] A first determination module, configured to determine a hierarchical coding SVC extension header for transmitting the original data and an actual sequence number corresponding to the original data;

[0012] The first processing module is configured to perform forward error correction code (FEC) processing on the original data based on the SVC extension header and the actual serial number to generate target transmission data corresponding to the original data.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor; wherein, the memory is used to store one or more computer instructions, and when the one or more computer instructions are executed by the processor, the data transmission method shown in the first aspect above is implemented.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer storage medium for storing a computer program, and when the computer program is executed by a computer, the data transmission method shown in the first aspect above is implemented.

[0015] In a fifth aspect, an embodiment of the present invention provides a computer program product, including: a computer program, when the computer program is executed by a processor of an electronic device, causing the processor to execute the data transmission method shown in the first aspect above.

[0016] In a sixth aspect, an embodiment of the present invention provides a data transmission method, including:

[0017] Receiving target transmission data sent by a sending end;

[0018] Determining an operating mode for transmitting the target transmission data, where the operating mode includes any one of the following: pass-through mode, decode pass-through mode, decode mode, encode-decode mode;

[0019] Sending the target transmission data to a receiving end based on the operating mode.

[0020] In a seventh aspect, an embodiment of the present invention provides a data transmission device, including:

[0021] A second receiving module, configured to receive target transmission data sent by a sending end;

[0022] A second determining module, configured to determine an operating mode for transmitting the target transmission data, where the operating mode includes any one of the following: pass-through mode, decode pass-through mode, decode mode, encode-decode mode;

[0023] A second processing module, configured to send the target transmission data to a receiving end based on the operating mode.

[0024] In an eighth aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor; wherein, the memory is used to store one or more computer instructions, and when the one or more computer instructions are executed by the processor, the data transmission method shown in the sixth aspect above is implemented.

[0025] In a ninth aspect, an embodiment of the present invention provides a computer storage medium for storing a computer program, and when the computer program is executed by a computer, it implements the data transmission method shown in the sixth aspect above.

[0026] In a tenth aspect, an embodiment of the present invention provides a computer program product, including: a computer program, when the computer program is executed by a processor of an electronic device, causing the processor to execute the data transmission method shown in the sixth aspect above.

[0027] In an eleventh aspect, an embodiment of the present application provides a data transmission method, including:

[0028] Receiving target transmission data sent by a server, where the target transmission data includes at least one of the following: FEC data, original data;

[0029] When packet loss occurs in the network, obtaining a virtual sequence number corresponding to the target transmission data based on an SVC extension header included in the target transmission data, where the virtual sequence number is used to make the sequence numbers of all received data continuous, and the SVC extension header includes a frame boundary of the original data;

[0030] Framing the target transmission data based on the virtual sequence number to obtain an image frame corresponding to the target transmission data.

[0031] In a twelfth aspect, an embodiment of the present application provides a data transmission device, including:

[0032] A third receiving module, configured to receive target transmission data sent by a server, where the target transmission data includes at least one of the following: FEC data, original data;

[0033] A third obtaining module, configured to obtain a virtual sequence number corresponding to the target transmission data based on an SVC extension header included in the target transmission data when packet loss occurs in the network, where the virtual sequence number is used to make the sequence numbers of all received data continuous, and the SVC extension header includes a frame boundary of the original data;

[0034] A third processing module, configured to frame the target transmission data based on the virtual sequence number to obtain an image frame corresponding to the target transmission data.

[0035] In a thirteenth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor; wherein, the memory is used to store one or more computer instructions, and when the one or more computer instructions are executed by the processor, the data transmission method shown in the eleventh aspect above is implemented.

[0036] In a fourteenth aspect, an embodiment of the present invention provides a computer storage medium for storing a computer program, which, when executed by a computer, implements the data transmission method shown in the eleventh aspect above.

[0037] In a fifteenth aspect, an embodiment of the present invention provides a computer program product, including: a computer program, which, when executed by a processor of an electronic device, causes the processor to execute the data transmission method shown in the eleventh aspect above.

[0038] The technical solution provided by the embodiments of the present application, after obtaining the original data at the sending end, performs a forward error correction code (FEC) processing operation on the original data based on the SVC extension header and the actual sequence number, thereby ensuring the quality and effect of generating the target transmission data. When sending the target transmission data to the server, when the receiving end receives the target transmission data sent by the server, if packet loss occurs in the network, in order to ensure that the sequence numbers of all the data received by the receiving end are consecutive sequence numbers, the virtual sequence number corresponding to the target transmission data can be determined based on the SVC extension header included in the target transmission data, and then the target transmission data can be framed based on the virtual sequence number. This effectively improves the quality and effect of obtaining the image frames, further improves the practicability of this method, and is conducive to market promotion and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 It is a schematic diagram of a scenario of a data transmission method provided by an embodiment of the present application;

[0041] Figure 2 It is a schematic flowchart of a data transmission method provided by an embodiment of the present application;

[0042] Figure 3 It is a schematic diagram of the SVC extension header provided by an embodiment of the present application;

[0043] Figure 4 It is a schematic flowchart of performing a forward error correction code (FEC) processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data provided by an embodiment of the present application Figure 1 ;

[0044] Figure 5a Schematic diagram of the original data RTP packet provided by the embodiment of the present application;

[0045] Figure 5b Schematic diagram of a kind of FEC data provided by the embodiment of the present application;

[0046] Figure 6 Schematic diagram of performing FEC encoding operation on the header information and payload information of the original data RTP packet provided by the embodiment of the present application to obtain FEC encoded data;

[0047] Figure 7 Schematic diagram of the process of performing forward error correction code FEC processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data Figure 2 ;

[0048] Figure 8 Schematic diagram of the process of another data transmission method provided by the embodiment of the present application;

[0049] Figure 9 Schematic diagram of the process of yet another data transmission method provided by the embodiment of the present application;

[0050] Figure 10 Schematic diagram of the process of sending the target transmission data to the receiving end based on the operating mode provided by the embodiment of the present application;

[0051] Figure 11 Schematic diagram of the process of another data transmission method provided by the embodiment of the present application;

[0052] Figure 12 Schematic diagram of the structure of a data transmission device provided by the embodiment of the present application;

[0053] Figure 13 For Figure 12 Schematic diagram of the structure of the electronic device corresponding to the data transmission device shown;

[0054] Figure 14 Schematic diagram of the structure of another data transmission device provided by the embodiment of the present application;

[0055] Figure 15 For Figure 14 Schematic diagram of the structure of the electronic device corresponding to the data transmission device shown;

[0056] Figure 16 Schematic diagram of the structure of a data transmission device provided by the embodiment of the present application;

[0057] Figure 17 For Figure 16Schematic diagram of the structure of the electronic device corresponding to the data transmission device shown. Detailed implementation manners

[0058] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0059] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "the" and "said" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Plural" generally includes at least two, but does not exclude the case of including at least one.

[0060] It should be understood that the term " / and" used herein is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the preceding and following associated objects.

[0061] Depending on the context, the words "if" and "when" as used herein may be interpreted as "when" or "when...", or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detecting (stated condition or event)" may be interpreted as "when determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)".

[0062] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a commodity or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such commodity or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the commodity or system including the said element.

[0063] In addition, the step time sequence in the following method embodiments is only an example, not a strict limitation.

[0064] Term explanation:

[0065] Forward Error Correction (FEC) is a method of error control. It refers to the technique where signals are pre-encoded according to a certain algorithm before being sent into the transmission channel, with redundant codes carrying the characteristics of the signals themselves added. At the receiving end, the received signals are decoded according to the corresponding algorithm to find and correct the error codes generated during the transmission process.

[0066] Scalable Video Coding (SVC) is a technology used to be compatible with different terminal devices and link bandwidths. Its characteristic is to layer the bitstream, where the lower-layer bitstream can be decoded independently, and the higher-layer bitstream can enhance the video quality.

[0067] To solve the problems existing in the prior art, such as "when transmitting real-time streaming media data from the sending end to the receiving end through a network server, the network server often directly sends the real-time streaming media data to the receiving end, which cannot guarantee the quality and effect of data transmission and will also increase the data transmission cost and bandwidth cost corresponding to the network server", this embodiment provides a data transmission method, device, and computer storage medium. Among them, the execution subject of the data transmission method is a data transmission system. Refer to the attached Figure 1 As shown, the data transmission system may include: a sending end, a server, and a receiving end. The sending end can communicate with the receiving end through the server. Specifically,

[0068] Both the sending end and the receiving end can be any computing device with a certain data transmission ability. Specifically, in implementation, the sending end and the receiving end can be mobile phones, tablet computers, set application programs, etc. In addition, the basic structure of the sending end and the receiving end may include: at least one processor. The number of processors depends on the configuration and type of the sending end and the receiving end. The sending end may also include a memory, which can be volatile, such as RAM, or non-volatile, such as Read-Only Memory (ROM), flash memory, etc., or may include both types at the same time. Usually, an operating system (OS), one or more application programs, and program data may be stored in the memory. In addition to the processing unit and the memory, the sending end and the receiving end also include some basic configurations, such as a network card chip, an IO bus, a display component, and some peripheral devices. Optionally, some peripheral devices may include, for example, a keyboard, a mouse, a stylus, a printer, etc. Other peripheral devices are well known in the art and will not be elaborated here.

[0069] A server refers to a device that can provide data transmission services in a network virtual environment. Generally, it refers to a device that uses the network for information planning and data transmission operations. In terms of physical implementation, a server can be any device that can provide computing services, respond to service requests, and perform processing. For example, it can be a cluster server, a conventional server, a cloud server, a cloud host, a virtual center, etc. The composition of a server mainly includes a processor, a hard disk, memory, a system bus, etc., which is similar to a general computer architecture.

[0070] In the above-mentioned embodiment, the sending end and the receiving end can be network-connected to the server, and this network connection can be a wireless or wired network connection. If the sending end and the receiving end are in a communication connection with the server, the network mode of this mobile network can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, 5G, etc.

[0071] The sending end is used to obtain the original data to be transmitted. In order to ensure the quality and effect of the transmission of the original data, it can determine the hierarchical coding SVC extension header for transmitting the original data and the actual sequence number corresponding to the original data. Then, it can perform a forward error correction code FEC processing operation on the original data based on the SVC extension header and the actual sequence number, so as to generate the target transmission data corresponding to the original data. It should be noted that the target transmission data can include the SVC extension header and the actual sequence number. After generating the target transmission data, it can send the target transmission data to the server.

[0072] The server is used to receive the target transmission data sent by the sending end. Since the server can correspond to multiple different operating modes, and different operating modes can correspond to different data transmission methods, therefore, in order to ensure the flexible reliability of the transmission of the target transmission data, it can determine the operating mode for transmitting the target transmission data, and this operating mode can include any one of the following: pass-through mode, decoded pass-through mode, decoding mode, encoding and decoding mode; then it can send the target transmission data to the receiving end based on the operating mode.

[0073] A receiving end, which is used to receive target transmission data sent by a server. Since the target transmission data transmitted in different operating modes of the server is not quite the same, the target transmission data received by the receiving end may include at least one of the following: FEC data, raw data. When packet loss occurs in the network, in order to make the sequence numbers of all the data received by the receiving end continuous, the virtual sequence number corresponding to the target transmission data can be obtained based on the SVC extension header included in the target transmission data, and then the target transmission data can be framed based on the virtual sequence number, so that an image frame can be obtained by framing the target transmission data with continuous sequence numbers, thus ensuring the accuracy and reliability of obtaining the image frame.

[0074] In the technical solution provided in this embodiment, after obtaining the raw data at the sending end, forward error correction code FEC processing is performed on the raw data based on the SVC extension header and the actual sequence number, thereby ensuring the quality and effect of generating the target transmission data. When sending the target transmission data to the server, since the server may have different operating modes, the operating mode of the server for transmitting the target transmission data can be determined, and then the target transmission data is transmitted to the receiving end based on the operating mode, effectively realizing the transmission operation of data based on multiple modes. This not only reduces the data transmission cost and bandwidth cost corresponding to the server, but also takes into account the effect of anti-packet loss, and at the same time makes this technical solution applicable to various application scenarios. When the receiving end receives the target transmission data sent by the server, if packet loss occurs in the network, in order to ensure that the sequence numbers of all the data received by the receiving end are continuous, the virtual sequence number corresponding to the target transmission data can be determined, and then the target transmission data can be framed based on the virtual sequence number, effectively improving the quality and effect of obtaining the image frame, further improving the practicability of this method and being conducive to market promotion and application.

[0075] The following will describe in detail some embodiments of the present invention with reference to the accompanying drawings. Without conflict between the embodiments, the embodiments and the features in the embodiments can be combined with each other. In addition, the step timings in the following method embodiments are only examples and are not strictly limited.

[0076] Figure 2 It is a schematic flowchart of a data transmission method provided by an embodiment of the present application; refer to the attached Figure 2 As shown, this embodiment provides a data transmission method. The execution subject of this method can be a data transmission device, and this data transmission device can be implemented as software, or a combination of software and hardware. In some instances, the data transmission device can be implemented as a sending end. Specifically, this data transmission method can include the following steps:

[0077] Step S201: Obtain the original data to be transmitted.

[0078] Step S202: Determine the hierarchical coding SVC extension header for transmitting the original data and the actual sequence number corresponding to the original data.

[0079] Step S203: Perform forward error correction code FEC processing on the original data based on the SVC extension header and the actual sequence number to generate the target transmission data corresponding to the original data.

[0080] The following is a detailed description of each of the above steps:

[0081] Step S201: Obtain the original data to be transmitted.

[0082] Among them, when the user has a need to transmit real-time data or non-real-time data, the data transmission device can obtain the original data to be transmitted. The original data can be real-time transmission data or non-real-time transmission data. Specifically, when the original data is real-time transmission data, it can include at least one of the following: real-time audio stream data, real-time video stream data, real-time image stream data, real-time animation stream data, etc. Those skilled in the art can configure different real-time transmission data according to specific application scenarios.

[0083] In addition, the specific acquisition method of the original data to be transmitted in this embodiment is not limited. In some instances, obtaining the original data to be transmitted may include: obtaining the display interface on the sending end for implementing interaction operations with the user, obtaining the data generation operations input by the user in the display interface, and generating and obtaining the original data to be transmitted based on the data generation operations; or, the original data to be transmitted is stored in a preset area or a preset device, and the original data to be transmitted can be obtained by accessing the preset area or the preset device. Of course, the acquisition method of the original data is not limited to the above implementation methods, and those skilled in the art can also use other methods to obtain the original data to be transmitted, as long as the accuracy and reliability of obtaining the original data to be transmitted can be ensured, which will not be elaborated here.

[0084] Step S202: Determine the hierarchical coding SVC extension header for transmitting the original data and the actual sequence number corresponding to the original data.

[0085] After obtaining the original data, in order to ensure the quality and effect of transmitting the original data, a hierarchical coding SVC extension header for transmitting the original data and an actual sequence number corresponding to the original data can be determined. It should be noted that when different original data are obtained, different SVC extension headers and different actual sequence numbers corresponding to different original data can be determined. Among them, the SVC extension header is used to identify the boundary information, frame category information, and layering information of the original data frame. Specifically, refer to the appendix Figure 3 As shown, the SVC extension header may include:

[0086] The T field, 1 bit, is used to identify whether the TID field is valid, and the default value is 0;

[0087] The TID field, which is used to identify the layer information, is 2-bit identification information, and generally can include 3 layers (0, 1, 2);

[0088] The I field, 1 bit, is used to identify whether the data frame is an I frame;

[0089] The S field, 1 bit, is used to identify whether the StartSeq|EndSeq field is valid;

[0090] The E field is used to identify whether there is an extension later, and an extension can be added (specifically to be determined, mainly used to expand Picture_ID and ref ID) 1: There is an extension: Picture_ID;

[0091] The StartSeq field, 16 bits, is the start sequence number of this frame;

[0092] The EndSeq field, 16 bits, is the end sequence number of this frame;

[0093] The RES field, there is a real seq after 1 bit and a 1-bit reserved field for future expansion, and the default value is 0;

[0094] The Picture_ID field, 16 bits, is used to identify the frame sequence number.

[0095] The actual sequence number corresponding to the original data can be used to transmit the real sequence number, and its length is 2 bytes, unsigned. Specifically, the actual sequence number can be obtained through a Uniform Resource Identifier (URI), for example: the actual sequence number can be obtained through the following Uniform Resource Identifier "uri: taobao: rtc-hdrext: video: CompositionTime".

[0096] In addition, this embodiment does not limit the specific implementation manner for determining the hierarchical coding SVC extension header used for transmitting the original data and the actual serial number corresponding to the original data. In some instances, the SVC extension header and the real-time serial number may be pre-configured. At this time, the pre-configured SVC extension header and the actual serial number may be stored in a preset area. When obtaining the original data to be transmitted, the SVC extension header and the actual serial number corresponding to the original data can be obtained by accessing the preset area. In other instances, the SVC extension header and the actual serial number may be generated based on the execution operation input by the user. Specifically, determining the hierarchical coding SVC extension header used for transmitting the original data and the actual serial number corresponding to the original data may include: obtaining an interaction interface for interacting with the user, obtaining the execution operation input by the user in the interaction interface, and generating and determining the SVC extension header and the actual serial number based on the execution operation, thereby effectively ensuring the accuracy and reliability of determining the SVC extension header and the actual serial number.

[0097] Step S203: Perform forward error correction code FEC processing on the original data based on the SVC extension header and the actual serial number to generate target transmission data corresponding to the original data.

[0098] After obtaining the SVC extension header and the actual serial number, a forward error correction code FEC processing operation can be performed on the original data based on the SVC extension header and the actual serial number to generate target transmission data corresponding to the original data. The generated target transmission data may include the SVC extension header and the actual serial number. Since the target transmission data includes the SVC extension header and the actual serial number, the quality and efficiency of transmitting the target transmission data can be ensured based on the SVC extension header and the actual serial number during the transmission of the target transmission data, so that this technical solution can be applied to different congestion control scenarios.

[0099] The data transmission method provided in this embodiment obtains the original data to be transmitted, then determines the hierarchical coding SVC extension header used for transmitting the original data and the actual serial number corresponding to the original data, and performs forward error correction code FEC processing on the original data based on the SVC extension header and the actual serial number, so as to generate target transmission data that includes the SVC extension header and the actual serial number and corresponds to the original data. In this way, during the transmission of the target transmission data, the quality and efficiency of transmitting the target transmission data can be ensured based on the SVC extension header and the actual serial number configured in the target transmission data, so that this method can be applied to different congestion control scenarios, further improving the practicability of this method and facilitating the market promotion and application.

[0100] Figure 4Flow diagram for forward error correction code (FEC) processing of original data based on the SVC extension header and actual sequence number to generate target transmission data provided by the embodiments of the present application Figure 1 ; Refer to the appendix Figure 4 As shown, this embodiment provides an implementation method for generating target transmission data corresponding to the original data. Specifically, the forward error correction code (FEC) processing of the original data based on the SVC extension header and actual sequence number to generate target transmission data corresponding to the original data in this embodiment may include:

[0101] Step S401: Generate an original data RTP packet based on the original data, SVC extension header, and actual sequence number.

[0102] After obtaining the original data, SVC extension header, and actual sequence number, the original data, SVC extension header, and actual sequence number can be analyzed and processed. Specifically, the original data, SVC extension header, and actual sequence number can be processed to generate an original data RTP packet. For the original data RTP packet, it can include not only the SVC extension header and actual sequence number, but also the RTP header. Specifically, refer to the appendix Figure 5a As shown, the RTP header can appear in each RTP packet. Specifically, the RTP header may include:

[0103] Version (V): 2 bits, this field defines the version of RTP. The version defined by this protocol can be 2. (Value 1 is used for the RTP draft version, and value 0 is used for the protocol used in the final "vat" voice tool)

[0104] Padding (P): 1 bit, if the padding bit is set, this packet contains one or more additional padding bits appended at the end. The padding bits are not considered part of the payload. The last byte of the padding indicates how many padding bits can be ignored. Padding may be used for some fixed-length encryption algorithms or for transmitting multiple RTP packets in the underlying data unit.

[0105] Extension (X): 1 bit, if the extension bit is set, a header extension follows the fixed header (only).

[0106] CSRC count (CC): 4 bits, the CSRC count contains the number of CSRC identifiers following the fixed header.

[0107] Marker (M): 1 bit, the interpretation of the marker is specified by the specific protocol. It is used to allow marking important events in the bit stream, such as frame boundaries.

[0108] Payload Type (PT): 7 bits. This field defines the format of the payload, and its interpretation is application - specific. The protocol may define a default mapping between payload type codes and payload formats. Other payload type codes can be defined dynamically using non - RTP methods. The RTP sender emits a single RTP payload type at any given time; this field is not used for multiplexing different media streams.

[0109] Sequence Number: 16 bits. For each RTP data packet sent, the sequence number is incremented by 1. The receiver can use this to detect packet loss and reconstruct the packet sequence. The initial value of the sequence number is random (unpredictable) to make it more difficult for a known - plaintext attack on the encryption algorithm, even when the source itself is not encrypted (sometimes packets pass through a translator and it does this).

[0110] Timestamp: 32 bits. The timestamp reflects the sampling time of the first byte in the RTP data packet. The clock frequency depends on the payload data format and is described in the profile. It can also be described dynamically for the payload format using RTP methods.

[0111] Synchronization Source (SSRC) Identifier: 32 bits, used to identify the synchronization source. This identifier is randomly chosen, and two synchronization sources participating in the same video conference cannot have the same SSRC. ssrc = len(actual RTP packet length).

[0112] Contributing Source (CSRC) Identifiers: Each CSRC identifier is 32 bits, and there can be 0 to 15 of them. Each CSRC identifies all the contributing sources included in the RTP message payload.

[0113] In addition, before performing the Forward Error Correction Code (FEC) processing operation on the original data, the FEC data can protect the RTP header information header and the payload information payload. Specifically, the FEC packet can protect the RTP header information header and the payload information payload by adding the corresponding SVC extension header and the actual sequence number real seqNumber.

[0114] Reference appendix Figure 5bAs shown, the payload information corresponds to the original data. The RTP header information may include an sn_base field, a k field, an n field, an i field, a max_grp_pkt_size field, a c field, and a reserved field. The above-mentioned sn_base field is used to identify the sequence number seq_num (16-bit) of the first original packet in the FEC packet. k is used to identify the original packet format (8-bit). n is used to identify the total number of original packets and redundant packets (8-bit). i is used to identify the relative sequence number of each packet in the packet group (8-bit). The max_grp_pkt_size field is used to identify the maximum packet length in the FEC packet (16-bit). c is 1 bit and is used to identify whether it crosses frames. 0 is used for intra-frame coding, and 1 is used for inter-frame coding. The reserved field is used to identify the reserved field (7-bit). Among them, inter-frame coding can perform coding operations on the set of non-key frames, which can save coding dimensions, thereby achieving a trade-off between data processing cost and data latency. Intra-frame coding is used to perform coding operations on key frames or large frames. Generally, when latency is not a concern, inter-frame coding can be preferentially used to encode data frames. In a real-time transmission scenario, intra-frame coding can be preferentially used to encode data frames.

[0115] In addition, after performing forward error correction code (FEC) processing on the original data, the FEC data can be separately configured with a synchronization source identifier (ssrc) and a pt identifier for identifying the payload type. The separately configured ssrc facilitates the management and maintenance operations of the FEC data. The above-mentioned ssrc can also be used to identify the actual RTP packet length. During the data transmission process, since the ssrc does not change, during the FEC encoding process of the original data, no processing operation needs to be performed on the ssrc. Similarly, since the pt identifier for identifying the payload type may change during the data transmission process, before transmitting the original data, pt can be set to zero, that is, pt = 0, and then the zeroed pt can be encoded and transmitted. After the receiving end obtains pt, the data recovery operation can be performed on pt to determine the payload type. During the transmission process of the original data, the sequence number seq may change, so an extension header for the actual sequence number needs to be added. The timestamp information does not change during the transmission process, and no processing operation needs to be performed during the transmission process of the original data. Other frame-level extension headers (such as the cts extension header) can be placed in the extension module (i.e., the ext info module) of the FEC data packet. In addition, the FEC data packet can also include an identification bit (C bit) for identifying whether cross-frame operation is performed, and intra-frame coding or inter-frame coding can be achieved through the identification bit.

[0116] Step S402: Perform FEC encoding operations on the header information and payload information of the original data RTP packet to obtain FEC-encoded data.

[0117] After obtaining the original data RTP packet, FEC encoding operations can be performed on the header information and payload information of the original data RTP packet, so that FEC-encoded data can be obtained. Specifically, as shown in the appendix Figure 6 The header information of the original data RTP packet may include: IP header information, transport header, RTP header. The payload information of the original data RTP packet may be the original data. In order to achieve stable transmission operations on the original data, FEC encoding operations can be performed on the header information and payload information of the original data RTP packet, so that FEC-encoded data can be obtained. At this time, the FEC-encoded data may include FEC redundant data corresponding to the original data. The data volume corresponding to the FEC redundant data is related to the FEC redundancy. Generally, when the FEC redundancy is higher, the data volume corresponding to the FEC redundant data is larger. It should be noted that the original data in different application scenarios can be configured with different FEC redundancies.

[0118] Step S403: Generate target transmission data corresponding to the original data based on the FEC-encoded data and the SVC extension header.

[0119] After obtaining the FEC-encoded data and the SVC extension header, analysis and processing can be performed on the FEC-encoded data and the SVC extension header. Specifically, the FEC-encoded data and the SVC extension header can be directly integrated to obtain the target transmission data.

[0120] In this embodiment, based on the original data, the SVC extension header, and the actual sequence number, an original data RTP packet is generated. Then, FEC encoding operations are performed on the header information and payload information of the original data RTP packet to obtain FEC-encoded data, and analysis and processing are performed on the FEC-encoded data and the SVC extension header to generate target transmission data corresponding to the original data, thereby effectively improving the accuracy and reliability of determining the target transmission data, and further ensuring the quality and effect of data transmission based on the target transmission data.

[0121] Figure 7 It is a flowchart showing the forward error correction code FEC processing of the original data based on the SVC extension header and the actual sequence number provided by the embodiment of the present application to generate target transmission data corresponding to the original data Figure 2 ; Refer to the appendix Figure 7As shown in the figure, this embodiment provides another implementation manner for generating target transmission data corresponding to the original data. Specifically, in this embodiment, forward error correction code (FEC) processing is performed on the original data based on the SVC extension header and the actual sequence number, and the generation of the target transmission data corresponding to the original data may include:

[0122] Step S701: When the b-frame is included in the original data, obtain the cts extension header, which is the difference time between the video display time and the decoding time, for encoding and decoding the b-frame in the original data.

[0123] Among them, when the b-frame is included in the original data, in order to ensure the stable and reliable transmission of the b-frame data, the cts extension header for encoding and decoding the b-frame in the original data can be obtained. The cts extension header is the difference time between the video display time and the decoding time. In some instances, the cts extension header corresponds to the transmitted video, and its unit is ms (millisecond). Specifically, this embodiment does not limit the specific manner of obtaining the cts extension header. In some instances, the cts extension header can be pre-configured. At this time, the pre-configured cts extension header can be stored in a preset area. When obtaining the original data to be transmitted and the original data includes the b-frame, the difference time cts extension header between the video display time and the decoding time for encoding and decoding the b-frame in the original data can be obtained by accessing the preset area. In other instances, the cts extension header can be generated based on the execution operation input by the user. Specifically, obtaining the cts extension header, which is the difference time between the video display time and the decoding time, for encoding and decoding the b-frame in the original data may include: obtaining an interaction interface for interacting with the user, obtaining the execution operation input by the user in the interaction interface, and generating and determining the cts extension header based on the execution operation, thereby effectively ensuring the accuracy and reliability of determining the cts extension header.

[0124] Step S702: Perform forward error correction code (FEC) processing on the original data based on the SVC extension header, the actual sequence number, and the cts extension header to generate target transmission data corresponding to the original data.

[0125] After obtaining the SVC extension header, the actual sequence number, and the cts extension header, forward error correction code (FEC) processing can be performed on the original data based on the SVC extension header, the actual sequence number, and the cts extension header to generate target transmission data corresponding to the original data. The generated target transmission data may include the SVC extension header, the actual sequence number, and the cts extension header.

[0126] In addition, the specific implementation method for generating the target transmission data corresponding to the original data in this embodiment is not limited. In some examples, a machine learning model for generating the target transmission data is pre-configured. After obtaining the original data, the SVC extension header, the actual sequence number, and the cts extension header, the original data, the SVC extension header, the actual sequence number, and the cts extension header can be input into the machine learning model, so that the target transmission data output by the machine learning model can be obtained.

[0127] In other examples, forward error correction code (FEC) processing is performed on the original data based on the SVC extension header, the actual sequence number, and the cts extension header. Generating the target transmission data corresponding to the original data may include: generating the original data RTP packet based on the original data, the SVC extension header, the cts extension header, and the actual sequence number; performing FEC encoding operations on the header information and payload information of the original data RTP packet to obtain FEC-encoded data; and generating the target transmission data corresponding to the original data based on the FEC-encoded data, the SVC extension header, and the cts extension header.

[0128] After obtaining the original data, the SVC extension header, the cts extension header, and the actual sequence number, analysis and processing can be performed on the original data, the SVC extension header, the cts extension header, and the actual sequence number. Specifically, the original data, the SVC extension header, the cts extension header, and the actual sequence number can be processed to generate the original data RTP packet. After obtaining the original data RTP packet, FEC encoding operations can be performed on the header information and payload information of the original data RTP packet. After obtaining the FEC-encoded data and the cts extension header, the target transmission data corresponding to the original data can be generated based on the FEC-encoded data, the SVC extension header, and the cts extension header. Specifically, the implementation method and implementation effect of the above steps in this embodiment are similar to those of steps S402 - S403 in the above embodiment. For specific reference, please refer to the above description and will not be elaborated here.

[0129] In this embodiment, when the original data includes B-frames, by obtaining the difference time cts extension header between the video display time and the decoding time for encoding and decoding the B-frames in the original data, generating the original data RTP packet based on the original data, the SVC extension header, the cts extension header, and the actual sequence number, then performing FEC encoding operations on the header information and payload information of the original data RTP packet to obtain FEC-encoded data, and analyzing and processing the FEC-encoded data, the SVC extension header, and the cts extension header to generate the target transmission data corresponding to the original data, the flexible reliability of determining the target transmission data is effectively improved, and further the quality and effect of data transmission based on the target transmission data are ensured.

[0130] Figure 8 A flowchart of another data transmission method provided by an embodiment of the present application; based on any of the above embodiments, with reference to the attached Figure 8 As shown, this embodiment provides an implementation manner for a sender to determine a corresponding operating mode of a server. Specifically, after generating target transmission data corresponding to the original data, the method in this embodiment may further include:

[0131] Step S801: Determine the network operating state and scenario information corresponding to the target transmission data.

[0132] After obtaining the target transmission data, in order to ensure the stable and reliable transmission of the target transmission data, the network operating state and scenario information corresponding to the target transmission data can be determined. Among them, the network operating state corresponding to the target transmission data may include: good network state, general network state, poor network state. Specifically, determining the network operating state corresponding to the target transmission data may include: obtaining the data transmission rate corresponding to the target transmission data; determining the network operating state corresponding to the target transmission data based on the data transmission rate. In some instances, when the data transmission rate is greater than or equal to a preset threshold, it is determined that the network operating state corresponding to the target transmission data is a good state; when the data transmission rate is less than the preset threshold, it is determined that the network operating state corresponding to the target transmission data is a poor state.

[0133] In addition, the scenario information may include at least one of the following: live broadcast scenario, real-time meeting scenario, real-time communication scenario (such as: voice call scenario, video call scenario), etc. Among them, the live broadcast scenario includes but is not limited to: distance education, live courses, telemedicine, etc. in the education scenario. For the scenario information, the specific implementation manner of the scenario information in this embodiment is not limited. In some instances, the scenario information may be directly configured by the user. At this time, determining the scenario information corresponding to the target transmission data may include: obtaining an interaction interface for interacting with the user, obtaining the execution operation input by the user in the interaction interface, and determining the scenario information corresponding to the target transmission data based on the execution operation. Or, in some other instances, the target transmission data may include scenario identification information for identifying the scenario information. After obtaining the target transmission data, the scenario identification information included in the target transmission data can be extracted, and then the scenario information corresponding to the target transmission data can be determined based on the scenario identification information, thereby effectively ensuring the accuracy and reliability of determining the scenario information.

[0134] Step S802: Based on the network operating status and scenario information, determine the operating mode corresponding to the server. The operating mode includes any one of the following: pass-through mode, decode-pass-through mode, decode mode, encode-decode mode.

[0135] After obtaining the network operating status and scenario information, the network operating status and scenario information can be analyzed and processed to determine the operating mode corresponding to the server. Among them, the operating mode can include at least one of the following: pass-through mode, decode-pass-through mode, decode mode, encode-decode mode.

[0136] Specifically, when the server is in the pass-through mode, after the server obtains the target transmission data, no data processing operation will be performed on the target transmission data, and the target transmission data will be directly sent to the receiving end. This pass-through mode can be applied to live application scenarios or one-to-one live scenarios with a relatively short network link. In addition, when the server is in the pass-through mode, the FEC redundancy corresponding to the target transmission data can also be dynamically adjusted according to the network operating status. For example, when the network operating status is good, the FEC redundancy corresponding to the target transmission data can be the first redundancy; when the network operating status is average, the FEC redundancy corresponding to the target transmission data can be the second redundancy, where the second redundancy is greater than the first redundancy, which is beneficial to saving network bandwidth costs and improving the utilization rate of data resources.

[0137] When the server is in the decode-pass-through mode, the server can obtain the target transmission data. After the server obtains the target transmission data, the first-hop data in the target transmission data can be decoded to obtain the first-hop original data corresponding to the first-hop data; then, based on the first-hop original data, the target transmission data is sent to the receiving end. This decode-pass-through mode can be applied to specific live scenarios, such as live scenarios that require specific protection to ensure that the first-hop data is not lost.

[0138] When the server is in the decode mode, the server can obtain the target transmission data. After the server obtains the target transmission data, the first-hop data in the target transmission data and other-hop data located after the first-hop data can be obtained; then, the FEC data included in the other-hop data is deleted to obtain the adjusted data; the first-hop data and the adjusted data are sent to the receiving end. When the server is in the encode-decode mode, after the server obtains the target transmission data, the target transmission data can be decoded to obtain the original data corresponding to the target transmission data; then, the original data is encoded to obtain the encoded data; and the encoded data is sent to the receiving end.

[0139] In addition, the specific implementation manner for determining the operating mode corresponding to the server in this embodiment is not limited. In some examples, a machine learning model for determining the operating mode is pre-trained. After obtaining the network operating state and scenario information, the network operating state, scenario information, and target transmission data can be input into the machine learning model, so that the operating mode corresponding to the server output by the machine learning model can be obtained. Alternatively, in other examples, determining the operating mode corresponding to the server based on the network operating state may include: determining an operating mode identifier for transmitting the target transmission data corresponding to the server based on the network operating state and scenario information; and determining the operating mode corresponding to the server based on the operating mode identifier.

[0140] Specifically, after obtaining the network operating state and scenario information, the network operating state and scenario information can be analyzed and processed to determine an operating mode identifier for transmitting the target transmission data corresponding to the server. In some examples, the data transmission device is communicatively connected to a management center. After the data transmission device obtains the network operating state and scenario information, the network operating state and scenario information can be sent to the management center. After the management center obtains the network operating state and scenario information, the operating mode corresponding to the server can be determined based on the network operating state and scenario information, and then the operating mode corresponding to the server can be sent to the data transmission device, so that the data transmission device can stably obtain or determine the operating mode corresponding to the server.

[0141] In this embodiment, by determining the network operating state and scenario information corresponding to the target transmission data, and then determining the operating mode corresponding to the server based on the network operating state and scenario information, the accuracy and reliability of determining the operating mode corresponding to the server are effectively ensured. Furthermore, the target transmission data can be transmitted based on the operating mode of the server, further ensuring the stability and reliability of transmitting the target transmission data.

[0142] Figure 9 It is a schematic flowchart of another data transmission method provided by an embodiment of the present application; refer to the appendix Figure 9 As shown, this embodiment provides another data transmission method. The execution subject of this method can be a data transmission device, and this data transmission device can be implemented as software, or a combination of software and hardware. In some examples, the data transmission device can be implemented as a server. Specifically, this data transmission method may include the following steps:

[0143] Step S901: Receive the target transmission data sent by the sending end.

[0144] Among them, when the target transmission data is obtained at the sending end and sent to the server, the server can receive the target transmission data sent by the sending end. The target transmission data can be real-time transmission data. Specifically, the real-time transmission data can include at least one of the following: real-time audio stream data, real-time video stream data, real-time image stream data, real-time animation stream data, etc. Those skilled in the art can configure different real-time transmission data according to specific application scenarios.

[0145] Step S902: Determine the operating mode for transmitting the target transmission data. The operating mode includes any one of the following: pass-through mode, decode pass-through mode, decode mode, encode-decode mode.

[0146] Since the server has different operating modes, different operating modes can correspond to different data transmission operations. Therefore, in order to ensure the stable and reliable transmission of the target transmission data, the operating mode for transmitting the target transmission data can be determined. The operating mode can include at least one of the following: pass-through mode, decode pass-through mode, decode mode, encode-decode mode.

[0147] In addition, the specific implementation method for determining the operating mode for transmitting the target transmission data in this embodiment is not limited. In some instances, the operating mode of the server can be configured by the sending end. Specifically, before processing the target transmission data, the sending end can determine the operating mode of the server corresponding to the target transmission data, and then can send a mode operation instruction to the server. After the server obtains the mode operation instruction, it can determine the operating mode it is in based on the mode operation instruction. Or, in other instances, the operating mode of the server can be determined by the server based on the network operating state and scenario information. Specifically, before processing the target transmission data, the method in this embodiment can further include: obtaining the network operating state and scenario information. After obtaining the network operating state and scenario information, the network operating state and scenario information can be analyzed and processed to determine the operating mode corresponding to the server. Among them, the operating mode can include at least one of the following: pass-through mode, decode pass-through mode, decode mode, encode-decode mode.

[0148] Among them, a machine learning model pre-trained for determining the operating mode can, after obtaining the network operating state and scenario information, input the network operating state, scenario information, and target transmission data into the machine learning model, so as to obtain the operating mode corresponding to the server output by the machine learning model. Alternatively, in some other instances, determining the operating mode corresponding to the server based on the network operating state and scenario information may include: obtaining a mapping table for determining the operating mode, performing a look-up operation based on the network operating state and scenario information, and determining an operating mode identifier corresponding to the server for transmitting the target transmission data; and determining the operating mode corresponding to the server based on the operating mode identifier.

[0149] Step S903: Send the target transmission data to the receiving end based on the operating mode.

[0150] After obtaining the operating mode of the server, the target transmission data can be sent to the receiving end based on the operating mode of the server, so that the receiving end can stably receive the target transmission data.

[0151] It should be noted that since the operating modes of the server can include the pass-through mode, the decoded pass-through mode, the decoding mode, the encoding and decoding mode, etc., and different operating modes can correspond to different data transmission strategies, in order to enable users to understand the differences between the above operating modes, the data transmission strategies corresponding to each operating mode of the server will be described in detail below:

[0152] Example 1: When the operating mode of the server is the pass-through mode, sending the target transmission data to the receiving end based on the operating mode may include: directly sending the target transmission data to the receiving end.

[0153] Specifically, when the operating mode of the server is the pass-through mode, full-link pass-through operation of the target transmission data is effectively achieved. At this time, the server will not perform any encoding or decoding processing on the target transmission data. This pass-through mode is applicable to scenarios of large-scale data transmission or scenarios with cost restrictions on the server, such as: live broadcast scenarios, one-to-one communication connection scenarios with very short network links, etc.

[0154] Example 2: When the operating mode of the server is the decoded pass-through mode, sending the target transmission data to the receiving end based on the operating mode may include: obtaining the first-hop data in the target transmission data; decoding the first-hop data to obtain the first-hop original data corresponding to the first-hop data; and sending the target transmission data to the receiving end based on the first-hop original data.

[0155] Among them, since the stability of the first-hop data transmission directly affects the transmission effect of the entire data. For example, if the first-hop data transmission of the target transmission data is abnormal, problems will occur in the transmission operations of all subsequent data of the target transmission data. Therefore, the transmission stability of the first-hop data has a great impact on the entire transmission process of the target transmission data. To ensure the stable and reliable transmission of the target transmission data, when the server is in the decoding transparent transmission mode, the server can decode the first-hop data in the received target transmission data to recover the first-hop original data corresponding to the first-hop data, and then can send the target transmission data to the receiving end based on the first-hop original data, which can effectively solve the problem of packet loss of the first-hop data. This decoding transparent transmission mode is applicable to scenarios of large-scale data transmission or scenarios with limited server costs, such as: live broadcast scenarios with specific protected objects, data transmission scenarios with specific protection strategies.

[0156] It should be noted that sending the target transmission data to the receiving end based on the first-hop original data may include: detecting whether there is a packet loss situation in the first-hop original data in the target transmission data. If there is no packet loss situation in the first-hop original data in the target transmission data, the target transmission data can be directly transparently transmitted to the receiving end; if there is a packet loss situation in the first-hop original data in the target transmission data, the first-hop original data obtained by decoding is used to replace the first-hop data included in the target transmission data to obtain the replaced transmission data, and then the replaced transmission data can be sent to the receiving end, thus effectively ensuring that the receiving end can stably receive the first-hop processing of the target transmission data, and further effectively avoiding the situation of first-hop data packet loss.

[0157] Example 3: When the server is in the decoding mode, sending the target transmission data to the receiving end based on the operating mode may include: obtaining the first-hop data in the target transmission data and other-hop data after the first-hop data; deleting the FEC data included in the other-hop data to obtain the adjusted data; sending the first-hop data and the adjusted data to the receiving end.

[0158] Among them, the decoding mode of the server can solve the problem of weak network in the first hop. Specifically, when the network bandwidth or network state for transmitting the target transmission data is poor, in order to ensure the stable and reliable transmission of the first-hop data, after the server obtains the target transmission data, it can determine the first-hop data included in the target transmission data and other-hop data located after the first-hop data. Affected by the network bandwidth or network state, in order to improve the quality and effect of transmitting the load data, valid data, and first-hop data, the FEC data (i.e., FEC-encoded redundant data) included in the other-hop data can be deleted, so that the adjusted data can be obtained. After obtaining the adjusted data and the first-hop data, the first-hop data and the adjusted data can be sent to the receiving end, effectively ensuring that the receiving end can obtain the first-hop data and the adjusted data. Among them, the first-hop data corresponds to an FEC encoding strategy, while the adjusted data does not correspond to an FEC encoding strategy, effectively realizing the transmission operation of the target transmission data in the case of a weak network.

[0159] Example 4: When the operating mode of the server is the encoding and decoding mode, sending the target transmission data to the receiving end based on the operating mode may include: performing decoding processing on the target transmission data to obtain the original data corresponding to the target transmission data; encoding the original data to obtain the encoded data; and sending the encoded data to the receiving end.

[0160] Among them, when the operating mode of the server is the encoding and decoding mode, it means that after the server obtains the target transmission data, encoding and decoding operations will be performed on each hop of data in the target transmission data. Specifically, first, the target transmission data is decoded to obtain the original data corresponding to the target transmission data, and then the original data is encoded to obtain the encoded data, and then the encoded data can be sent to the receiving end, that is, the server realizes point-to-point encoding and decoding operations, which will reduce the transmission delay of the entire communication link and ensure the quality and effect of transmitting the target transmission data, but will increase the cost of the server's central processing unit (CPU).

[0161] As can be seen from the above, different operating modes of the server can achieve different data transmission operations and have different data transmission effects. Those skilled in the art can configure the operating mode of the server according to specific application scenarios and application requirements, which will not be elaborated here.

[0162] The data transmission method provided in this embodiment determines an operating mode for transmitting the target transmission data by receiving the target transmission data sent by the sending end, and sends the target transmission data to the receiving end based on the operating mode. Among them, since the operating modes that the server can be in can include the pass-through mode, the decoded pass-through mode, the decoding mode, and the encoding and decoding mode, and different operating modes can correspond to different data transmission strategies, it effectively solves the problem of high CPU cost and bandwidth cost in the server in the scenario of large-scale use of FEC. Specifically, in different operating modes that the server is in, different strategies can be adopted to perform flexible and effective data transmission operations on the target transmission data. For example, it supports the combined operation of the congestion control scenario and the FEC strategy, supports the full-link pass-through operation, supports multiple application scenarios, and supports dynamic interaction operations. This not only reduces the bandwidth cost but also takes into account the effect of anti-packet loss, further improving the flexible reliability of the method and facilitating the market promotion and application.

[0163] Figure 10 It is a schematic flow diagram for sending target transmission data to the receiving end based on the operating mode provided in the embodiment of the present application; refer to the appendix Figure 10 As shown, an implementation method for transmitting target transmission data in a congestion control scenario is provided in this embodiment. Specifically, the sending of the target transmission data to the receiving end based on the operating mode in this embodiment can include:

[0164] Step S1001: Obtain congestion control parameters, where the congestion control parameters characterize the degree of network congestion for transmitting the target transmission data.

[0165] Among them, after the server obtains the target transmission data, in order to ensure the stable reliability of transmitting the target transmission data, congestion control parameters can be obtained. The congestion control parameters characterize the degree of network congestion for transmitting the target transmission data. In some instances, the congestion control parameters can be positively correlated with the degree of network congestion, that is, when the degree of network congestion is greater, the congestion control parameters are greater, and when the degree of network congestion is smaller, the congestion control parameters are smaller.

[0166] In addition, the specific method for obtaining the congestion control parameters in this embodiment is not limited. In some instances, the congestion control parameters can be pre-configured. The pre-configured congestion control parameters can be stored in a preset area or a preset device, and the congestion control parameters can be obtained by accessing the preset area or the preset device. Or, in other instances, the congestion control parameters can be automatically determined based on the degree of network congestion. At this time, obtaining the congestion control parameters can include: obtaining the degree of network congestion for transmitting the target transmission data. Generally, the degree of network congestion is related to the network transmission rate, network bandwidth, and the data volume of the target transmission data. Therefore, the degree of network congestion can be determined by the network transmission rate, network bandwidth, and the data volume of the target transmission data. After obtaining the degree of network congestion, the congestion control parameters corresponding to the degree of network congestion can be determined, thus effectively ensuring the accuracy and reliability of obtaining the congestion control parameters.

[0167] Step S1002: Adjust the target transmission data based on the congestion control parameters to obtain the adjusted transmission data.

[0168] Since different congestion control parameter identifiers indicate different degrees of network congestion for transmitting the target transmission data, when transmitting the target transmission data using networks with different degrees of congestion, in order to ensure the stable and reliable transmission of the target transmission data, after obtaining the congestion control parameters, the target transmission data can be adjusted based on the congestion control parameters to obtain the adjusted transmission data, and the data volume corresponding to the adjusted transmission data is less than the data volume corresponding to the target transmission data.

[0169] In addition, the specific implementation method for adjusting the target transmission data based on the congestion control parameters in this embodiment is not limited. In some instances, adjusting the target transmission data based on the congestion control parameters to obtain the adjusted transmission data can include: determining the data volume to be deleted corresponding to the target transmission data based on the congestion control parameters; performing a deletion operation on the target transmission data based on the data volume to be deleted, thereby obtaining the adjusted transmission data. Or, in other instances, adjusting the target transmission data based on the congestion control parameters to obtain the adjusted transmission data can include: obtaining the priority corresponding to each data in the target transmission data; deleting some data in the target transmission data based on the priority and the congestion control parameters to obtain the adjusted transmission data.

[0170] Among them, the target transmission data includes various data such as original data and FEC-encoded redundant data. The FEC-encoded redundant data is used to implement data recovery operations in case of data packet loss, that is, to ensure the stable and reliable transmission of the original data. When the network is congested, it is necessary to first ensure the stable transmission of the original data. At this time, for each data such as the original data and FEC-encoded redundant data in the target transmission data, different transmission priorities can be configured. Generally, the transmission priority of the original data is higher than that of the FEC-encoded redundant data.

[0171] As can be seen from the above, in order to be able to stably and effectively transmit the target transmission data using networks with different congestion levels, the priorities corresponding to each data in the target transmission data can be obtained. Among them, the priorities corresponding to each data in the target transmission data can be pre-configured. For example: the priority corresponding to the original data is the first priority, the priority corresponding to the FEC-encoded redundant data is the second priority, the priority corresponding to other data is the third priority, and so on. Among them, the transmission priority of the first priority is higher than that of the second priority, and the transmission priority of the second priority is higher than that of the third priority. After obtaining the priorities corresponding to each data in the target transmission data, some data in the target transmission data can be deleted based on the priorities and congestion control parameters to obtain the adjusted transmission data.

[0172] It should be noted that deleting some data in the target transmission data based on the priorities and congestion control parameters can include: based on the SVC extension header information included in the target transmission data, determining the hierarchical information corresponding to the target transmission data based on the SVC extension header information, and then performing hierarchical deletion operations on some data in the target transmission data based on the priorities, congestion control parameters, and hierarchical information, so as to obtain the adjusted transmission data.

[0173] For example, the SVC extension header information in the target transmission data can be used to determine that the target transmission data includes original data and FEC data. Among them, the priority of the original data is higher than that of the FEC data. The FEC data can include three layers of data, namely, layer 0 FEC data, layer 1 FEC data, and layer 2 FEC data. Among them, the priorities corresponding to the above-mentioned layer 0 FEC data, layer 1 FEC data, and layer 2 FEC data can be the same or different. In some instances, when data is hierarchically processed, the smaller the layer number corresponding to the data, the higher the priority of the data, that is, the priority of the above-mentioned layer 0 FEC data is higher than that of the layer 1 FEC data, and the priority of the layer 1 FEC data is higher than that of the layer 2 FEC data. After obtaining the congestion control parameters, in some cases, only one layer of data in the FEC data needs to be deleted based on the congestion control parameters and the priority. For example, the layer 2 FEC data can be deleted, and the layer 0 FEC data and the layer 1 FEC data can be retained, so that the adjusted transmission data including the original data, the layer 0 FEC data, and the layer 1 FEC data can be obtained. In some cases, two layers of data in the FEC data need to be deleted based on the congestion control parameters and the priority. For example: the layer 2 FEC data and the layer 1 FEC data can be deleted, and the layer 0 data can be retained, so that the adjusted transmission data including the original data and the layer 0 FEC data can be obtained. In still other cases, all three layers of data in the FEC data need to be deleted based on the congestion control parameters and the priority. For example: the layer 0 FEC data, the layer 1 FEC data, and the layer 2 data can be deleted, so that the adjusted transmission data including only the original data can be obtained.

[0174] Step S1003: Send the adjusted transmission data to the receiving end based on the operating mode.

[0175] After obtaining the adjusted transmission data, the adjusted transmission data can be sent to the receiving end based on the operating mode, so that the receiving end can stably receive the adjusted transmission data. Among them, the implementation method and implementation effect of sending the adjusted transmission data to the receiving end based on the operating mode in this embodiment are similar to those of step S903 in the above embodiment. For specific reference, please refer to the above content and will not be elaborated here.

[0176] In this embodiment, by obtaining the congestion control parameters, then adjusting the target transmission data based on the congestion control parameters to obtain the adjusted transmission data, and sending the adjusted transmission data to the receiving end based on the operating mode, it effectively realizes flexible and effective transmission operations on the target transmission data based on different operating modes of the server in cases of different network congestion degrees, further improving the practicability and application scope of this method.

[0177] Figure 11 It is a schematic flowchart of another data transmission method provided by an embodiment of the present application; refer to the attached Figure 11 As shown, this embodiment provides another data transmission method. The execution subject of this method can be a data transmission device, which can be implemented as software, or a combination of software and hardware. In some instances, the data transmission device can be implemented as a receiving end. Specifically, this data transmission method can include the following steps:

[0178] Step S1101: Receive target transmission data sent by the server. The target transmission data includes at least one of the following: FEC data, original data.

[0179] Among them, when the server sends target transmission data to the receiving end, the receiving end can receive the target transmission data. It should be noted that since the server has different operating modes and can adjust and process the target transmission data based on the network congestion degree, the target transmission data received by the receiving end can include at least one of the following: FEC data, original data. That is, in some instances, the target transmission data received by the receiving end can only include FEC data. At this time, the original data in the target transmission data is lost during the transmission process. In some instances, the target transmission data received by the receiving end can only include original data. At this time, the FEC data in the target transmission data is lost or deleted during the transmission process. In some instances, the target transmission data received by the receiving end can include original data and FEC data. At this time, no packet loss occurs during the transmission of the target transmission data.

[0180] Step S1102: When packet loss occurs in the network, obtain a virtual sequence number corresponding to the target transmission data based on the SVC extension header included in the target transmission data. The virtual sequence number is used to make the sequence numbers of all received data continuous, where the SVC extension header includes the frame boundary of the original data.

[0181] During the process of transmitting target transmission data, packet loss may occur due to the operating state of the network. For example, when the network is unstable or there is a temporary failure of network devices, packet loss will occur in the network; or, when network congestion occurs, the server will actively discard frames to reduce the bit rate, and at this time, packet loss will also occur in the network. Regardless of the reason for packet loss in the network, when packet loss occurs in the network, in order to make the sequence numbers in all the data received by the receiving end continuous, the virtual sequence number corresponding to the target transmission data can be obtained. Specifically, the specific implementation method for obtaining the virtual sequence number corresponding to the target transmission data in this embodiment is not limited. In some instances, the virtual sequence number can be configured by the user. At this time, obtaining the virtual sequence number corresponding to the target transmission data may include: obtaining a display interface for interacting with the user, obtaining the execution operation input by the user on the display interface, and determining the virtual sequence number corresponding to the target transmission data based on the execution operation. Or, in other instances, obtaining the virtual sequence number corresponding to the target transmission data based on the SVC extension header included in the target transmission data may include: obtaining the actual sequence number corresponding to the target transmission data; determining the mapping relationship between the virtual sequence number and the actual sequence number; and determining the virtual sequence number corresponding to the actual sequence number based on the mapping relationship and the frame boundary of the original data included in the SVC extension header.

[0182] Among them, the target transmission data may include the actual sequence number corresponding to the actual transmission data. The actual sequence number can be stored in the RTP data packet corresponding to the original data. Therefore, after obtaining the target transmission data, feature extraction operations can be performed on the target transmission data, so as to obtain the actual sequence number corresponding to the target transmission data. For the target transmission data, a mapping relationship between the virtual sequence number and the actual sequence number is pre-configured. One implementation method of the mapping relationship can be seen in the following table:

[0183] Virtual serial number V1 V2 V3 V4 V5 V6 V7 ...... V12 V13 Actual serial number r5 r6 r10 r11 r15 r20 r21

[0184] After obtaining the actual sequence number, the virtual sequence number corresponding to the actual sequence number can be determined based on the mapping relationship and the frame boundary of the original data included in the SVC extension header, thus effectively realizing the accuracy and reliability of determining the virtual sequence number.

[0185] It should be noted that during the transmission of the target transmission data, packet loss or out-of-order situations often occur. For example, there may be scenarios of partial packet loss or the complete loss of corresponding frames. In different scenarios, there can be different ways to determine the virtual sequence number. Specifically, in the scenario of partial packet loss, the virtual sequence number corresponding to the actual sequence number can be determined based on the frame boundary and mapping relationship of the original data included in the SVC extension header to implement data recovery operations. For example, after obtaining the SVC extension header, the data frames that need to be protected can be determined based on the frame boundary included in the SVC extension header. When the frame boundary is start = r20 and end = r24, it can be further determined that the data frames protected by the SVC extension header are from r20 to r24, that is, r20 to r40 can form a data frame. When the target transmission data obtained only includes some data frames, and the actual sequence numbers corresponding to some data frames are r20 and r21, the virtual sequence number corresponding to the actual sequence number r20 can be v6, and the virtual sequence number corresponding to the actual sequence number r21 can be v7, while other data frames are lost. In order to implement the data recovery operation, it can be deduced based on the above SVC extension header and mapping relationship that the virtual sequence number corresponding to the actual sequence number r22 can be v8, the virtual sequence number corresponding to the actual sequence number r23 can be v9, and the virtual sequence number corresponding to the actual sequence number r24 can be v10, thus effectively realizing the accuracy and reliability of determining the virtual sequence number. After determining the virtual sequence number, the data recovery operation can be performed based on the virtual sequence number.

[0186] In a scenario where all corresponding frames are lost, the frame boundaries and mapping relationships of the original data included in the SVC extension header can be utilized to endeavor to determine the virtual sequence numbers corresponding to the actual sequence numbers, so as to endeavor to implement data recovery operations. For example, the actual sequence numbers and virtual sequence numbers corresponding to the target transmission data can be respectively: virtual sequence numbers (V1, V2, V3, V4, V5......V12, V13), actual sequence numbers (r20, r21, r22, r23, r24......). During the transmission of the target transmission data, if the situation where all corresponding frames are lost occurs, for example: only the data packets corresponding to the actual sequence numbers r30 to r32 are obtained, and all the data packets located in front are lost. At this time, in order to accurately recover the data packets, the data packets corresponding to the obtained actual sequence numbers r30 to r32 can be first cached in a preset area, and then data can be recovered as much as possible based on the data packets received later. For example: after the data packets corresponding to the virtual sequence numbers V12 to V13 are received later, the actual sequence number corresponding to the above V12 is r32 or r33, and the actual sequence number corresponding to the above V13 is r34. Since the actual sequence number corresponding to the above V12 being r32 or r33 overlaps or is adjacent to the actual sequence number r32 cached in the preset area, therefore, the virtual sequence numbers corresponding to r30 and r32 can be deduced or estimated based on the frame boundaries and mapping relationships included in the SVC, that is, the virtual sequence number corresponding to r30 can be V10 or V9, the virtual sequence number corresponding to r31 can be V11 or V10, and the virtual sequence number corresponding to r32 can be V12 or V11. Thus, it is effectively realized to endeavor to determine the virtual sequence numbers corresponding to the actual sequence numbers by using the frame boundaries and mapping relationships of the original data included in the SVC extension header, so as to endeavor to implement data recovery operations.

[0187] Step S1103: Frame the target transmission data based on the virtual sequence numbers to obtain an image frame corresponding to the target transmission data.

[0188] After obtaining the virtual sequence numbers, the target transmission data can be framed based on the virtual sequence numbers, so that an image frame corresponding to the target transmission data can be obtained. In some instances, framing the target transmission data based on the virtual sequence numbers to obtain an image frame corresponding to the target transmission data may include: directly adding the virtual sequence numbers to the target transmission data to form new target transmission data, and then the new target transmission data can be framed, that is, several consecutive frames of target transmission data are framed into an image frame, thus effectively ensuring the accurate reliability of obtaining the image frame.

[0189] In some other examples, framing the target transmission data based on the virtual sequence number to obtain an image frame corresponding to the target transmission data may include: obtaining the original sequence number included in the FEC data in the target transmission data; replacing the original sequence number with the virtual sequence number to obtain adjusted data corresponding to the target transmission data; and performing framing processing based on the adjusted data to obtain an image frame corresponding to the target transmission data.

[0190] Specifically, since the target transmission data includes a pre-configured original sequence number, which corresponds to the payload data corresponding to the target transmission data, if packet loss occurs during the transmission of the target transmission data, the original sequence number may not correspond to the payload data included in the target transmission data. Therefore, after obtaining the virtual sequence number, the original sequence number can be replaced with the virtual sequence number, so as to obtain adjusted data corresponding to the target transmission data. The virtual sequence number included in the adjusted data corresponds to the payload data received by the receiving end. Therefore, it is ensured that the sequence numbers of all the data received by the receiving end are continuous, and then framing processing can be performed based on the adjusted data to obtain an image frame corresponding to the target transmission data.

[0191] The data transmission method provided in this embodiment receives the target transmission data sent by the server. When packet loss occurs in the network, it obtains the virtual sequence number corresponding to the target transmission data, and then frames the target transmission data based on the virtual sequence number, so as to obtain an image frame corresponding to the target transmission data. In this way, it effectively realizes that when packet loss occurs in the network, the sequence numbers of all the data received by the receiving end are continuous, which does not affect the framing processing of the target transmission data, and further ensures the accurate and reliable acquisition of the image frame corresponding to the target transmission data, further improving the stable and reliable use of this method, which is beneficial to the market promotion and application.

[0192] Figure 12 It is a schematic structural diagram of a data transmission device provided in an embodiment of the present application; refer to the appendix Figure 12 As shown, this embodiment provides a data transmission device, which is used to execute the above Figure 2 shown data transmission method. Specifically, the data transmission device may include:

[0193] A first acquisition module 11, configured to acquire the original data to be transmitted;

[0194] A first determination module 12, configured to determine a hierarchical coding SVC extension header for transmitting the original data and an actual sequence number corresponding to the original data;

[0195] The first processing module 13 is configured to perform forward error correction code (FEC) processing on the original data based on the SVC extension header and the actual sequence number, and generate target transmission data corresponding to the original data.

[0196] In some instances, when the first processing module 13 performs forward error correction code (FEC) processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data, the first processing module 13 is configured to execute: generate an original data RTP packet based on the original data, the SVC extension header, and the actual sequence number; perform an FEC encoding operation on the header information and payload information of the original data RTP packet to obtain FEC-encoded data; generate target transmission data corresponding to the original data based on the FEC-encoded data and the SVC extension header.

[0197] In some instances, when the first processing module 13 performs forward error correction code (FEC) processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data, the first processing module 13 is configured to execute: when the original data includes B frames, obtain a cts extension header for encoding and decoding the B frames in the original data, which is the difference time between the video display time and the decoding time; perform forward error correction code (FEC) processing on the original data based on the SVC extension header, the actual sequence number, and the cts extension header to generate target transmission data corresponding to the original data.

[0198] In some instances, when the first processing module 13 performs forward error correction code (FEC) processing on the original data based on the SVC extension header, the actual sequence number, and the cts extension header to generate target transmission data corresponding to the original data, the first processing module 13 is configured to execute: generate an original data RTP packet based on the original data, the SVC extension header, the cts extension header, and the actual sequence number; perform an FEC encoding operation on the header information and payload information of the original data RTP packet to obtain FEC-encoded data; generate target transmission data corresponding to the original data based on the FEC-encoded data, the SVC extension header, and the cts extension header.

[0199] In some instances, after generating the target transmission data corresponding to the original data, the first determination module 12 and the first processing module 13 in this embodiment are configured to execute the following steps:

[0200] The first determination module 12 is configured to determine the network operating status and scenario information corresponding to the target transmission data;

[0201] The first processing module 13 is configured to determine the operating mode corresponding to the server based on the network operating status and scenario information, and the operating mode includes any one of the following: pass-through mode, decode and pass-through mode, decode mode, encode and decode mode.

[0202] In some examples, when the first processing module 13 determines the operating mode corresponding to the server based on the network operating status and scenario information, the first processing module 13 is configured to perform: determining, based on the network operating status and scenario information, an operating mode identifier for transmitting target transmission data corresponding to the server; and determining the operating mode corresponding to the server based on the operating mode identifier.

[0203] Figure 12 The illustrated device can execute Figures 1 - 8 the method of the illustrated embodiment. For parts not described in detail in this embodiment, reference may be made to the relevant descriptions of Figures 1 - 8 the illustrated embodiment. For the execution process and technical effects of this technical solution, refer to the descriptions in Figures 1 - 8 the illustrated embodiment and will not be elaborated herein.

[0204] In a possible design, Figure 12 the structure of the illustrated data transmission device can be implemented as an electronic device, which can be various devices such as a mobile phone, a tablet computer, a personal computer (PC), etc. As Figure 13 illustrated, the electronic device may include: a first processor 21 and a first memory 22. Among them, the first memory 22 is used to store a program for the corresponding electronic device to execute the data transmission method provided in the Figures 1 - 8 illustrated embodiment, and the first processor 21 is configured to execute the program stored in the first memory 22.

[0205] The program includes one or more computer instructions. When the one or more computer instructions are executed by the first processor 21, the following steps can be implemented: obtaining the original data to be transmitted; determining a hierarchical coding SVC extension header for transmitting the original data and an actual sequence number corresponding to the original data; and performing forward error correction code (FEC) processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data.

[0206] Further, the first processor 21 is also configured to execute all or part of the steps in the Figures 1 - 8 illustrated embodiment.

[0207] Among them, the structure of the electronic device may further include a first communication interface 23 for the electronic device to communicate with other devices or communication networks.

[0208] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by an electronic device, which includes a program involved in executing the data transmission method in the Figures 1 - 8 illustrated method embodiment.

[0209] In addition, in this embodiment, a computer program product is provided, which includes: a computer program that, when executed by a processor of an electronic device, causes the processor to execute the above-mentioned Figures 1 - 8 data transmission method in the method embodiment shown.

[0210] Figure 14 FIG. 6 is a schematic structural diagram of another data transmission device provided in an embodiment of the present application; referring to the accompanying Figure 14 FIG. 6, in this embodiment, another data transmission device is provided, and this data transmission device can execute the above-mentioned Figure 9 data transmission method shown. Specifically, this data transmission device may include:

[0211] A second receiving module 31, configured to receive target transmission data sent by a sending end;

[0212] A second determining module 32, configured to determine an operating mode for transmitting the target transmission data, where the operating mode includes any one of the following: a pass-through mode, a decoded pass-through mode, a decoding mode, and an encoding and decoding mode;

[0213] A second processing module 33, configured to send the target transmission data to a receiving end based on the operating mode.

[0214] In some instances, when the second processing module 33 sends the target transmission data to the receiving end based on the operating mode, the second processing module 33 is configured to execute: when the operating mode is the pass-through mode, directly send the target transmission data to the receiving end.

[0215] In some instances, when the second processing module 33 sends the target transmission data to the receiving end based on the operating mode, the second processing module 33 is configured to execute: when the operating mode is the decoded pass-through mode, obtain first-hop data in the target transmission data; decode the first-hop data to obtain first-hop original data corresponding to the first-hop data; and send the target transmission data to the receiving end based on the first-hop original data.

[0216] In some instances, when the second processing module 33 sends the target transmission data to the receiving end based on the operating mode, the second processing module 33 is configured to execute: when the operating mode is the decoding mode, obtain first-hop data in the target transmission data and other-hop data located after the first-hop data; delete FEC data included in the other-hop data to obtain adjusted data; and send the first-hop data and the adjusted data to the receiving end.

[0217] In some examples, when the second processing module 33 sends the target transmission data to the receiving end based on the operating mode, the second processing module 33 is configured to perform: when the operating mode is the decoding mode, decode the target transmission data to obtain the original data corresponding to the target transmission data; encode the original data to obtain the encoded data; and send the encoded data to the receiving end.

[0218] In some examples, when the second processing module 33 sends the target transmission data to the receiving end based on the operating mode, the second processing module 33 is configured to perform: obtain congestion control parameters, where the congestion control parameters characterize the degree of network congestion for transmitting the target transmission data; adjust the target transmission data based on the congestion control parameters to obtain the adjusted transmission data; and send the adjusted transmission data to the receiving end based on the operating mode.

[0219] In some examples, when the second processing module 33 adjusts the target transmission data based on the congestion control parameters to obtain the adjusted transmission data, the second processing module 33 is configured to perform: obtain the priority corresponding to each data in the target transmission data; and delete some data in the target transmission data based on the priority and the congestion control parameters to obtain the adjusted transmission data.

[0220] Figure 14 The device shown can execute Figures 9 - 10 the method of the embodiment shown. For parts not described in detail in this embodiment, reference may be made to the relevant description of the Figures 9 - 10 embodiment shown. For the execution process and technical effects of this technical solution, refer to the description in the Figures 9 - 10 embodiment shown, which will not be elaborated here.

[0221] In a possible design, Figure 14 the structure of the data transmission device shown can be implemented as an electronic device, and the electronic device can be various devices such as a mobile phone, a tablet computer, a personal computer (PC), etc. As Figure 15 shown, the electronic device may include: a second processor 41 and a second memory 42. Among them, the second memory 42 is used to store a program for the corresponding electronic device to execute the data transmission method provided in the Figure 9 embodiment shown, and the second processor 41 is configured to execute the program stored in the second memory 42.

[0222] The program includes one or more computer instructions. When one or more computer instructions are executed by the second processor 41, the following steps can be implemented: receive the target transmission data sent by the sending end; determine the operating mode for transmitting the target transmission data, where the operating mode includes any one of the following: pass-through mode, decode-pass-through mode, decoding mode, encode-decode mode; and send the target transmission data to the receiving end based on the operating mode.

[0223] Further, the second processor 41 is further configured to execute all or part of the steps in the foregoing Figures 9 - 10 illustrated embodiments.

[0224] Wherein, the structure of the electronic device may further include a second communication interface 43 for the electronic device to communicate with other devices or communication networks.

[0225] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the electronic device, which includes a program involved in the data transmission method in the foregoing Figures 9 - 10 illustrated method embodiments.

[0226] In addition, this embodiment provides a computer program product, which includes: a computer program, when the computer program is executed by the processor of the electronic device, the processor is caused to execute the foregoing Figures 9 - 10 illustrated data transmission method in the method embodiments.

[0227] Figure 16 FIG. is a schematic structural diagram of a data transmission device provided by an embodiment of the present application; referring to the attached Figure 16 As shown, this embodiment provides a data transmission device, which can execute the foregoing Figure 11 illustrated data transmission method. Specifically, the data transmission device may include:

[0228] A third receiving module 51, configured to receive target transmission data sent by the server, where the target transmission data includes at least one of the following: FEC data, raw data;

[0229] A third obtaining module 52, configured to obtain a virtual sequence number corresponding to the target transmission data based on the SVC extension header included in the target transmission data when a packet loss occurs in the network, where the virtual sequence number is used to make the sequence numbers of all received data continuous, and wherein the SVC extension header includes the frame boundary of the raw data;

[0230] A third processing module 53, configured to frame the target transmission data based on the virtual sequence number to obtain an image frame corresponding to the target transmission data.

[0231] In some instances, when the third obtaining module 52 obtains the virtual sequence number corresponding to the target transmission data based on the SVC extension header included in the target transmission data, the third obtaining module 52 is configured to execute: obtaining an actual sequence number corresponding to the target transmission data; determining a mapping relationship between the virtual sequence number and the actual sequence number; and determining the virtual sequence number corresponding to the actual sequence number based on the mapping relationship and the frame boundary of the raw data included in the SVC extension header.

[0232] In some examples, when the third processing module 53 frames the target transmission data based on the virtual sequence number to obtain an image frame corresponding to the target transmission data, the third processing module 53 is configured to perform: obtaining the original sequence number included in the FEC data in the target transmission data; replacing the original sequence number with the virtual sequence number to obtain adjusted data corresponding to the target transmission data; and performing framing processing based on the adjusted data to obtain an image frame corresponding to the target transmission data.

[0233] Figure 16 The device shown can execute Figure 11 the method of the embodiment shown. For parts not described in detail in this embodiment, reference can be made to the relevant description of the Figure 11 embodiment shown. For the execution process and technical effects of this technical solution, refer to the description in the Figure 11 embodiment shown, which will not be elaborated here.

[0234] In a possible design, Figure 16 the structure of the data transmission device shown can be implemented as an electronic device, which can be various devices such as a mobile phone, a tablet computer, a personal computer (PC), etc. As Figure 17 shown, the electronic device can include: a third processor 61 and a third memory 62. Among them, the third memory 62 is used to store a program for the corresponding electronic device to execute the data transmission method provided in the Figure 11 embodiment shown, and the third processor 61 is configured to execute the program stored in the third memory 62.

[0235] The program includes one or more computer instructions. When one or more computer instructions are executed by the third processor 61, the following steps can be implemented: receiving target transmission data sent by a server, where the target transmission data includes at least one of the following: FEC data, original data; when packet loss occurs in the network, obtaining a virtual sequence number corresponding to the target transmission data based on the SVC extension header included in the target transmission data, where the virtual sequence number is used to make the sequence numbers of all received data continuous, and the SVC extension header includes the frame boundary of the original data; and framing the target transmission data based on the virtual sequence number to obtain an image frame corresponding to the target transmission data.

[0236] Further, the third processor 61 is further configured to execute all or part of the steps in the Figure 11 embodiment shown.

[0237] Among them, the structure of the electronic device may further include a third communication interface 63 for the electronic device to communicate with other devices or communication networks.

[0238] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by an electronic device, which includes a program for executing the data transmission method involved in the method embodiment shown above. Figure 11 The program involved in the data transmission method in the method embodiment shown above.

[0239] Furthermore, this embodiment provides a computer program product, which includes: a computer program that, when executed by a processor of an electronic device, causes the processor to execute the data transmission method in the method embodiment shown above. Figure 11 The data transmission method in the method embodiment shown above.

[0240] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0241] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of adding a necessary general hardware platform, and of course, it can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a computer product. This application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0242] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general computer, a special computer, an embedded processor, or other programmable devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable devices generate a device for implementing the functions specified in one Figure 1 One process or multiple processes and / or blocks Figure 1 One block or multiple blocks.

[0243] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable device to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one process Figure 1 or more processes and / or boxes Figure 1 or more boxes specified in the box.

[0244] These computer program instructions can also be loaded onto a computer or other programmable device, such that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 or more processes and / or boxes Figure 1 or more boxes specified in the box.

[0245] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0246] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

[0247] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0248] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data transmission method, characterized in that, including: obtaining the original data to be transmitted; determining a hierarchical coding SVC extension header for transmitting the original data and an actual sequence number corresponding to the original data; performing forward error correction code FEC processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data; after generating the target transmission data corresponding to the original data, the method further includes: determining the network operating state and scenario information corresponding to the target transmission data; determining an operating mode corresponding to the server based on the network operating state and the scenario information, where the operating mode includes any one of the following: pass-through mode, decode and pass-through mode, decode mode, encode and decode mode.

2. The method according to claim 1, wherein performing forward error correction code FEC processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data, including: generating an original data RTP packet based on the original data, the SVC extension header, and the actual sequence number; performing an FEC encoding operation on the header information and payload information of the original data RTP packet to obtain FEC-encoded data; generating target transmission data corresponding to the original data based on the FEC-encoded data and the SVC extension header.

3. The method according to claim 1, wherein performing forward error correction code FEC processing on the original data based on the SVC extension header and the actual sequence number to generate target transmission data corresponding to the original data, including: when the original data includes B frames, obtaining a difference time cts extension header for encoding and decoding the B frames in the original data; performing forward error correction code FEC processing on the original data based on the SVC extension header, the actual sequence number, and the cts extension header to generate target transmission data corresponding to the original data.

4. The method according to claim 3, wherein performing forward error correction code FEC processing on the original data based on the SVC extension header, the actual sequence number, and the cts extension header to generate target transmission data corresponding to the original data, including: generating an original data RTP packet based on the original data, the SVC extension header, the cts extension header, and the actual sequence number; performing an FEC encoding operation on the header information and payload information of the original data RTP packet to obtain FEC-encoded data; generating target transmission data corresponding to the original data based on the FEC-encoded data, the SVC extension header, and the cts extension header.

5. The method according to claim 1, wherein determining an operating mode corresponding to the server based on the network operating state and the scenario information, including: determining an operating mode identifier corresponding to the server for transmitting the target transmission data based on the network operating state and the scenario information; determining the operating mode corresponding to the server based on the operating mode identifier.

6. A data transmission method, characterized in that, including: Receive the target transmission data sent by the sending end. The target transmission data is generated by performing forward error correction code (FEC) processing on the original data based on a hierarchical coding SVC extension header and the actual sequence number corresponding to the original data. The SVC extension header is used to transmit the original data. Determine an operating mode for transmitting the target transmission data based on the network operating status and scenario information corresponding to the target transmission data. The operating mode includes any one of the following: pass-through mode, decode and pass-through mode, decode mode, encode and decode mode. Send the target transmission data to the receiving end based on the operating mode.

7. The method according to claim 6, wherein Sending the target transmission data to the receiving end based on the operating mode includes: When the operating mode is the decode and pass-through mode, obtain the first-hop data in the target transmission data. Decode the first-hop data to obtain the first-hop original data corresponding to the first-hop data. Send the target transmission data to the receiving end based on the first-hop original data.

8. The method according to claim 6, wherein Sending the target transmission data to the receiving end based on the operating mode includes: When the operating mode is the decode mode, obtain the first-hop data and other-hop data after the first-hop data in the target transmission data. Delete the FEC data included in the other-hop data to obtain adjusted data. Send the first-hop data and the adjusted data to the receiving end.

9. The method according to claim 6, characterized in that Sending the target transmission data to the receiving end based on the operating mode includes: When the operating mode is the decode mode, perform decoding processing on the target transmission data to obtain the original data corresponding to the target transmission data. Encode the original data to obtain encoded data. Send the encoded data to the receiving end.

10. The method according to claim 6, wherein Sending the target transmission data to the receiving end based on the operating mode includes: Obtain congestion control parameters, where the congestion control parameters characterize the degree of network congestion for transmitting the target transmission data. Adjust the target transmission data based on the congestion control parameters to obtain adjusted transmission data. Send the adjusted transmission data to the receiving end based on the operating mode.

11. A data transmission method, characterized in that, Includes: The receiving server receives the target transmission data sent based on the operating mode. The target transmission data includes at least one of the following: FEC data, original data. The operating mode is determined based on the network operating status and scenario information corresponding to the target transmission data. The operating mode includes any one of the following: pass-through mode, decode and pass-through mode, decode mode, encode and decode mode. When packet loss occurs in the network, obtain the virtual sequence number corresponding to the target transmission data based on the SVC extension header included in the target transmission data. The virtual sequence number is used to make the sequence numbers of all received data continuous. Among them, the frame boundary of the original data is included in the SVC extension header. Frame the target transmission data based on the virtual sequence number to obtain an image frame corresponding to the target transmission data.

Images