A srt streaming upload, download hardware acceleration method and programmable chip

Abstract

The application provides an SRT stream uploading and downloading hardware acceleration method and a programmable chip.The uploading method comprises the following steps: obtaining video stream data, encapsulating the video stream data into TS stream packets, and storing the TS stream packets in a cache; judging whether an acknowledgement character arranged according to a serial number is received in real time; under the premise that the acknowledgement character is received, obtaining the corresponding stream packet from the cache; encrypting the stream packet to obtain an encrypted stream packet, and then uploading the encrypted stream packet to a cloud end.The downloading method comprises the following steps: obtaining the encrypted stream packet from the cloud server, analyzing the SRT protocol, decrypting the stream packet, storing the decrypted stream packet in the cache, feeding back the acknowledgement character to the cloud end, obtaining the decrypted stream packet from the cache, and performing TS disassembly to obtain the video stream data.The application further provides a programmable chip.The application realizes the uploading and downloading of the SRT stream in a hardware mode, reduces power consumption and CPU usage, and reduces the delay caused by encryption and decryption.

Description

Technical Field

[0001] The present invention belongs to the field of video transmission technology, and in particular relates to an SRT stream uploading and downloading hardware acceleration method and a programmable chip. Background Art

[0002] Online video streaming is booming, and streaming protocols are crucial in this application. Major vendors and consortiums have developed numerous different protocols, including RTMP, HLS, and SRT, each with its own advantages and disadvantages. RTMP (Real-Time Messaging Protocol), also known as the Instant Messaging Protocol, is a proprietary protocol originally developed by Macromedia for transmitting streaming audio, video, and data between Flash players and a server over international networks. The RTMP transport protocol is a mature and comprehensive streaming protocol. Originally developed by Adobe for use on the Internet, RTMP is based on TCP due to the inherent instability of network connections. Although RTMP is currently the most popular streaming protocol, it has not been updated since 2012. Many believe that SRT may replace RTMP in the future. Because SRT is a UDP-based streaming protocol, it is two to three times faster than RTMP.

[0003] SRT: (Secure Reliable Transport) is the abbreviation of Secure Reliable Transport Protocol, a streaming protocol developed by Haivision. SRT is an audio and video streaming transmission protocol that mainly connects two endpoints with the purpose of delivering low-latency audio and video streaming over the unstable public Internet. It can solve problems such as packet loss, jitter and bandwidth fluctuations, while maintaining the integrity and image quality of the audio and video. With SRT, the stream can be kept secure and easily pass through firewalls. Although SRT can solve packet loss and encryption security functions, this greatly increases the computing load, especially encryption operations (through AES256 operations), thereby increasing the CPU load, power consumption load and the delay caused by the increased computing power. The SRT technology on the market is still developed by software. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a hardware acceleration method for uploading and downloading SRT streams, which realizes the uploading and downloading of SRT streams through hardware, reduces power consumption and CPU usage, and reduces the delay caused by encryption and decryption calculations.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention proposes an SRT stream upload acceleration method, which runs on a programmable chip and includes the following steps:

[0007] Obtain video streaming data, encapsulate the video streaming data into TS stream packets and put them into the cache;

[0008] Real-time determination of whether the TS stream packet confirmation character arranged in sequence is received in the buffer, and obtaining the corresponding TS stream packet from the buffer if the confirmation character is received;

[0009] After encrypting the TS stream packet, an encrypted stream packet is obtained, and the encrypted stream packet is uploaded to the cloud server.

[0010] Furthermore, the method for obtaining video streaming data includes: receiving the video streaming data through a PCIE transmission interface on a programmable chip.

[0011] Furthermore, after encapsulating the video stream data into TS stream packets and placing them into the cache, the method further includes arranging the video stream data according to sequence numbers.

[0012] Furthermore, the process of uploading the encrypted streaming packet to the cloud server includes:

[0013] The encrypted stream packet is re-encapsulated into an SRT packet, and the SRT packet is converted into a UDP packet, and then uploaded to the cloud server through the UDP transmission port of the programmable chip.

[0014] The present invention proposes an SRT streaming download acceleration method, which is characterized by running on a programmable chip and comprising:

[0015] After receiving the encrypted streaming packet from the cloud server, it parses the SRT protocol, decrypts the streaming packet and stores it in the cache, while also sending a confirmation string back to the cloud server.

[0016] The decrypted streaming packet is obtained from the cache and then TS is disassembled to obtain the video streaming data.

[0017] Furthermore, the encrypted streaming packet is obtained from the cloud server and then the SRT protocol is parsed:

[0018] Obtaining an encrypted stream packet in the form of a UDP packet through a UDP port of a programmable chip; then converting the encrypted stream packet in the form of a UDP packet into an encrypted stream packet in the form of an SRT packet;

[0019] Decrypt the encrypted stream packets after parsing the SRT protocol.

[0020] Furthermore, the process of decrypting the stream packet and storing it in the cache and simultaneously feeding back the confirmation character to the cloud server includes:

[0021] The decrypted stream packets are arranged in sequence and the confirmation characters arranged in sequence are fed back to the cloud server.

[0022] Furthermore, the method further includes obtaining the decrypted stream packet from the cache, performing TS decomposition, and transmitting the video stream data through the PCIE system.

[0023] The present invention also proposes a programmable chip, comprising a TS encapsulation unit, an SRT unit and a cache unit; the TS encapsulation unit is communicatively connected to the cache unit; the cache unit is communicatively connected to the SRT unit;

[0024] The TS encapsulation unit is used to encapsulate the video stream data into TS stream packets, or to decompose the TS stream packets into video stream data.

[0025] The SRT unit is used to implement ARQ packet loss verification, encryption and decryption of video streaming data, as well as encapsulation and parsing of SRT protocols;

[0026] The cache unit temporarily stores the video stream data after arranging them in sequence.

[0027] Furthermore, the chip also includes a PCIE transmission interface and a UDP transmission interface;

[0028] The PCIE transmission interface is used to receive or send video streaming data according to the PCIE protocol;

[0029] The UDP transmission interface is used to receive video streaming data from a cloud server or upload video streaming data to a cloud server according to the UDP protocol.

[0030] The effects provided in the summary of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0031] The present invention proposes a hardware acceleration method for SRT stream upload and download and a programmable chip. This acceleration method runs on a programmable chip. The upload method specifically includes: obtaining video stream data, encapsulating the video stream data into TS stream packets, and storing them in a cache; determining in real time whether the cache has received a serially numbered confirmation character for the TS stream packet; and upon receipt of the confirmation character, retrieving the corresponding TS stream packet from the cache; encrypting the TS stream packet to obtain an encrypted stream packet, and uploading the encrypted stream packet to a cloud server. The download method specifically includes: obtaining the encrypted stream packet from the cloud server, parsing the SRT protocol, decrypting the stream packet, storing it in a cache, and simultaneously feeding back a confirmation character to the cloud server; obtaining the decrypted stream packet from the cache, and performing TS decomposition to obtain the video stream data. Based on this hardware acceleration method for SRT stream upload and download, the present invention also proposes a programmable chip. The present invention implements SRT stream upload and download through hardware, reducing power consumption and CPU usage, and reducing delays caused by encryption and decryption calculations.

[0032] The present invention adds an ARQ packet loss retransmission mechanism and high hardware time sensitivity during implementation, thereby improving the ability to combat packet loss and delay.

[0033] The present invention adopts TS packet processing and SRT protocol processing during the implementation process. Both TS packet processing and SRT protocol processing have high-speed processing characteristics and are suitable for multi-channel 8K high-transmission rate packet processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] like Figure 1 This is a schematic diagram of the structural connection of an SRT streaming upload according to embodiment 1 of the present invention;

[0035] like Figure 2 This is a flowchart of a method for uploading SRT streams according to embodiment 1 of the present invention;

[0036] like Figure 3 Schematic diagram of the ARQ packet loss retransmission mechanism in Example 1 of the present invention;

[0037] like Figure 4 1 is a flow chart of the preferred states of the sending end and the receiving end buffer in embodiment 1 of the present invention;

[0038] like Figure 5 This is a flowchart of the poor state of the sending end and the receiving end buffer in Example 1 of the present invention;

[0039] like Figure 6 This is a schematic diagram of the structural connection of an SRT streaming download according to embodiment 2 of the present invention;

[0040] like Figure 7This is a flowchart of a method for SRT streaming downloading according to embodiment 2 of the present invention;

[0041] like Figure 8 This is a schematic diagram of the structure of a programmable chip according to embodiment 3 of the present invention. DETAILED DESCRIPTION

[0042] The present disclosure will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the relevant content and are not intended to limit the present disclosure. It should also be noted that, for ease of description, only the portions relevant to the present disclosure are shown in the accompanying drawings.

[0043] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] Unless otherwise stated, the exemplary embodiments / examples shown are to be understood as providing exemplary features of various details of some ways in which the technical concepts of the present disclosure can be implemented in practice. Therefore, unless otherwise stated, the features of the various embodiments / examples may be further combined, separated, interchanged, and / or rearranged without departing from the technical concepts of the present disclosure.

[0045] The use of cross hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise indicated, the presence or absence of cross hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the order described. In addition, the same figure numbers represent the same components.

[0046] When a component is referred to as being “on,” “over,” “connected to,” or “coupled to” another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being “directly on,” “directly connected to,” or “directly coupled to” another component, there are no intervening components present. For this purpose, the term “connected” may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0047] The terms used herein are for the purpose of describing specific embodiments and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, the features, integral bodies, steps, operations, parts, assemblies and / or their groups stated are indicated, but the presence or addition of one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups is not excluded. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, and as such, they are used to explain the inherent deviations of the measured values, calculated values ​​and / or values ​​provided that will be recognized by those of ordinary skill in the art.

[0048] Example 1

[0049] Embodiment 1 of the present invention proposes an SRT stream upload acceleration method, which runs on a programmable chip and includes the following steps:

[0050] Obtain video streaming data, encapsulate the video streaming data into TS stream packets and put them into the cache;

[0051] Real-time determination of whether the TS stream packet confirmation character arranged in sequence is received in the buffer, and obtaining the corresponding TS stream packet from the buffer if the confirmation character is received;

[0052] After encrypting the TS stream packet, an encrypted stream packet is obtained, and the encrypted stream packet is uploaded to the cloud server.

[0053] The method for obtaining video stream data of the present invention comprises: receiving the video stream data through a PCIE transmission interface on a programmable chip.

[0054] The received video stream data is first cut and then encapsulated into TS stream packets and placed in the cache. The video stream data is also arranged according to the sequence number. The size of each TS stream packet is 188 bytes.

[0055] The process of uploading the encrypted streaming packet to the cloud server includes: re-encapsulating the encrypted streaming packet into an SRT packet, converting the SRT packet into a UDP packet, and then uploading it to the cloud server through the UDP transmission port of the programmable chip.

[0056] ARQ (Automatic Repeat-reQuest) is an error-correction protocol used in the data link and transport layers of the OSI model. It uses acknowledgments and timeouts to achieve reliable information transmission over unreliable services. If the sender does not receive an acknowledgment frame within a certain period of time after sending, it typically retransmits the frame. ARQ can include stop-and-wait and continuous ARQ protocols, as well as error detection, positive acknowledgment, retransmission after timeout, and negative acknowledgment and retransmission mechanisms.

[0057] The ARQ mechanism requires a retransmission buffer on the sender side, and a buffer on the receiver side to store packets sequentially. Meanwhile, the SRT protocol specifies the maximum available capacity of both the send and receive buffers by setting latency.

[0058] like Figure 3 Schematic diagram of the ARQ packet loss retransmission mechanism in Example 1 of the present invention;

[0059] After processing, the TS encapsulation unit adds a sequence number to each packet. After receiving the packet, the buffer unit responds with an ACK to the video data uploader based on the sequence number. For example, if packet 3 is lost, then the buffer unit receives packet 2 followed by packet 4. Therefore, the ACKs the buffer unit responds with will be ack1, ack2, ack4, and ack5 in order. Therefore, the video data uploader will not receive ack3, and thus will know that packet 3 is lost, so it will retransmit packet 3.

[0060] The send buffer is used to store packets that may need to be retransmitted, that is, packets that have not yet received a positive acknowledgment (ACK). If the sender receives a positive acknowledgment for a packet, the packet will be kicked out of the sender's buffer. If no reply ACK signal is received, the data cannot be stored in the sender's buffer forever. The SRT protocol stipulates that the maximum storage time is the delay amount.

[0061] like Figure 4This is a flowchart of the optimal state of the sender and receiver buffers in Example 1 of the present invention. Imagine the delay as a window sliding from left to right on both the sender and receiver. In the figure, packets 1 through 6 have all received positive acknowledgments (ACKs) and have been removed from the sender's buffer. Packet 7 remains in the buffer awaiting a response from the receiver. This is an optimal transmission state, with minimal usage of the sender's buffer and most data being received promptly and ACKed.

[0062] like Figure 5 This is a flowchart of the poor state of the sender and receiver buffers in Example 1 of the present invention; the receiver did not receive data packet No. 3 and replied with a negative acknowledgment (NAK) to the sender. Due to various reasons, this process was delayed, resulting in data packet No. 3 being at the far left of the window. As the window slides, it will be kicked out of the sender buffer in the next step. In this case, if the retransmission of data packet No. 3 is lost or delayed again, the data packet will be lost, and the image at the decoding end may also have problems. At the same time, we also notice that the sender buffer in the figure is already full, which also indicates a transmission state at the critical point of packet loss. Therefore, the less the sender buffer is occupied, the safer the link. If the sender buffer is full, the link is likely to lose packets.

[0063] The function of the receive buffer is to sort the received data packets (the SRT protocol records the precise timestamp in the SRT packet header). On the one hand, sorting is required for decoding, and on the other hand, it is to find out the data packets that did not arrive in time, return a negative acknowledgment (NAK) to the sender, and wait for retransmission. Figure 4 Data packet No. 1 at the receiving end has been sent to the decoder, and data packet No. 2 will also be sent to the decoder as the delay window slides. At the same time, data packets No. 2 to No. 6 have been saved in the receiving buffer. The receiving buffer is almost filled with data packets, which is a good link status. Figure 5 This indicates a poor link state. The receiver did not receive packet 3, and packet 3 is located at the far left of the receiver window, meaning it was likely skipped, resulting in issues with the decoded image. In this case, the low number of packets in the receive buffer may be due to packet loss or a mismatch between network bandwidth and video bitrate.

[0064] Therefore, the ideal state of the receive buffer is that its usage should be slightly lower than the delay amount. If the receive buffer occupancy becomes zero, problems are likely to occur in decoding the image.

[0065] This method is based on an SRT streaming upload structure, such as Figure 1A schematic diagram of the structure connection of an SRT streaming upload in embodiment 1 of the present invention is provided. It includes: a PCIE transmission interface, a TS encapsulation unit, an SRT unit, a cache unit, and a UDP transmission interface;

[0066] The PCIE transmission interface is used to receive video streaming data according to the PCIE protocol; after receiving the video streaming data, the PCIE transmission interface sends it to the TS encapsulation unit; the TS encapsulation unit is used to encapsulate the video streaming data into TS stream packets; the TS encapsulation unit is communicated with the cache unit; the cache unit temporarily stores the video streaming data after arranging them by sequence number; the cache unit is communicated with the SRT unit; the SRT unit is used to implement AES data encryption and ARQ packet loss retransmission mechanism processing; finally, the encrypted video streaming data encapsulated in the SRT protocol is uploaded to the cloud server through the UDP transmission interface.

[0067] In this application, the PCIE transmission interface, TS encapsulation unit, SRT unit, cache unit and UDP transmission interface are all located on the programmable chip.

[0068] The programmable chip in this application adopts a programmable logic chip FPGA. The scope of protection of this application is not limited to the chips listed in Example 1, and other programmable chips can be used, such as CPLD, single-chip microcomputer, etc.

[0069] In this application: TS: is a standard digital packaging format used to transmit and store video, audio, channel, and program information, and is used in digital television broadcasting systems.

[0070] like Figure 2 This is a flowchart of a method for uploading SRT streams according to embodiment 1 of the present invention;

[0071] In step S200, video stream data is received via the PCIE transmission interface on the programmable chip;

[0072] In step S210, the video stream data is encapsulated into TS stream packets and then placed in the buffer; the video file received by the DMA is cut and encapsulated into TS stream packets, each TS stream packet is 188 bytes, and then the data is transferred to the buffer.

[0073] In step S220, it is determined in real time whether the buffer has received the acknowledgment character of the TS stream packet arranged in sequence. If the acknowledgment character is received, the corresponding TS stream packet is retrieved from the buffer. When each stream packet is transmitted to the buffer module, the buffer module returns an ACK, indicating that the stream packet has been received. If the ACK is not received, it means that the stream packet was lost during transmission. Therefore, it is determined in real time whether the ACK is received in sequence.

[0074] In step S230, the TS stream packet is encrypted using AES, with either 128-bit or 256-bit AES encryption being selected. AES (Advanced Encryption Standard), also known as Rijndael in cryptography, is a block encryption standard adopted by the U.S. federal government. This standard, designed to replace the original DES, has been extensively analyzed and is widely used worldwide. After a five-year selection process, the Advanced Encryption Standard was published by the National Institute of Standards and Technology (NIST) in FIPS PUB 197 on November 26, 2001, and became an effective standard on May 26, 2002. By 2006, the Advanced Encryption Standard had become one of the most popular algorithms for symmetric key encryption.

[0075] In step S240, a packet sequence number is added and the SRT protocol is encapsulated.

[0076] In step S250 , the SRT packet is converted into a UDP packet, which is packaged into a UDP packet with 7 TS stream packets, and then uploaded to the cloud server through the UDP transmission port of the programmable chip.

[0077] The SRT stream upload acceleration method proposed in Example 1 of the present invention implements the upload of SRT streams through hardware, thereby reducing power consumption and CPU usage, and reducing the delay caused by encryption and decryption calculations.

[0078] The SRT stream upload acceleration method proposed in Example 1 of the present invention adds an ARQ packet loss retransmission mechanism and high hardware time sensitivity during implementation, thereby improving the ability to combat packet loss and delay.

[0079] The first embodiment of the present invention proposes an SRT streaming upload acceleration method, which uses TS packet processing and SRT protocol processing during the implementation process. Both TS packet processing and SRT protocol processing have high-speed processing characteristics and are suitable for multi-channel 8K high-transmission rate packet processing.

[0080] Example 2

[0081] Embodiment 2 of the present invention proposes an SRT streaming download acceleration method, which runs on a programmable chip. The method specifically includes:

[0082] After receiving the encrypted streaming packet from the cloud server, it parses the SRT protocol, decrypts the streaming packet and stores it in the cache, while also sending a confirmation string back to the cloud server.

[0083] The decrypted streaming packet is obtained from the cache and then TS is disassembled to obtain the video streaming data.

[0084] Obtain the encrypted streaming packet from the cloud server and parse the SRT protocol:

[0085] Obtaining an encrypted stream packet in the form of a UDP packet through a UDP port of a programmable chip; then converting the encrypted stream packet in the form of a UDP packet into an encrypted stream packet in the form of an SRT packet;

[0086] Decrypt the encrypted stream packets after parsing the SRT protocol.

[0087] The process of decrypting the stream packets and putting them into the cache and feeding back the confirmation characters to the cloud server includes: arranging the decrypted stream packets according to the sequence numbers and feeding back the confirmation characters arranged according to the sequence numbers to the cloud server.

[0088] The method further includes obtaining the decrypted stream packet from the cache, performing TS decomposition, and transmitting the video stream data through the PCIE system.

[0089] This method is based on an SRT streaming download structure, such as Figure 6 This is a schematic diagram of the structural connection of an SRT streaming download according to embodiment 2 of the present invention; it includes: a PCIE transmission interface, a TS encapsulation unit, an SRT unit, a cache unit, and a UDP transmission interface;

[0090] The UDP transmission interface is used to obtain encrypted streaming packets from the cloud server; after the encrypted streaming packets are obtained by the UDP transmission interface, they are sent to the SRT unit; the SRT unit is used to implement encrypted streaming packet decryption and ARQ packet loss retransmission mechanism processing; the cache unit is communicated with the SRT unit, and the cache unit temporarily stores the decrypted streaming packets after arranging them by sequence number; the TS encapsulation unit is used to obtain the decrypted streaming packets from the cache and then perform TS disassembly to obtain video streaming data; the PCIE transmission interface is used to transmit the video streaming data obtained by TS disassembly.

[0091] ARQ (Automatic Repeat-reQuest) is an error-correction protocol used in the data link and transport layers of the OSI model. It uses acknowledgments and timeouts to achieve reliable information transmission over unreliable services. If the sender does not receive an acknowledgment frame within a certain period of time after sending, it typically retransmits the frame. ARQ can include stop-and-wait and continuous ARQ protocols, as well as error detection, positive acknowledgment, retransmission after timeout, and negative acknowledgment and retransmission mechanisms.

[0092] like Figure 3 Schematic diagram of the ARQ packet loss retransmission mechanism in Example 1 of the present invention;

[0093] After processing, the TS encapsulation unit adds a sequence number to each packet. After receiving the packet, the buffer unit responds with an ACK to the video data uploader based on the sequence number. For example, if packet 3 is lost, then the buffer unit receives packet 2 followed by packet 4. Therefore, the ACKs the buffer unit responds with will be ack1, ack2, ack4, and ack5 in order. Therefore, the video data uploader will not receive ack3, and thus will know that packet 3 is lost, so it will retransmit packet 3.

[0094] The send buffer is used to store packets that may need to be retransmitted, that is, packets that have not yet received a positive acknowledgment (ACK). If the sender receives a positive acknowledgment for a packet, the packet will be kicked out of the sender's buffer. If no reply ACK signal is received, the data cannot be stored in the sender's buffer forever. The SRT protocol stipulates that the maximum storage time is the delay amount.

[0095] like Figure 4 This is a flowchart of the optimal state of the sender and receiver buffers in Example 1 of the present invention. Imagine the delay as a window sliding from left to right on both the sender and receiver. In the figure, packets 1 through 6 have all received positive acknowledgments (ACKs) and have been removed from the sender's buffer. Packet 7 remains in the buffer awaiting a response from the receiver. This is an optimal transmission state, with minimal usage of the sender's buffer and most data being received promptly and ACKed.

[0096] like Figure 5This is a flowchart of the poor state of the sender and receiver buffers in Example 1 of the present invention; the receiver did not receive data packet No. 3 and replied with a negative acknowledgment (NAK) to the sender. Due to various reasons, this process was delayed, resulting in data packet No. 3 being at the far left of the window. As the window slides, it will be kicked out of the sender buffer in the next step. In this case, if the retransmission of data packet No. 3 is lost or delayed again, the data packet will be lost, and the image at the decoding end may also have problems. At the same time, we also notice that the sender buffer in the figure is already full, which also indicates a transmission state at the critical point of packet loss. Therefore, the less the sender buffer is occupied, the safer the link. If the sender buffer is full, the link is likely to lose packets.

[0097] The function of the receive buffer is to sort the received data packets (the SRT protocol records the precise timestamp in the SRT packet header). On the one hand, sorting is required for decoding, and on the other hand, it is to find out the data packets that did not arrive in time, return a negative acknowledgment (NAK) to the sender, and wait for retransmission. Figure 4 Data packet No. 1 at the receiving end has been sent to the decoder, and data packet No. 2 will also be sent to the decoder as the delay window slides. At the same time, data packets No. 2 to No. 6 have been saved in the receiving buffer. The receiving buffer is almost filled with data packets, which is a good link status. Figure 5 This indicates a poor link state. The receiver did not receive packet 3, and packet 3 is located at the far left of the receiver window, meaning it was likely skipped, resulting in issues with the decoded image. In this case, the low number of packets in the receive buffer may be due to packet loss or a mismatch between network bandwidth and video bitrate.

[0098] Therefore, the ideal state of the receive buffer is that its usage should be slightly lower than the delay amount. If the receive buffer occupancy becomes zero, problems are likely to occur in decoding the image.

[0099] In this application, the PCIE transmission interface, TS encapsulation unit, SRT unit, cache unit and UDP transmission interface are all located on the programmable chip.

[0100] The programmable chip in this application adopts a programmable logic chip FPGA. The scope of protection of this application is not limited to the chips listed in Example 2, and other programmable chips can be used, such as CPLD, single-chip microcomputer, etc.

[0101] like Figure 7 This is a flowchart of a method for SRT streaming downloading according to embodiment 2 of the present invention;

[0102] In step S700 , the encrypted streaming packet is obtained from the cloud server and the UDP protocol is decoded.

[0103] In step S710 , the SRT protocol is decoded.

[0104] In step S720, the packet is decrypted using AES, which can be AES128 or AES256 according to the setting information.

[0105] In step S730, the sequence number of the packet is read, and the data is stored in the corresponding buffer location according to the sequence number, and an ACK is sent back to the cloud server.

[0106] In step S740 , the TS stream is disassembled to obtain video data.

[0107] In step S750, the acquired video data is downloaded via the PCIE transmission interface.

[0108] The SRT stream download acceleration method proposed in Example 2 of the present invention implements the upload of SRT streams through hardware, thereby reducing power consumption and CPU usage, and reducing the delay caused by encryption and decryption calculations.

[0109] The SRT streaming download acceleration method proposed in Example 2 of the present invention adds an ARQ packet loss retransmission mechanism and high hardware time sensitivity during implementation, thereby improving the ability to combat packet loss and delay.

[0110] An SRT streaming download acceleration method proposed in Example 2 of the present invention adopts TS packet processing and SRT protocol processing during the implementation process. Both TS packet processing and SRT protocol processing have high-speed processing characteristics and are suitable for multi-channel 8K high-transmission rate packet processing.

[0111] Example 3

[0112] Based on the SRT stream upload acceleration method proposed in Example 1 of the present invention and the SRT stream download acceleration method proposed in Example 2 of the present invention, Example 3 of the present invention further proposes a programmable chip, such as Figure 8 This is a schematic diagram of the structure of a programmable chip according to embodiment 3 of the present invention.

[0113] The chip includes: a TS encapsulation unit, an SRT unit and a cache unit; the TS encapsulation unit is communicatively connected to the cache unit; the cache unit is communicatively connected to the SRT unit;

[0114] The TS encapsulation unit is used to encapsulate the video stream data into TS stream packets, or to decompress the TS stream packets into video stream data.

[0115] The SRT unit is used to implement ARQ packet loss verification, encryption and decryption of video streaming data, as well as encapsulation and parsing of the SRT protocol;

[0116] The cache unit temporarily stores the video stream data after arranging them in sequence.

[0117] The chip also includes PCIE transmission interface and UDP transmission interface;

[0118] The PCIE transmission interface is used to receive or send video streaming data according to the PCIE protocol;

[0119] The UDP transmission interface is used to receive video streaming data from a cloud server or upload video streaming data to a cloud server according to the UDP protocol.

[0120] A programmable chip provided in Example 3 of the present application is used to implement an SRT stream upload acceleration method in Example 1 of the present invention and an SRT stream download acceleration method in Example 2 of the present invention.

[0121] In the process of implementing an SRT streaming upload acceleration method: the functions of each unit inside a programmable chip are: a PCIE transmission interface is used to receive video streaming data according to the PCIE protocol; the PCIE transmission interface sends the video streaming data to the TS encapsulation unit after receiving it; the TS encapsulation unit is used to encapsulate the video streaming data into TS streaming packets; the TS encapsulation unit is communicated with the cache unit; the cache unit temporarily stores the video streaming data after arranging them according to sequence numbers; the cache unit is communicated with the SRT unit; the SRT unit is used to implement AES data encryption and ARQ packet loss retransmission mechanism processing; finally, the encrypted video streaming data encapsulated as the SRT protocol is uploaded to the cloud server through the UDP transmission interface.

[0122] A programmable chip incorporates Automatic Repeat-reQuest (ARQ), an error-correction protocol used in the data link and transport layers of the OSI model. It uses acknowledgments and timeouts to achieve reliable information transmission over unreliable services. If the sender does not receive an acknowledgment frame within a certain period of time after sending, it typically retransmits. ARQ can include stop-and-wait and continuous ARQ protocols, as well as error detection, positive acknowledgment, retransmission after timeout, and negative acknowledgment and retransmission mechanisms.

[0123] like Figure 3 Schematic diagram of the ARQ packet loss retransmission mechanism in Example 1 of the present invention;

[0124] After processing, the TS encapsulation unit adds a sequence number to each packet. After receiving the packet, the buffer unit responds with an ACK to the video data uploader based on the sequence number. For example, if packet 3 is lost, then the buffer unit receives packet 2 followed by packet 4. Therefore, the ACKs the buffer unit responds with will be ack1, ack2, ack4, and ack5 in order. Therefore, the video data uploader will not receive ack3, and thus will know that packet 3 is lost, so it will retransmit packet 3.

[0125] The send buffer is used to store packets that may need to be retransmitted, that is, packets that have not yet received a positive acknowledgment (ACK). If the sender receives a positive acknowledgment for a packet, the packet will be kicked out of the sender's buffer. If no reply ACK signal is received, the data cannot be stored in the sender's buffer forever. The SRT protocol stipulates that the maximum storage time is the delay amount.

[0126] like Figure 4 This is a flowchart of the optimal state of the sender and receiver buffers in Example 1 of the present invention. Imagine the delay as a window sliding from left to right on both the sender and receiver. In the figure, packets 1 through 6 have all received positive acknowledgments (ACKs) and have been removed from the sender's buffer. Packet 7 remains in the buffer awaiting a response from the receiver. This is an optimal transmission state, with minimal usage of the sender's buffer and most data being received promptly and ACKed.

[0127] like Figure 5 This is a flowchart of the poor state of the sender and receiver buffers in Example 1 of the present invention; the receiver did not receive data packet No. 3 and replied with a negative acknowledgment (NAK) to the sender. Due to various reasons, this process was delayed, resulting in data packet No. 3 being at the far left of the window. As the window slides, it will be kicked out of the sender buffer in the next step. In this case, if the retransmission of data packet No. 3 is lost or delayed again, the data packet will be lost, and the image at the decoding end may also have problems. At the same time, we also notice that the sender buffer in the figure is already full, which also indicates a transmission state at the critical point of packet loss. Therefore, the less the sender buffer is occupied, the safer the link. If the sender buffer is full, the link is likely to lose packets.

[0128] The function of the receive buffer is to sort the received data packets (the SRT protocol records the precise timestamp in the SRT packet header). On the one hand, sorting is required for decoding, and on the other hand, it is to find out the data packets that did not arrive in time, return a negative acknowledgment (NAK) to the sender, and wait for retransmission. Figure 4Data packet No. 1 at the receiving end has been sent to the decoder, and data packet No. 2 will also be sent to the decoder as the delay window slides. At the same time, data packets No. 2 to No. 6 have been saved in the receiving buffer. The receiving buffer is almost filled with data packets, which is a good link status. Figure 5 This indicates a poor link state. The receiver did not receive packet 3, and packet 3 is located at the far left of the receiver window, meaning it was likely skipped, resulting in issues with the decoded image. In this case, the low number of packets in the receive buffer may be due to packet loss or a mismatch between network bandwidth and video bitrate.

[0129] Therefore, the ideal state of the receive buffer is that its usage should be slightly lower than the delay amount. If the receive buffer occupancy becomes zero, problems are likely to occur in decoding the image.

[0130] The specific implementation process of an SRT streaming upload acceleration method is as follows: obtaining video streaming data, encapsulating the video streaming data into TS streaming packets at one time, and then storing them in a cache; determining in real time whether the cache has received confirmation characters arranged in sequence in the TS streaming packets, and obtaining the corresponding TS streaming packets from the cache if the confirmation characters have been received; encrypting the TS streaming packets to obtain encrypted streaming packets, and uploading the encrypted streaming packets to a cloud server.

[0131] like Figure 2 This is a schematic diagram of a specific process of a method for uploading SRT streams according to embodiment 1 of the present invention;

[0132] In step S200, video stream data is received via the PCIE transmission interface on the programmable chip;

[0133] In step S210, the video stream data is encapsulated into TS stream packets and then placed in the buffer; the video file received by the DMA is cut and encapsulated into TS stream packets, each TS stream packet is 188 bytes, and then the data is transferred to the buffer.

[0134] In step S220, it is determined in real time whether the buffer has received the acknowledgment character of the TS stream packet arranged in sequence. If the acknowledgment character is received, the corresponding TS stream packet is retrieved from the buffer. When each stream packet is transmitted to the buffer module, the buffer module returns an ACK, indicating that the stream packet has been received. If the ACK is not received, it means that the stream packet was lost during transmission. Therefore, it is determined in real time whether the ACK is received in sequence.

[0135] In step S230 , the TS stream packet is AES encrypted, and 128-bit or 256-bit AES encryption can be determined.

[0136] In step S240, a packet sequence number is added and the SRT protocol is encapsulated.

[0137] In step S250 , the SRT packet is converted into a UDP packet, which is packaged into a UDP packet with 7 TS stream packets, and then uploaded to the cloud server through the UDP transmission port of the programmable chip.

[0138] In the process of implementing an SRT streaming download acceleration method, the functions of each unit inside a programmable chip are as follows: a UDP transmission interface is used to obtain encrypted streaming packets from a cloud server; after the encrypted streaming packets are obtained at the UDP transmission interface, they are sent to the SRT unit; the SRT unit is used to implement encrypted streaming packet decryption and ARQ packet loss retransmission mechanism processing; a cache unit is communicated with the SRT unit, and the cache unit implements temporary storage after arranging the decrypted streaming packets according to sequence numbers; a TS encapsulation unit is used to obtain the decrypted streaming packets from the cache and then perform TS disassembly to obtain video streaming data; the PCIE transmission interface is used to transmit the video streaming data obtained by TS disassembly.

[0139] The specific implementation process of an SRT streaming download acceleration method is as follows: after obtaining the encrypted streaming packet from the cloud server, the SRT protocol is parsed, and then the streaming packet is decrypted and stored in the cache, and a confirmation character is fed back to the cloud server; after obtaining the decrypted streaming packet from the cache, the TS is disassembled to obtain the video streaming data such as Figure 7 This is a flowchart of a method for SRT streaming downloading according to embodiment 2 of the present invention;

[0140] In step S700 , the encrypted streaming packet is obtained from the cloud server and the UDP protocol is decoded.

[0141] In step S710 , the SRT protocol is decoded.

[0142] In step S720, the packet is decrypted using AES, which can be AES128 or AES256 according to the setting information.

[0143] In step S730, the sequence number of the packet is read, and the data is stored in the corresponding buffer location according to the sequence number, and an ACK is sent back to the cloud server.

[0144] In step S740 , the TS stream is disassembled to obtain video data.

[0145] In step S750, the acquired video data is downloaded via the PCIE transmission interface.

[0146] A programmable chip proposed in Example 3 of the present invention implements uploading and downloading of SRT streams through hardware, thereby reducing power consumption and CPU usage, and reducing delays caused by encryption and decryption calculations.

[0147] The programmable chip proposed in Example 3 of the present invention adds an ARQ packet loss retransmission mechanism during implementation, and has high hardware time sensitivity, thereby improving the ability to combat packet loss and delay.

[0148] A programmable chip proposed in Example 3 of the present invention adopts TS packet processing and SRT protocol processing during the implementation process. Both TS packet processing and SRT protocol processing have high-speed processing characteristics and are suitable for multi-channel 8K high-transmission rate packet processing.

[0149] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms are not necessarily the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments / methods or examples. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are mutually inconsistent.

[0150] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0151] Those skilled in the art will appreciate that the above embodiments are merely intended to clearly illustrate the present disclosure and are not intended to limit the scope of the present disclosure. Other changes or modifications may be made based on the above disclosure, and such changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for accelerating SRT streaming upload, characterized in that: Running on a programmable chip, uploading SRT streams through hardware includes the following steps: Obtaining video stream data, and encapsulating the video stream data into TS stream packets and then storing them in a cache; after encapsulating the video stream data into TS stream packets and then storing them in the cache, further comprising arranging the video stream data according to sequence numbers; the method for obtaining video stream data comprises: receiving the video stream data via a PCIE transmission interface on a programmable chip; Real-time determination of whether the TS stream packet confirmation character arranged in sequence is received in the buffer, and obtaining the corresponding TS stream packet from the buffer if the confirmation character is received; After encrypting the TS stream packet, an encrypted stream packet is obtained, and the encrypted stream packet is uploaded to the cloud server. The process of uploading the encrypted stream packet to the cloud server includes: re-encapsulating the encrypted stream packet into an SRT packet, converting the SRT packet into a UDP packet, and then uploading it to the cloud server through the UDP transmission port of the programmable chip.

2. A method for accelerating SRT streaming download, characterized in that: Running on a programmable chip, the method implements the download of SRT streams through hardware, including: After receiving the encrypted streaming packet from the cloud server, the SRT protocol is parsed, the streaming packet is decrypted and stored in a cache, and a confirmation character is fed back to the cloud server. The process of receiving the encrypted streaming packet from the cloud server and parsing the SRT protocol is as follows: obtaining the encrypted streaming packet in the form of a UDP packet through the UDP port of the programmable chip; then converting the encrypted streaming packet in the form of a UDP packet into an encrypted streaming packet in the form of an SRT packet; and decrypting the encrypted streaming packet after parsing the SRT protocol. The process of storing the decrypted stream packets in a cache and simultaneously feeding back the confirmation characters to the cloud server includes: arranging the decrypted stream packets according to sequence numbers and feeding back the confirmation characters arranged according to the sequence numbers to the cloud server; Obtain the decrypted streaming packet from the cache and then disassemble the TS to obtain the video streaming data; The method further includes obtaining the decrypted stream packet from the cache, performing TS decomposition, and transmitting the video stream data through the PCIE system.

3. A programmable chip, characterized in that: It includes a TS encapsulation unit, an SRT unit and a cache unit; the TS encapsulation unit is communicatively connected to the cache unit; the cache unit is communicatively connected to the SRT unit; The TS encapsulation unit is used to encapsulate the video stream data into TS stream packets, or to decompress the TS stream packets into video stream data; The SRT unit is used to implement ARQ packet loss verification, encryption and decryption of video streaming data, as well as encapsulation and parsing of SRT protocols; The cache unit temporarily stores the video stream data after arranging them in sequence; The chip also includes a PCIE transmission interface and a UDP transmission interface; The PCIE transmission interface is used to receive or send video streaming data according to the PCIE protocol; The UDP transmission interface is used to receive video streaming data from a cloud server or upload video streaming data to a cloud server according to the UDP protocol.

Images