A video transmission method, device, medium and program product

Abstract

The application relates to the technical field of video communication, and discloses a video transmission method, a device, a medium and a program product. The method comprises the following steps: establishing a single transmission control protocol transmission link with a media forwarding server; acquiring an end-to-end round-trip delay according to real-time transmission control protocol control signaling, and acquiring a target code rate operating state according to the round-trip delay and a current code rate operating state; acquiring a target encoding code rate according to the target code rate operating state, and performing video encoding processing according to the target encoding code rate to generate real-time transmission protocol video data; and sending the real-time transmission protocol video data to the media forwarding server through the single transmission control protocol transmission link, so that the media forwarding server transparently forwards the real-time transmission protocol video data to a video receiving terminal. The application can realize unified format video transmission, efficient and accurate code rate control, data congestion avoidance and video transmission delay reduction.

Description

Technical Field

[0001] This invention relates to the field of video communication technology, and in particular to a video transmission method, device, medium, and program product. Background Technology

[0002] Real-time video communication has been widely used in production and daily life. Its latency directly affects the communication effect, which is especially important in interactive video services.

[0003] Currently, existing video transmission methods typically employ the Real-Time Streaming Protocol (RTSP) with its Transmission Control Protocol (TCP) mode. When network bandwidth is sufficient, the latency of this method is acceptable. However, once network bandwidth begins to decrease, because RTSP itself lacks flow rate control, large amounts of data can easily accumulate when using the TCP protocol, resulting in significant video latency. Secondly, video stream forwarding between terminals usually requires a server. However, if terminals rely solely on TCP's own flow rate control for video transmission, when one terminal's TCP transmission speed is lower than another's, the higher-speed terminal cannot detect this and continues to send large amounts of data, causing data backlog and latency during server forwarding. Finally, when some terminals in the system use User Datagram Protocol (UDP) to transmit video, the incompatibility between UDP and TCP transport protocols means that ordinary gateways performing protocol conversion will further increase real-time video latency. Summary of the Invention

[0004] This invention provides a video transmission method, device, medium, and program product that can achieve video transmission in a unified format, achieve efficient and accurate bitrate control, avoid data congestion, and reduce video transmission latency.

[0005] According to one aspect of the present invention, a video transmission method is provided, applied to a video transmitting terminal accessed using a transmission control protocol, comprising: A single transmission control protocol transmission link is established with a media forwarding server. The media forwarding server is used to transparently forward real-time transmission protocol video data and real-time transmission control protocol control signaling between terminals of different access types. The single transmission control protocol transmission link carries both real-time transmission protocol video data and real-time transmission control protocol control signaling. According to the control signaling of the real-time transmission control protocol, the end-to-end round-trip time is obtained, and according to the round-trip time and the current bit rate operating status, the target bit rate operating status is obtained; Based on the target bitrate operating status, the target encoding bitrate is obtained, and video encoding processing is performed based on the target encoding bitrate to generate real-time transmission protocol video data. The real-time transmission protocol video data is sent to the media forwarding server via the single transmission control protocol transmission link, so that the media forwarding server can transparently forward it to the video receiving terminal.

[0006] According to another aspect of the present invention, a video transmission apparatus is provided, applied to a video transmitting terminal accessed using a transmission control protocol, comprising: The link establishment module is used to establish a single transmission control protocol transmission link with the media forwarding server. The media forwarding server is used to transparently forward real-time transmission protocol video data and real-time transmission control protocol control signaling between terminals of different access types. The single transmission control protocol transmission link carries both real-time transmission protocol video data and real-time transmission control protocol control signaling. The status acquisition module is used to acquire the end-to-end round-trip time according to the control signaling of the real-time transmission control protocol, and to acquire the target bitrate operating status according to the round-trip time and the current bitrate operating status. The video encoding module is used to obtain the target encoding bitrate according to the target bitrate running status, and perform video encoding processing according to the target encoding bitrate to generate real-time transmission protocol video data; The data sending module is used to send the real-time transmission protocol video data to the media forwarding server through the single transmission control protocol transmission link, so that the media forwarding server can transparently forward it to the video receiving terminal.

[0007] According to another aspect of the present invention, a terminal device is provided, the terminal device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video transmission method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program configured to cause a processor to execute and implement the video transmission method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the video transmission method described in any embodiment of the present invention.

[0010] The technical solution of this invention employs a video transmitting terminal with Transmission Control Protocol (TCP) access to establish a single TCP transmission link with a media forwarding server. The media forwarding server transparently forwards real-time transmission protocol (RTP) video data and RTP control signaling between terminals of different access types. The single TCP transmission link simultaneously carries both RTP video data and RTP control signaling. Based on the RTP control signaling, the end-to-end round-trip time is obtained, and based on the round-trip time and the current bitrate operating status, the target bitrate operating status is obtained. Based on the target bitrate operating status, the target encoding bitrate is obtained, and video encoding processing is performed based on the target encoding bitrate to generate... The system generates real-time transmission protocol (RTP) video data. Through a single transmission control protocol (TCP) link, the RTP video data is sent to a media forwarding server, which then transparently forwards it to the video receiving terminal. By configuring the media forwarding server to transparently forward data between terminals of different access types, and establishing a single TCP link to simultaneously carry RTP video data and RTP control signaling, the system performs round-trip delay assessment based on the RTP control signaling and automatically adjusts the encoding bitrate based on the assessment results. This enables unified format video transmission, efficient and accurate bitrate control, avoids data congestion, and reduces video transmission latency.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a video transmission method provided according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of a video transmission method provided according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a video transmission device according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of a terminal device that implements the video transmission method of this invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1 This is a flowchart of a video transmission method provided in Embodiment 1 of the present invention. This embodiment is applicable to low-latency real-time video transmission in a hybrid TCP and UDP network. The method can be applied to a video transmitting terminal that uses a transmission control protocol for access. This video transmitting terminal can be a terminal device, such as a computer device, a mobile terminal, etc. Figure 1 As shown, the method includes: S110. Establish a single Transmission Control Protocol (TCP) transmission link with the media forwarding server.

[0017] The media forwarding server is used to transparently forward Real-time Transport Protocol (RTP) video data and Real-time Transport Control Protocol (RTCP) control signaling between terminals of different access types. Each single RTCP transmission link carries both RTP video data and RTCP control signaling.

[0018] In this embodiment, a media forwarding server can be added between the video sending terminal and the video receiving terminal. This server is equipped with a media forwarding service for sending and receiving video data from the terminals, and for transmitting RTP and RTCP data between the two terminals in a video call. The media forwarding service can listen on a TCP port for communication with TCP network terminals, and can also bind to a UDP port for communication with UDP network terminals. It is understood that the media forwarding service can also be deployed on hardware with independent public Internet Protocol addresses, such as distributed server clusters or edge computing nodes.

[0019] The access type can include TCP network access and UDP network access. For UDP network terminals, the RTP / RTCP protocol can be used to control the terminal bitrate. For TCP network terminals, each terminal uses one, rather than two, TCP connections to transmit RTP and RTCP, which ensures that RTCP and RTP maintain the same latency characteristics, thereby accurately assessing network latency. There is no such restriction for terminals using UDP to transmit video; RTP and RTCP can use the same or different ports. Typically, this embodiment allows TCP network terminals to establish video communication with TCP network terminals, TCP network terminals to UDP network terminals, and UDP network terminals to UDP network terminals through a media forwarding service. Each terminal uses either a TCP or UDP network, connects independently to the media forwarding service, and uniformly uses RTP and RTCP to carry video data and control signaling.

[0020] In this embodiment, the terminal can choose the access protocol independently based on its own network environment. For terminals with no UDP port blocking, no ISP speed limits, and a packet loss rate that meets the requirements, the UDP protocol is used for access. For terminals with excessively high UDP packet loss rates, UDP port blocking by firewalls, or ISP speed limits for UDP, the TCP protocol is used for access.

[0021] Specifically, when the video sending terminal is a TCP network terminal, it first establishes a TCP connection with the media forwarding server as a single TCP transmission link. Simultaneously, the video receiving terminal establishes a connection with the media forwarding server. The media forwarding server is responsible for receiving RTP / RTCP data from the terminal and transparently forwarding the received data to the other terminal. Transparent forwarding means that the media forwarding server does not perform any processing on the received data (such as protocol conversion or additional data buffering) and directly forwards it to the other terminal.

[0022] S120. According to the control signaling of the real-time transmission control protocol, obtain the end-to-end round-trip time, and according to the round-trip time and the current bit rate operating status, obtain the target bit rate operating status.

[0023] Specifically, after establishing the transmission link, the video sending terminal generates a probe message based on RTCP control signaling and sends it to the media forwarding server via a single TCP transmission link. The media forwarding server transparently forwards the probe message to the video receiving terminal through the access connection (TCP or UDP) corresponding to the video receiving terminal. After receiving the probe message, the video receiving terminal returns an acknowledgment message to the video sending terminal along the original path via the media forwarding server. The video sending terminal calculates the round-trip time (RTT), denoted as Ti, based on the sending time of the probe message and the receiving time of the acknowledgment message. This Ti covers the entire process of the access link and media forwarding at both ends, reflecting the congestion status of the entire end-to-end link, rather than just the status of the sending terminal's own TCP connection.

[0024] Then, based on the current RTT and the preset mapping relationship between latency range and adjustment type, the current adjustment type is obtained; further, based on the adjustment type, the target bitrate operating state is obtained. The adjustment type can include decrease, maintain, and increase. The bitrate operating state can include rising, maintain, and falling states. Typically, if the adjustment type is decrease, the target bitrate operating state is determined to be falling; if the adjustment type is increase, the target bitrate operating state is determined to be rising; if the adjustment type is maintain, the target bitrate operating state is determined to be maintain.

[0025] S130. Based on the target bitrate operating status, obtain the target encoding bitrate, and perform video encoding processing based on the target encoding bitrate to generate real-time transmission protocol video data.

[0026] Specifically, according to a preset bitrate adjustment rule, the current encoding bitrate is dynamically adjusted based on the target bitrate's operating status to obtain the target encoding bitrate. The video to be transmitted is then encoded according to this target bitrate to obtain RTP video data. The preset bitrate adjustment rule can be as follows: if the bitrate is in an increasing state, the current encoding bitrate is increased by a preset bitrate per unit time; if the bitrate is in a decreasing state, the current encoding bitrate is decreased by a preset bitrate per unit time; if the bitrate is in a holding state, the current encoding bitrate is maintained.

[0027] Optionally, obtaining the target coding bitrate based on the target bitrate running status may include: When the target bitrate is in a decreasing state, the product of the current coding bitrate and the first preset coefficient is calculated in each cycle to be used as the target coding bitrate. When the target bitrate is in an increasing state, the product of the current coding bitrate and the second preset coefficient is calculated in each cycle to be used as the target coding bitrate. Wherein, the first preset coefficient is greater than 0 and less than 1, and the second preset coefficient is greater than 1. Typically, the first preset coefficient is 0.5, the second preset coefficient is 1.2, and the period is one second.

[0028] In this embodiment, when the target bitrate is in a decreasing state, the target coding bitrate per second is the product of the coding bitrate of the previous second and a first preset coefficient; when the target bitrate is in a increasing state, the target coding bitrate per second is the product of the coding bitrate of the previous second and a second preset coefficient. When the target bitrate is in a holding state, the target coding bitrate per second is equal to the coding bitrate of the previous second.

[0029] It is worth noting that the encoding bitrate needs to be limited to the range of [150, 2000], that is, the bitrate cannot exceed the range, in order to avoid poor image quality or wasted bandwidth.

[0030] Optionally, the technical solution of this embodiment of the invention may further include: Obtain the network type, and based on the network type, obtain the startup encoding bitrate; Obtain the expected stable bitrate, and obtain the initial encoding bitrate based on the expected stable bitrate and the starting encoding bitrate.

[0031] In this embodiment, different initial encoding bitrates S can be preset for different network types, such as wired networks, Wi-Fi networks, and 5G mobile communication networks. Secondly, the desired stable video bitrate under the terminal service can be obtained based on the user's configuration operations, and used as the desired stable bitrate X. Finally, the smaller value between S and X can be selected as the initial encoding bitrate based on the formula min(S,X). The initial encoding bitrate can be the encoding bitrate used when a single TCP transmission link is first established.

[0032] It should be noted that during the transmission link establishment phase, the target bitrate is in an increasing state by default. Before reaching the set value X, the bitrate will be increased as much as possible within the network's limits. Once the bitrate reaches X, even if it is determined that the bitrate needs to be increased, X will be kept constant.

[0033] S140. The real-time transmission protocol video data is sent to the media forwarding server through the single transmission control protocol transmission link, so that the media forwarding server can transparently forward it to the video receiving terminal.

[0034] Specifically, the video sending terminal transmits RTP video data to the media forwarding server via a single TCP transmission link. The media forwarding server then forwards the RTP video data to the video receiving terminal to achieve video transmission.

[0035] The technical solution of this invention employs a video transmitting terminal with Transmission Control Protocol (TCP) access to establish a single TCP transmission link with a media forwarding server. The media forwarding server transparently forwards real-time transmission protocol (RTP) video data and RTP control signaling between terminals of different access types. The single TCP transmission link simultaneously carries both RTP video data and RTP control signaling. Based on the RTP control signaling, the end-to-end round-trip time is obtained, and based on the round-trip time and the current bitrate operating status, the target bitrate operating status is obtained. Based on the target bitrate operating status, the target encoding bitrate is obtained, and video encoding processing is performed based on the target encoding bitrate to generate... The system generates real-time transmission protocol (RTP) video data. Through a single transmission control protocol (TCP) link, the RTP video data is sent to a media forwarding server, which then transparently forwards it to the video receiving terminal. By configuring the media forwarding server to transparently forward data between terminals of different access types, and establishing a single TCP link to simultaneously carry RTP video data and RTP control signaling, the system performs round-trip delay assessment based on the RTP control signaling and automatically adjusts the encoding bitrate based on the assessment results. This enables unified format video transmission, efficient and accurate bitrate control, avoids data congestion, and reduces video transmission latency.

[0036] Example 2 Figure 2 This is a flowchart illustrating a video transmission method according to Embodiment 2 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with one or more of the above implementation methods. For example... Figure 2 As shown, the method includes: S210. Establish a single Transmission Control Protocol (TCP) transmission link with the media forwarding server.

[0037] S220. Obtain the end-to-end round-trip delay according to the control signaling of the real-time transmission control protocol.

[0038] S230. Based on the round-trip delay, obtain the delay change trend, and based on the delay change trend and the current bitrate running status, obtain the target bitrate running status.

[0039] It should be noted that because the media forwarding server uses a transparent forwarding method for the terminal's RTCP, RTT can measure the overall latency between terminals, and the network status of any terminal's TCP connection can be reflected in the RTT value. Specifically, when the video transmission rate is less than or equal to the transmission processing rate in the end-to-end TCP network, the Ti value remains stable; when the transmission rate is greater than the transmission rate in the end-to-end TCP network, the Ti value begins to rise; and when the transmission rate is less than the transmission rate in the end-to-end TCP network, the Ti value begins to fall. The transmission rate of the end-to-end TCP network changes constantly, and the RTT value can be used to assess whether the transmitted video data has reached the upper limit of the transmission rate in the end-to-end TCP network. Comparing the current Ti value with its most recent historical value allows us to determine the current trend of Ti.

[0040] In this embodiment, multiple RTTs within a certain time period can be statistically analyzed, and the trend of RTT changes can be determined. The trend includes an upward trend, a downward trend, and a stable state. Furthermore, the current bitrate operating state is dynamically adjusted based on the RTT change trend to obtain the target bitrate operating state.

[0041] S240. Based on the target bitrate operating status, obtain the target encoding bitrate, and perform video encoding processing based on the target encoding bitrate to generate real-time transmission protocol video data.

[0042] S250. The real-time transmission protocol video data is sent to the media forwarding server through the single transmission control protocol transmission link, so that the media forwarding server can transparently forward it to the video receiving terminal.

[0043] The technical solution of this invention obtains the latency change trend based on the round-trip latency, and obtains the target bitrate operating status based on the latency change trend and the current bitrate operating status. By dynamically adjusting the bitrate operating status according to the latency change trend, data congestion when bandwidth is reduced can be avoided, while improving the utilization of network bandwidth, thus balancing the stability and efficiency of video transmission.

[0044] Optionally, obtaining the delay change trend based on the round-trip delay may include: Get the number of round-trip times within the most recent time window; When the number is greater than or equal to a preset value, the round-trip delays within the most recent time window are taken as candidate round-trip delays; when the number is less than a preset value, the round-trip delay of the most recent preset value is taken as a candidate round-trip delay. Based on the candidate round-trip delays, the delay change trend is obtained.

[0045] The time window can be 200 milliseconds. Specifically, first, the number of Ti values ​​within the most recent 200 milliseconds is counted. Then, if this number is greater than or equal to a preset value, such as 3, all Ti values ​​within the 200 milliseconds are considered candidate round-trip times (RTCs). Second, if this number is less than the preset value, the three most recent Ti values ​​are counted as candidate RTCs, regardless of whether they are within the 200 milliseconds. Finally, time series analysis is performed on all candidate RTCs to obtain the RTC trend.

[0046] Optionally, obtaining the delay change trend based on each of the candidate round-trip delays may include: If the round-trip delay of each candidate is monotonically increasing, then the delay change trend is determined to be an upward trend; if the round-trip delay of each candidate is monotonically decreasing, then the delay change trend is determined to be a downward trend. If the candidate round-trip delays do not satisfy monotonicity, or if the candidate round-trip delays satisfy monotonicity but the ratio of the minimum to the maximum value is greater than a preset ratio, then the delay change trend is determined to be a stable state.

[0047] In this embodiment, when acquiring the latency change trend, firstly, it can be determined whether each candidate round-trip latency satisfies monotonicity. If so, it is further determined whether the ratio of the minimum to the maximum value of the candidate round-trip latency is greater than a preset ratio, for example, 0.8. If it is determined to be less than or equal to 0.8, then when all candidate round-trip latency is monotonically increasing, the latency change trend is directly determined to be an upward trend; and when all candidate round-trip latency is monotonically decreasing, the latency change trend is directly determined to be a downward trend. If it is determined to be greater than 0.8, then the latency change trend is determined to be a stable state. Secondly, if it is determined that all candidate round-trip latency does not satisfy monotonicity, then the latency change trend is also determined to be a stable state.

[0048] Optionally, obtaining the target bitrate operating status based on the latency change trend and the current bitrate operating status may include: When the latency change trend is downward, the target bitrate operating state is set to a holding state; when the latency change trend is upward, the target bitrate operating state is set to a downward state. When the latency change trend is stable, the current bitrate running state is progressively switched to obtain the target bitrate running state.

[0049] In this embodiment, if Ti is in a decreasing trend, the current bitrate is kept unchanged, i.e., the target bitrate operating state is the held state. If Ti is in an increasing trend, the encoding bitrate needs to be reduced, i.e., the target bitrate operating state is the decreasing state. Next, if Ti is in a stable state, the subsequent target bitrate operating state is determined based on the current bitrate operating state. For example, if the current bitrate operating state is decreasing, it is changed to a held state to serve as the target bitrate operating state; if the current bitrate operating state is held, it is changed to an increasing state to serve as the target bitrate operating state; if the current bitrate operating state is increasing, it is maintained as the target bitrate operating state.

[0050] In one specific implementation of this embodiment, taking a video call between two TCP network terminals using a media forwarding service as an example, assuming the terminal uses a Wi-Fi network, the initial encoding bitrate S is set to 300 kilobits per second (kbps), the expected stable bitrate X is 2000 kbps, the bitrate increase coefficient (second preset coefficient) is set to 1.2, and the bitrate decrease coefficient (first preset coefficient) is set to 0.5. For wired networks, S can be set to a higher value of 1000 kbps.

[0051] First, both terminals establish TCP connections with the media forwarding server, with the initial encoding bitrate set to min(300,2000) = 300kbps. Then, the RTT (Relay Time To-Time) Ti between the two terminals is calculated using RTCP, and the latency trend is obtained based on the current Ti value and its most recent historical value. Based on this latency trend and the current bitrate operating state, the target bitrate operating state is determined. When the target bitrate operating state is decreasing, the bitrate per second decreases to 0.5 times the current value; when it is increasing, the bitrate per second increases to 1.2 times the current value; and when it is holding steady, the current bitrate remains unchanged. Finally, video frames are encoded into RTP video data according to the target encoding bitrate, and the RTP video data is smoothly and uniformly transmitted to the media forwarding server, matching the currently estimated available network bandwidth.

[0052] This invention provides a low-latency video transmission method using a hybrid TCP and UDP network. It maintains low-latency real-time video transmission even when using a TCP network, allowing for mixed TCP and UDP terminal networking. This maximizes the utilization of terminal network resources and ensures low-latency real-time video transmission. The media forwarding service can forward the same or different network protocols between terminals, enabling terminals of various network types to conduct video calls and detect available end-to-end transmission bandwidth, thus expanding its applicability.

[0053] The technical solution of this invention can avoid video data accumulation caused by server forwarding, greatly reducing latency. Terminals on different networks can initiate video calls arbitrarily without additional data buffering and protocol conversion. Furthermore, it allows terminals to choose between TCP or UDP protocols to transmit video based on their own network conditions, thereby achieving optimal low-latency video call performance.

[0054] Example 3 Figure 3 This is a schematic diagram of a video transmission device provided in Embodiment 3 of the present invention. Figure 3 As shown, this device is applied to a video transmitting terminal that uses a transmission control protocol for access, and includes: a link establishment module 310, a status acquisition module 320, a video encoding module 330, and a data transmission module 340; wherein, The link establishment module 310 is used to establish a single transmission control protocol transmission link with the media forwarding server. The media forwarding server is used to transparently forward real-time transmission protocol video data and real-time transmission control protocol control signaling between terminals of different access types. The single transmission control protocol transmission link carries both real-time transmission protocol video data and real-time transmission control protocol control signaling. The status acquisition module 320 is used to acquire the end-to-end round-trip time according to the control signaling of the real-time transmission control protocol, and to acquire the target bit rate operating status according to the round-trip time and the current bit rate operating status. The video encoding module 330 is used to obtain the target encoding bitrate according to the target bitrate running status, and perform video encoding processing according to the target encoding bitrate to generate real-time transmission protocol video data; The data sending module 340 is used to send the real-time transmission protocol video data to the media forwarding server through the single transmission control protocol transmission link, so that the media forwarding server can transparently forward it to the video receiving terminal.

[0055] The technical solution of this invention employs a video transmitting terminal with Transmission Control Protocol (TCP) access to establish a single TCP transmission link with a media forwarding server. The media forwarding server transparently forwards real-time transmission protocol (RTP) video data and RTP control signaling between terminals of different access types. The single TCP transmission link simultaneously carries both RTP video data and RTP control signaling. Based on the RTP control signaling, the end-to-end round-trip time is obtained, and based on the round-trip time and the current bitrate operating status, the target bitrate operating status is obtained. Based on the target bitrate operating status, the target encoding bitrate is obtained, and video encoding processing is performed based on the target encoding bitrate to generate... The system generates real-time transmission protocol (RTP) video data. Through a single transmission control protocol (TCP) link, the RTP video data is sent to a media forwarding server, which then transparently forwards it to the video receiving terminal. By configuring the media forwarding server to transparently forward data between terminals of different access types, and establishing a single TCP link to simultaneously carry RTP video data and RTP control signaling, the system performs round-trip delay assessment based on the RTP control signaling and automatically adjusts the encoding bitrate based on the assessment results. This enables unified format video transmission, efficient and accurate bitrate control, avoids data congestion, and reduces video transmission latency.

[0056] Optionally, the status acquisition module 320 includes: The status acquisition unit is used to acquire the latency change trend based on the round-trip latency, and to acquire the target bitrate operating status based on the latency change trend and the current bitrate operating status.

[0057] Optionally, the status acquisition unit includes: The trend acquisition sub-unit is used to obtain the number of round-trip delays within the most recent time window; When the number is greater than or equal to a preset value, the round-trip delays within the most recent time window are taken as candidate round-trip delays; when the number is less than a preset value, the round-trip delay of the most recent preset value is taken as a candidate round-trip delay. Based on the candidate round-trip delays, the delay change trend is obtained.

[0058] Optionally, the trend acquisition subunit is specifically used to determine that the delay change trend is an upward trend if the round-trip delay of each candidate is monotonically increasing, and to determine that the delay change trend is a downward trend if the round-trip delay of each candidate is monotonically decreasing. If the candidate round-trip delays do not satisfy monotonicity, or if the candidate round-trip delays satisfy monotonicity but the ratio of the minimum to the maximum value is greater than a preset ratio, then the delay change trend is determined to be a stable state.

[0059] Optionally, the status acquisition unit may also include: The state acquisition subunit is used to acquire the target bitrate operating state as a hold state when the latency change trend is decreasing, and to acquire the target bitrate operating state as a decrease state when the latency change trend is increasing. When the latency change trend is stable, the current bitrate running state is progressively switched to obtain the target bitrate running state.

[0060] Optionally, the video encoding module 330 is specifically used to calculate the product of the current encoding bitrate and the first preset coefficient in each cycle when the target bitrate is in a decreasing state, so as to use it as the target encoding bitrate; When the target bitrate is in an increasing state, the product of the current coding bitrate and the second preset coefficient is calculated in each cycle to be used as the target coding bitrate. Wherein, the first preset coefficient is greater than 0 and less than 1, and the second preset coefficient is greater than 1.

[0061] Optionally, the video transmission device also includes: The startup encoding bitrate acquisition module is used to obtain the network type and, based on the network type, obtain the startup encoding bitrate. The initial encoding bitrate acquisition module is used to acquire the expected stable bitrate and, based on the expected stable bitrate and the starting encoding bitrate, acquire the initial encoding bitrate.

[0062] The video transmission device provided in the embodiments of the present invention can execute the video transmission method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0063] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0064] Example 4 Figure 4A schematic diagram of a terminal device 40 that can be used to implement embodiments of the present invention is shown. Terminal device 40 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Terminal device 40 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0065] like Figure 4 As shown, the terminal device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from the storage unit 48 into the RAM 43. The RAM 43 can also store various programs and data required for the operation of the terminal device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0066] Multiple components in terminal device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of displays, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows terminal device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0067] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as video transmission methods.

[0068] In some embodiments, the video transmission method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on terminal device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the video transmission method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the video transmission method by any other suitable means (e.g., by means of firmware).

[0069] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), system-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0070] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0071] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0072] To provide interaction with the user, the systems and techniques described herein can be implemented on terminal device 40, which includes: a display device (e.g., a cathode ray tube or liquid crystal display) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to terminal device 40. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0073] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0074] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact via a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server.

[0075] This embodiment may also include a computer program product, which includes a computer program that, when executed by a processor, implements the video transmission method provided in any embodiment of the present invention.

[0076] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0077] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A video transmission method, characterized in that, Applied to video transmitting terminals that use Transmission Control Protocol (TCP) access, including: A single transmission control protocol transmission link is established with a media forwarding server. The media forwarding server is used to transparently forward real-time transmission protocol video data and real-time transmission control protocol control signaling between terminals of different access types. The single transmission control protocol transmission link carries both real-time transmission protocol video data and real-time transmission control protocol control signaling. According to the control signaling of the real-time transmission control protocol, the end-to-end round-trip time is obtained, and according to the round-trip time and the current bit rate operating status, the target bit rate operating status is obtained; Based on the target bitrate operating status, the target encoding bitrate is obtained, and video encoding processing is performed based on the target encoding bitrate to generate real-time transmission protocol video data. The real-time transmission protocol video data is sent to the media forwarding server via the single transmission control protocol transmission link, so that the media forwarding server can transparently forward it to the video receiving terminal.

2. The method according to claim 1, characterized in that, Based on the round-trip time and the current bitrate operating status, the target bitrate operating status is obtained, including: Based on the round-trip time delay, obtain the delay change trend, and based on the delay change trend and the current bitrate running status, obtain the target bitrate running status.

3. The method according to claim 2, characterized in that, Based on the round-trip time delay, the time delay change trend is obtained, including: Get the number of round-trip times within the most recent time window; When the number is greater than or equal to a preset value, the round-trip delays within the most recent time window are taken as candidate round-trip delays; when the number is less than a preset value, the round-trip delay of the most recent preset value is taken as a candidate round-trip delay. Based on the candidate round-trip delays, the delay change trend is obtained.

4. The method according to claim 3, characterized in that, Based on the candidate round-trip times, the trend of the time delay change is obtained, including: If the round-trip delay of each candidate is monotonically increasing, then the delay change trend is determined to be an upward trend; if the round-trip delay of each candidate is monotonically decreasing, then the delay change trend is determined to be a downward trend. If the candidate round-trip delays do not satisfy monotonicity, or if the candidate round-trip delays satisfy monotonicity but the ratio of the minimum to the maximum value is greater than a preset ratio, then the delay change trend is determined to be a stable state.

5. The method according to claim 2, characterized in that, Based on the latency change trend and the current bitrate operating status, the target bitrate operating status is obtained, including: When the latency change trend is downward, the target bitrate operating state is set to a holding state; when the latency change trend is upward, the target bitrate operating state is set to a downward state. When the latency change trend is stable, the current bitrate running state is progressively switched to obtain the target bitrate running state.

6. The method according to claim 1, characterized in that, Based on the target bitrate operating status, the target coding bitrate is obtained, including: When the target bitrate is in a decreasing state, the product of the current coding bitrate and the first preset coefficient is calculated in each cycle to be used as the target coding bitrate. When the target bitrate is in an increasing state, the product of the current coding bitrate and the second preset coefficient is calculated in each cycle to be used as the target coding bitrate. Wherein, the first preset coefficient is greater than 0 and less than 1, and the second preset coefficient is greater than 1.

7. The method according to claim 1, characterized in that, Also includes: Obtain the network type, and based on the network type, obtain the startup encoding bitrate; Obtain the expected stable bitrate, and obtain the initial encoding bitrate based on the expected stable bitrate and the starting encoding bitrate.

8. A terminal device, characterized in that, The terminal device includes: At least one processor, and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video transmission method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the video transmission method according to any one of claims 1-7.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the video transmission method according to any one of claims 1-7.

Images