High-quality audio and video transmission method under weak link

Abstract

The invention discloses a high-quality audio and video transmission method under a weak link, which comprises the following steps: firstly, acquiring audio and video signals from a signal source in real time, performing AVC / H.264 or HEVC / H.265-based coding compression on the signal source, then detecting the transmission quality of the link, and setting matched coding parameters to a video coding module according to the transmission quality of the link. According to the method, original data of coding compression are reduced under the condition that the system is changed and enough information of the picture is fully reserved, so that better coding quality can be obtained under the same code rate, and a proper picture comparison is automatically selected at a video program acquisition end according to the bandwidth change of a link to carry out equal-ratio transformation on the input original picture, so that the coding quality is improved. And packaging the parameter information of the equal-ratio change into a transmission packet for a program receiving end to carry out decoding restoration reference, carrying out reverse decoding at the receiving end according to the coding parameters of the acquisition end, and restoring an original image through an image enhancement algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of audio and video transmission, and in particular to a high-quality audio and video transmission method under a weak link. Background Art

[0002] With the advancement of urban construction, high-rise buildings are rising in cities. The original method of using satellites and microwaves as the main means of video program transmission has gradually lost its technical advantages. At the same time, with the development of mobile Internet technology, mobile network coverage components have become popular and bandwidth has gradually increased, especially the deployment of 5G networks, making video program transmission through mobile networks possible.

[0003] In the prior art, for example, Chinese patent CN107846594A discloses a method for real-time transmission of high-quality audio and video in a network, including a server side, wherein the server side includes audio acquisition and video acquisition, audio acquisition is connected to audio compression, audio compression is connected to sending data 1, video acquisition is connected to video compression, video compression is connected to sending data 2, sending data 1 and sending data 2 are connected to the network, the other end of the network is connected to receiving data 1 and receiving data 2, receiving data 1 is connected to audio decompression, audio decompression is connected to audio display, receiving data 2 is connected to video decompression, video decompression is connected to video display, and video display and audio display are connected to a client.

[0004] However, in the prior art, with the continuous improvement of network infrastructure, the data bandwidth that can be carried is constantly increasing, but at the same time, the video format is also constantly improving, which puts forward more stringent requirements on the transmission link. At the same time, there are still many weaknesses in the basic network coverage, especially the mobile network coverage; in conventional video transmission, the video signal of the signal source is generally collected, and then the video signal is subjected to preliminary processing such as noise filtering, and then encoded, and finally transmitted through the transmission link. If the bandwidth of the transmission link is not enough to carry the bandwidth of the encoded data, data packet loss will occur, thereby affecting the video quality of the destination end. In response to the above two problems, the general practice is to calculate the link bandwidth as accurately as possible and limit the output bit rate of the video encoding compression to an appropriate range. However, when the link fluctuation range is very large, in order to maintain the smoothness of the video, the bit rate needs to be compressed to a very low level, which will greatly degrade the effect of the video program; at the same time, the link bandwidth is wasted for a large amount of time. Summary of the invention

[0005] The purpose of the present invention is to provide a high-quality audio and video transmission method under a weak link to solve the problem proposed in the above background technology that with the continuous improvement of network infrastructure, the data bandwidth that can be carried is constantly increasing, but at the same time, the video format is also constantly improving, which puts forward more stringent requirements on the transmission link. At the same time, there are still many weaknesses in the basic network coverage, especially the mobile network coverage; in conventional video transmission, the video signal of the signal source is generally collected, and then the video signal is subjected to preliminary processing such as noise filtering, and then encoded, and finally transmitted through the transmission link. If the bandwidth of the transmission link is not enough to carry the bandwidth of the encoded data, data packet loss will occur, thereby affecting the video quality of the destination end. For the above two problems, the general practice is to calculate the link bandwidth as accurately as possible and limit the output bit rate of the video encoding compression to an appropriate range. However, when the link fluctuation range is very large, in order to maintain the smoothness of the video, the bit rate needs to be compressed to a very low level, which will greatly degrade the effect of the video program; at the same time, the link bandwidth is wasted for a large amount of time.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a method for high-quality audio and video transmission under a weak link, comprising first real-time acquisition of audio and video signals from a signal source, performing AVC / H.264 or HEVC / H.265 encoding compression on the signal source, then detecting the transmission quality of the link, and setting matching encoding parameters to a video encoding module according to the link transmission quality, setting a corresponding relationship between an audio and video standard and a minimum bit rate, and during the audio and video transmission process, when the link quality decreases, if the corresponding encoding output bit rate is lower than the minimum bit rate threshold of the standard, automatically performing geometric transformation on the YUV of the signal source according to a pre-set video change algorithm, and conversely, when the link quality changes from poor to good, gradually increasing the frame size according to the corresponding relationship between the video standard and the minimum bit rate, and carrying both the original standard and the minimum bit rate in the encoded data package. The transformed format information is used for reverse operation at the receiving end. At the receiving end of the audio and video program, decoding is first performed according to the settlement algorithm to obtain the initial YUV, and then the picture is up-converted based on the AI ​​enhancement technology to restore the original picture and audio at the acquisition end. By changing the format, the original data of encoding compression is reduced while fully retaining sufficient information of the picture, so that better encoding quality can be obtained at the same bit rate. At the video program acquisition end, the appropriate aspect ratio is automatically selected according to the bandwidth change of the link to perform geometric transformation on the input original picture, and the geometrically changed parameter information is encapsulated into the transmission packet for decoding and restoration reference by the program receiving end. At the receiving end, reverse decoding is performed according to the encoding parameters of the acquisition end, and the original image is restored through the image enhancement algorithm. Not only will the effect of the video program not be degraded, but the link bandwidth will not be wasted.

[0007] Preferably, the audio and video signals collected from the signal source are transmitted separately by audio and video, and the audio and video are compressed separately by an encoder.

[0008] Preferably, when the video is sent, the compressed video data is cut to reduce the length of the data packet and then packaged, and the video data type and the serial number of the data packet are indicated in the packet header. The data packets belonging to the same single-frame video have the same serial number in the packet header, and their subsequence numbers are filled in the packet header in the order of segmentation, and the timestamp is written to send the edited video data packet.

[0009] Preferably, the audio data is directly packaged after being compressed, and the audio data type and sequence number are indicated in the packet header, and then the timestamp is filled in, and the edited audio data packet is sent.

[0010] Preferably, a memory buffer is set at the receiving end of the audio and video data packets, and the received audio and video data packets are placed in the memory buffer, so as to filter out the delay jitter caused by the network.

[0011] Preferably, the received video data packets are inserted into the memory buffer according to the sequence number and the subsequence number. If all the data packets after the segmentation of a complete frame of image have been received, they are decoded and reassembled to form a complete single-frame video data, which are placed into the playback buffer according to the sequence number; If the received single-frame image data is incomplete, its serial number is determined in sequence with the serial number of the currently played data. If it is behind the played data, it continues to wait in the buffer. If it is in front of the played data, it is simply discarded. When the video playback buffer is full, it starts to play in sequence.

[0012] Preferably, the received audio data packet is decoded and inserted into the memory buffer according to the serial number. If the serial number is located in front of the played data, it is simply discarded. When the audio playback buffer is full, it starts to be played in sequence, and the synchronization of audio and video is guaranteed by the timestamp in the data packet.

[0013] Preferably, when transmitting the collected audio and video data packets, network transmission is adopted, and VPN networking is used to establish point-to-point communication between the collection end and the client under the 4G network; Among them, the LAN is suitable for short-distance transmission and can provide a low-cost and high-bandwidth transmission network, while the 4G network is more in line with the needs of long-distance mobile transmission. The LAN is used for data transmission first, and the 4G network is used for transmission when leaving the LAN environment.

[0014] Compared with the prior art, the present invention has the following beneficial effects: In the present invention, the format is first changed, and then the original data of coding compression is reduced while fully retaining sufficient information of the picture, so that better coding quality can be obtained at the same bit rate. At the video program acquisition end, a suitable picture ratio is automatically selected according to the bandwidth change of the link to perform geometric transformation on the input original picture, and the parameter information of the geometric change is encapsulated into a transmission packet for reference by the program receiving end for decoding and restoration. At the receiving end, reverse decoding is performed according to the coding parameters of the acquisition end, and the original image is restored through an image enhancement algorithm. In this way, not only the effect of the video program will not be degraded, but also the link bandwidth will not be wasted. DETAILED DESCRIPTION

[0016] Embodiment: A high-quality audio and video transmission method under a weak link, firstly, collect audio and video signals from a signal source in real time, perform AVC / H.264 or HEVC / H.265 encoding compression on the signal source, and send audio and video separately. When sending the video, cut the compressed video data to reduce the length of the data packet and then pack it, and indicate the video data type and the sequence number of the data packet in the packet header. The data packets belonging to the same single-frame video have the same sequence number in the packet header, and fill in their subsequence numbers in the packet header in the order of segmentation, and write a timestamp, send the edited video data packet, and directly pack the audio data after compression, and indicate the audio data type and sequence number in the packet header, and then fill in the timestamp, send the edited audio data packet, compress the audio and video separately through the encoder, then implement the detection of the transmission quality of the link, and set the matching encoding parameters to the video encoding module according to the link transmission quality, set a corresponding relationship between the audio and video format and the minimum bit rate, and the bit rate value in this corresponding relationship can be manually set according to the scene requirements, for example: V_min_UHD=6Mbps:2880*1620; V_min_UHD=4Mbps:2304*1296; V_min_UHD=2Mbps:1920*1080; V_min_UHD=1Mbps:960*540; During the audio and video transmission process, when the link quality decreases, if the corresponding encoding output bit rate is lower than the minimum bit rate threshold of the standard, the YUV of the signal source is automatically transformed proportionally according to the pre-set video change algorithm. Conversely, when the link quality changes from poor to good, the frame size is gradually increased according to the corresponding relationship between the video standard and the minimum bit rate. The original standard and the transformed standard information are carried in the encoded data package for the receiving end to perform reverse operations. A memory buffer is set at the receiving end of the audio and video data packet, and the received audio and video data packets are placed in the memory buffer, so as to filter out the delay jitter caused by the network, and the received video data packets are inserted into the memory buffer according to the sequence number and subsequence number. If all the data packets after the segmentation of a complete frame of the image have been received, they are decoded and reorganized to form a complete single-frame video data, and are placed in the playback buffer according to the sequence number. If the received single-frame image data is incomplete, the sequence of its sequence number and the sequence number of the currently played data is determined. If it is located in the played data It will continue to wait in the buffer. If it is in front of the played data, it will be simply discarded. When the video playback buffer is full, it will start to play in sequence. The received audio data packet will be decoded and inserted into the memory buffer according to the sequence number. If the sequence number is in front of the played data, it will be simply discarded. When the audio playback buffer is full, it will start to play in sequence. The synchronization of audio and video is guaranteed by the timestamp in the data packet. At the receiving end of the audio and video program, it is first decoded according to the settlement algorithm to obtain the initial YUV, and then the picture is up-converted based on AI enhancement technology to restore the original picture and audio of the acquisition end. Among them, the collected audio and video data packets are transmitted through the network, and VPN networking is used to establish point-to-point communication between the acquisition end and the client under the 4G network. Among them, the local area network is suitable for short-distance transmission and can provide a low-cost and high-bandwidth transmission network. The 4G network is more in line with the needs of long-distance mobile transmission. The local area network is used for data transmission first, and the 4G network is used for transmission when leaving the local area network environment.

[0017] In the present invention, the format is first changed, and then the original data of coding compression is reduced while fully retaining sufficient information of the picture, so that better coding quality can be obtained at the same bit rate. At the video program acquisition end, a suitable picture ratio is automatically selected according to the bandwidth change of the link to perform geometric transformation on the input original picture, and the parameter information of the geometric change is encapsulated into a transmission packet for reference by the program receiving end for decoding and restoration. At the receiving end, reverse decoding is performed according to the coding parameters of the acquisition end, and the original image is restored through an image enhancement algorithm. In this way, not only the effect of the video program will not be degraded, but also the link bandwidth will not be wasted.

[0018] By transforming the image frame, relatively good image quality can be obtained at a low bit rate.

[0019] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for high-quality audio and video transmission under weak links, characterized in that: First, the audio and video signals from the signal source are collected in real time, and the signal source is compressed based on AVC / H.264 or HEVC / H.265 encoding. Then, the transmission quality of the link is detected, and the matching encoding parameters are set to the video encoding module according to the link transmission quality. A corresponding relationship between the audio and video standard and the minimum bit rate is set. During the audio and video transmission process, when the link quality decreases, if the corresponding encoding output bit rate is lower than the minimum bit rate threshold of the standard, the YUV of the signal source is automatically transformed proportionally according to the pre-set video change algorithm. Conversely, when the link quality changes from poor to good, the picture size is gradually increased according to the corresponding relationship between the video standard and the minimum bit rate. The original standard and the transformed standard information are carried in the encoded data package at the same time, which is used for the reverse operation at the receiving end. At the receiving end of the audio and video program, decoding is first performed according to the settlement algorithm to obtain the initial YUV, and then the picture is up-converted based on the AI ​​enhancement technology to restore the original picture and audio at the acquisition end.

2. The method for high-quality audio and video transmission under weak link according to claim 1, characterized in that: The audio and video signals collected from the signal source are sent separately by audio and video, and the audio and video are compressed separately by an encoder.

3. The method for high-quality audio and video transmission under weak link according to claim 2, characterized in that: When sending the video, the compressed video data is cut to reduce the length of the data packet and then packaged. The video data type and the serial number of the data packet are indicated in the packet header. The data packets belonging to the same single-frame video have the same serial number in the packet header, and their subsequence numbers are filled in the packet header in the order of segmentation, and the timestamp is written to send the edited video data packet.

4. The method for high-quality audio and video transmission under weak link according to claim 2, characterized in that: After the audio data is compressed, it is directly packaged, and the audio data type and serial number are marked in the packet header, and then the timestamp is filled in, and the edited audio data packet is sent.

5. The method for high-quality audio and video transmission under weak link according to claim 1, characterized in that: A memory buffer is set at the receiving end of the audio and video data packets, and the received audio and video data packets are placed in the memory buffer, thereby filtering out the delay jitter caused by the network.

6. The method for high-quality audio and video transmission under weak link according to claim 5, characterized in that: Insert the received video data packets into the memory buffer according to the sequence number and subsequence number. If all the data packets after the segmentation of a complete frame of image have been received, decode and reassemble them to form a complete single-frame video data, and put them into the playback buffer according to the sequence number; If the received single-frame image data is incomplete, its serial number is determined in sequence with the serial number of the currently played data. If it is behind the played data, it continues to wait in the buffer. If it is in front of the played data, it is simply discarded. When the video playback buffer is full, it starts to play in sequence.

7. The method for high-quality audio and video transmission under weak link according to claim 5, characterized in that: The received audio data packet is decoded and inserted into the memory buffer according to the serial number. If the serial number is in front of the played data, it is simply discarded. When the audio playback buffer is full, it starts to play in sequence, and the synchronization of audio and video is guaranteed by the timestamp in the data packet.

8. The method for high-quality audio and video transmission under weak link according to claim 1, characterized in that: When transmitting the collected audio and video data packets, network transmission is adopted, and VPN networking is used to establish point-to-point communication between the collection end and the client under the 4G network; Among them, the LAN is suitable for short-distance transmission and can provide a low-cost and high-bandwidth transmission network, while the 4G network is more in line with the needs of long-distance mobile transmission. The LAN is used for data transmission first, and the 4G network is used for transmission when leaving the LAN environment.