Video narrowband transmission method and device under limited bandwidth condition and electronic equipment

Abstract

The invention belongs to the technical field of streaming media video narrow-band transmission, and particularly discloses a video narrow-band transmission method and device under a bandwidth limited condition and electronic equipment, and the method comprises the steps that a transmitting end obtains a target video stream, and decodes the target video stream to obtain a high-resolution original image video stream; extracting structured information of a dynamic target in an original picture video stream picture; and obtaining the packet loss rate of the original picture video stream, carrying out quality reduction transcoding processing on the original picture video stream under the condition that the packet loss rate is greater than 0, generating a low-bit-rate video stream, and sending the low-bit-rate video stream after the quality reduction transcoding processing and the structured information to a receiving end, otherwise, directly sending the original picture video stream and the structured information to the receiving end. And when a receiving end receives the low-bit-rate video stream after quality reduction and transcoding, image amplification and image quality enhancement are carried out on the low-bit-rate video stream by using the structured information. According to the method, the problem of video packet loss and jamming during high-definition narrowband video transmission application can be avoided.

Description

Technical Field

[0001] The present application belongs to the technical field of narrowband transmission of streaming media video, and more specifically, to a method, device and electronic device for narrowband transmission of video under bandwidth-constrained conditions. Background Art

[0002] Surveillance video is widely used in smart cities, regional security, emergency command, maritime police and other fields. With the improvement of the environmental reconnaissance and perception capabilities of unmanned equipment and mobile equipment, more and more different wireless transmission links such as satellite communication, Tiantong or shortwave are added to the original topology network based on Ethernet or optical fiber and other wired video transmission. For the command center, in order to achieve the integration of front-end and back-end real-time video command and manned and unmanned collaborative operations, it is necessary to solve the stability and reliability of narrowband transmission of video streams under bandwidth-limited conditions. Therefore, most of them solve this problem by optimizing the encoding and decoding algorithm to improve the compression ratio and reduce the bit rate of the video stream.

[0003] The current mainstream encoding and decoding algorithms mainly include two categories: bit rate priority and image quality priority. Bit rate priority (Constant bitrate, CBR) uses a fixed bit rate to mainly ensure a constant bit rate, but the transmission quality is unstable, especially when there are many dynamic pictures; Image quality priority uses a dynamic bit rate (variable bitrate, VBR), which allows the encoding bit rate to fluctuate within the bit rate statistics time, thereby ensuring the stability of the encoded image quality, especially when the bit rate is very small relative to static pictures, but it is easy to have a very high bit rate when the image changes greatly.

[0004] However, whether it is CBR, VBR, or the combination of the two modes, the variable bit rate (Adaptive Variable Bit Rate, AVBR) and the constrained variable bit rate (Constrained Variable Bit Rate, CVBR), they are only meaningful under the premise that the bit rate is close to the actual bandwidth. Taking the satellite communication terminal as an example, the available bandwidth is usually about 2M. In addition to other services, the allocated bandwidth for the video channel is often only about 200Kbps, while a high-definition network camera video stream usually has 4M~10Mbps, and 720P also has 1Mbps. In the most extreme case, the compression ratio of H264 / H265 video needs to be increased to nearly 20 times. Therefore, the conventional video compression algorithm optimization method cannot meet the narrowband transmission requirements under bandwidth-constrained conditions. When there are large changes in the picture, it is easy to generate compressed data packets that exceed the limited bandwidth, resulting in packet loss and causing screen distortion and freezes. Summary of the invention

[0005] In view of the defects of the prior art, the purpose of this application is to provide a method, device and electronic device for narrowband video transmission under bandwidth-constrained conditions, aiming to solve the problem that the prior art is prone to generate compressed data packets exceeding the limited bandwidth when performing narrowband transmission under bandwidth-constrained conditions, resulting in packet loss and causing screen distortion and freezes.

[0006] To achieve the above objectives, in a first aspect, the present application provides a method for narrowband video transmission under bandwidth-constrained conditions, which is applied to a transmitting end and includes: Obtaining a target video stream, and decoding the target video stream to obtain a high-resolution original picture video stream; Extracting structural information of dynamic objects in the original video stream; Obtain the packet loss rate of the original picture video stream, perform downgrading transcoding processing on the original picture video stream when the packet loss rate is greater than 0, generate a low-bitrate video stream, and send the low-bitrate video stream after the downgrading transcoding processing and the structured information to the receiving end, otherwise directly send the original picture video stream and the structured information to the receiving end.

[0007] This application decodes and extracts structured information from high-resolution surveillance video streams, and then performs quality reduction transcoding to generate a low-bitrate video stream that complies with the narrowband network environment and sends it to the receiving end, thereby avoiding video packet loss and freeze problems when the network bandwidth is insufficient.

[0008] According to a method for transmitting narrowband video under bandwidth-constrained conditions provided by the present application, the quality-degrading transcoding of the original picture video stream includes: The original picture video stream is scaled by 1 / 2, the frame rate and quantization parameters are adjusted, and the video is re-encoded based on a constant bit rate CBR compression mode to generate a video stream with a target bit rate.

[0009] This application generates a low-bitrate video stream that meets the narrowband network environment and sends it to the receiving end by downgrading the high-resolution surveillance video stream and encoding the video based on the CBR compression mode, thereby avoiding the problem of video packet loss and freeze when the network bandwidth is insufficient.

[0010] According to a method for narrowband video transmission under bandwidth-constrained conditions provided by the present application, the method further includes: Obtaining a packet loss rate of the low bit rate video stream during transmission; Based on the packet loss rate, a target resolution and an encoding output frame rate in a downgraded transcoding process are adjusted.

[0011] This application can automatically adjust the target resolution and encoding output frame rate of the downgraded transcoding processing according to the real-time packet loss rate of the video stream during the transmission process, which can maximize the use of bandwidth and ensure the clarity and smoothness of the video narrowband transmission.

[0012] In a second aspect, the present application provides a method for narrowband video transmission under bandwidth-constrained conditions, which is applied to a receiving end and includes: Receive the original video stream or low-bitrate video stream and structured information sent by the transmitter; The low bit rate video stream is upscaled in resolution and enhanced in image quality based on the structured information.

[0013] After receiving the low-bitrate video stream at the receiving end, the present application uses the structured information in the original resolution picture to enlarge the resolution and enhance the image quality of the low-resolution video stream.

[0014] According to a video narrowband transmission method under bandwidth-constrained conditions provided by the present application, the resolution amplification and image quality enhancement of the original picture video stream based on the structured information includes: Based on the structured information, training an image generator based on a Super-Resolution Generative Adversarial Network (SRGAN) to obtain a trained image generator; Based on the trained image generator and the deep learning-based super-resolution reconstruction technology, the low-bitrate video stream is amplified in resolution and enhanced in image quality.

[0015] This application uses the structured information in the original resolution picture to train the SRGAN-based image generator, and performs super-resolution reconstruction and video restoration without prior knowledge after receiving the low-bitrate video stream at the receiving end. At the same time, the trained image generator is used to generate realistic high-resolution images, improve the image super-resolution reconstruction effect and reduce the computational complexity, improve the real-time performance of real-time video stream processing, and avoid video packet loss and freeze problems when facing high-definition narrowband video transmission applications.

[0016] According to a method for narrowband video transmission under bandwidth-constrained conditions provided by the present application, the image generator based on a generative adversarial network (SRGAN) is trained based on the structured information, including: Initialize the image generator; Initializing a discriminator based on the structured information; Using the low bit rate video stream as input of a picture generator to generate an original resolution image; Extracting second structural information of the dynamic target in the original resolution image, and calculating the Euclidean distance between the second structural information and the structural information; Based on the Euclidean distance, the network weights of the image generator and the discriminator are updated alternately, and the reconstruction capability of the image generator is gradually improved by competing with each other; The image generator after the model converges is obtained as the trained image generator.

[0017] In a third aspect, the present application provides a video narrowband transmission device under bandwidth-constrained conditions, comprising: An acquisition module is used to acquire a target video stream and decode the target video stream to obtain a high-resolution original picture video stream; An extraction module, used for extracting structural information of dynamic objects in the original picture video stream; The sending module is used to obtain the packet loss rate of the original picture video stream, perform downgrading transcoding processing on the original picture video stream when the packet loss rate is greater than 0, generate a low-bitrate video stream, and send the original picture video stream after the downgrading transcoding processing and the structured information to the receiving end, otherwise directly send the original picture video stream and the structured information to the receiving end.

[0018] In a fourth aspect, the present application provides a video narrowband transmission device under bandwidth-constrained conditions, comprising: A receiving module, used for receiving the original picture video stream or low bit rate video stream and structured information sent by the sending end; An upsampling module is used to enlarge the resolution and enhance the image quality of the low bit rate video stream based on the structured information.

[0019] In a fifth aspect, the present application provides an electronic device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the programs stored in the memory are executed, the processor is used to execute the video narrowband transmission method under bandwidth-constrained conditions described in the first aspect or any possible implementation of the first aspect.

[0020] In a sixth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the video narrowband transmission method under bandwidth-constrained conditions described in the first aspect or any possible implementation of the first aspect.

[0021] In a seventh aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the video narrowband transmission method under bandwidth-constrained conditions described in the first aspect or any possible implementation of the first aspect.

[0022] It can be understood that the beneficial effects of the third to seventh aspects mentioned above can be found in the relevant descriptions of the first to second aspects mentioned above, and will not be repeated here.

[0023] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art: (1) This application decodes and extracts structured information from high-resolution surveillance video streams, and then performs quality reduction transcoding to generate low-bitrate video streams that meet narrowband network environments and send them to the receiving end, thereby avoiding the video packet loss and freeze problems that are prone to occur when bandwidth is limited. After the receiving end receives the low-bitrate video stream, the structured information in the original resolution image is used to enlarge the resolution and enhance the image quality of the low-resolution video stream.

[0024] (2) Automatically adjust the target resolution and encoding output frame rate of the downgraded transcoding process according to the real-time packet loss rate of the video stream during transmission, so as to maximize the use of bandwidth and ensure the clarity and smoothness of narrowband video transmission.

[0025] (3) The structured information in the original resolution image is used to train the SRGAN-based image generator. After receiving the low-bitrate video stream at the receiving end, super-resolution reconstruction is performed to restore the video without prior knowledge. At the same time, the trained image generator is used to generate realistic high-resolution images, thereby improving the image super-resolution reconstruction effect and reducing the computational complexity, improving the real-time performance of real-time video stream processing, and avoiding video packet loss and freeze problems in high-definition narrowband video transmission applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 This is one of the flow diagrams of the method for narrowband video transmission under bandwidth-constrained conditions provided by an embodiment of the present application; Figure 2 is a processing flow chart of a low-pass code rate control module provided in an embodiment of the present application; Figure 3 This is the second flow chart of the method for narrowband video transmission under bandwidth-constrained conditions provided in an embodiment of the present application; Figure 4 is an application block diagram of a narrowband transmission system provided by an embodiment of the present application; Figure 5It is a flow chart of the training steps of the SRGAN-based image generator provided in an embodiment of the present application; Figure 6 It is a video stream data transmission processing flow diagram provided by an embodiment of the present application; Figure 7 This is one of the structural schematic diagrams of a video narrowband transmission device under bandwidth-limited conditions provided by an embodiment of the present application; Figure 8 This is the second structural schematic diagram of the video narrowband transmission device under bandwidth-limited conditions provided by an embodiment of the present application; Fig. 9 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0030] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0031] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0032] Next, combine Figure 1-Figure 6 The video narrowband transmission method under bandwidth-constrained conditions provided in an embodiment of the present application is introduced.

[0033] Figure 1 FIG. 1 is one of the flow charts of the method for transmitting video in narrowband under bandwidth-constrained conditions provided by an embodiment of the present application, such as Figure 1 As shown, the method is applied to the sending end and includes the following steps: Step 100, obtaining a target video stream, and decoding the target video stream to obtain a high-resolution original picture video stream; Video is transmitted between the sender and the receiver. Before transmission, the sender first needs to obtain the target video stream.

[0034] Optionally, the target video stream may be any video stream that needs to be transmitted. In one embodiment of the present application, the target video stream is a target monitoring video stream.

[0035] Optionally, the present application does not limit the method for the sender to obtain the target video stream, which may be receiving the target video stream sent by other devices, or directly importing the target video stream to the sender, etc.

[0036] After acquiring the target video stream, the sender obtains the high-resolution original image through decoding.

[0037] In one embodiment of the present application, after the sender obtains the monitoring video stream of the Internet Protocol Camera (IPC) and the like, it decodes it based on H264 / H265 to obtain the original resolution image, such as 1080P@30fps or 1080P@60fps.

[0038] Step 110, extracting structural information of dynamic objects in the original video stream; After acquiring the decoded high-resolution picture, the transmitter extracts the structural information of the dynamic target in the picture.

[0039] Optionally, the structured information includes features such as target semantics, location, and contour.

[0040] In one embodiment of the present application, taking the traffic monitoring screen as an example, the main objects include moving targets such as people and cars. The monitoring video of people and cars can be detected by using a detection algorithm such as YOLOV5, the target contours in the image can be extracted, and then the attribute features can be extracted using the FasterRCNN network.

[0041] The basic structured information form that can define a target is as follows: ,in is multidimensional attribute feature data, is the target object, and M is the target number.

[0042] Step 120, obtain the packet loss rate of the original picture video stream, and perform downgrading transcoding processing on the original picture video stream when the packet loss rate is greater than 0 to generate a low-bitrate video stream, and send the low-bitrate video stream and structured information after the downgrading transcoding processing to the receiving end, otherwise the original picture video stream and structured information are directly sent to the receiving end.

[0043] Since the video transmission method of the present application is completed under narrowband conditions, it is necessary to determine whether the sent video stream can be transmitted normally. The present application makes a judgment by obtaining the packet loss rate of the current bit rate video stream. When the packet loss rate is greater than 0, the video stream is downgraded and transcoded, that is, the resolution and bit rate of the video stream are reduced so that it can be transmitted under narrowband conditions, and then it is sent to the receiving end together with the structured information. If the packet loss rate is not greater than 0, it is directly sent to the receiving end together with the structured information.

[0044] The present application provides a video narrowband transmission method, which decodes and extracts structured information from a high-resolution surveillance video stream, and then performs quality reduction transcoding processing to generate a low-bitrate video stream that complies with the narrowband network environment and sends it to a receiving end, thereby avoiding the problem of video packet loss and freeze when the network bandwidth is insufficient.

[0045] In some embodiments, the downgrading transcoding process of the original picture video stream in step 120 specifically includes: Step 1201 , scale the original picture video stream by 1 / 2, adjust the frame rate and quantization parameters, and re-encode the video based on the constant bit rate CBR compression mode to generate a video stream with a target bit rate.

[0046] This application performs downgrading and transcoding processing on high-resolution surveillance video streams, including 1 / 2 scaling, adjusting frame rate and quantization parameters, and video encoding based on CBR compression mode, to generate a low-bitrate video stream that complies with the narrowband network environment and send it to the receiving end, avoiding video packet loss and freeze problems when the network bandwidth is insufficient.

[0047] In one embodiment of the present application, taking 1080P as an example, the sending end uses the resolution scaling function of the built-in video processing subsystem (VPSS) of HiSilicon to reduce the resolution according to the D1 resolution, reduce the frame rate and increase the quantization parameter (QP) value to achieve resolution and image quality adjustment, and then follow the CBR mode and perform the initial compression to reduce the bit rate from the average 4Mbps to a constant peak value below 500kbps, thereby ensuring that it can pass smoothly in the current-limited communication device, and then send the code stream end-to-end to the narrowband switching device according to the streaming media protocol of the Real-time Transport Protocol (RTP), and finally to the receiving end.

[0048] In some embodiments, the method further comprises: Step 130, obtaining a packet loss rate of a low bit rate video stream during transmission; Step 140: adjusting the target resolution and the encoding output frame rate in the downgraded transcoding process based on the packet loss rate.

[0049] After receiving the low-bit-rate video stream, the receiving end can collect statistics on the packet loss rate of the low-bit-rate video stream during the transmission process and reply to the sending end.

[0050] After receiving the packet loss rate, the sender diagnoses the packet loss rate of the current constant bit rate video stream on the narrowband transmission network, outputs the target resolution to the constant low bit rate stream generation module, and adjusts the encoding output frame rate to achieve the purpose of reducing the bit rate; when the packet loss rate is 0, the target resolution of the constant low bit rate stream generation module and its own encoding output frame rate are increased to achieve maximum utilization of the narrowband bandwidth.

[0051] Figure 2 is a processing flow chart of the low-pass code rate control module provided by this application, such as Figure 2 As shown, in one embodiment of the present application, a low-pass bit rate control module is used to adjust the target resolution and the encoded output frame rate in the downgraded transcoding process. Specifically, assuming that the current packet loss rate is R, the scaling factor is S, and the bit rate is C, if the packet loss rate is greater than 0, the scaling factor is adjusted to S / 2, and then the target resolution is calculated based on the original resolution and the scaling factor, and the resolution is scaled according to the current target resolution. The frame rate and quantization parameter (Quantizer Parameter, QP) value are lowered, and the CBR mode is used for encoding, and the output bit rate is C / 2.

[0052] If the packet loss rate is not greater than 0, adjust the scaling factor to 2S. If 2S>1, adjust the scaling factor to 1 and calculate the target resolution based on the original resolution and the scaling factor. Otherwise, directly calculate the target resolution based on the original resolution and the scaling factor, and then scale according to the current target resolution, increase the frame rate and QP value, use CBR mode for encoding, and output the bit rate 2C.

[0053] The above-mentioned adaptive adjustment method can achieve maximum utilization of narrowband bandwidth.

[0054] Figure 3 FIG. 2 is a flow chart of a method for transmitting video in narrowband under bandwidth-constrained conditions provided in an embodiment of the present application. Figure 3 As shown, the method is applied to the receiving end and includes the following steps: Step 300, receiving the original picture video stream or low bit rate video stream and structured information sent by the sending end; The receiving end is a terminal for receiving video, and can receive the original picture video stream or low bit rate video stream and structured information sent by the sending end in a narrowband environment.

[0055] In one embodiment, both the transmitting end and the receiving end use the HiSilicon 3531DV200 as the main chip for codec and video edge processing. The chip supports H264 and H265 codec functions up to 4K@30fps, and supports built-in 1.5TFlops neural network acceleration capability, supports ResNet, YoloV5 and other models, and can realize the segmentation of target contours in the monitoring screen and extract target features.

[0056] Step 310: Amplify the resolution and enhance the image quality of the low bit rate video stream based on the structured information.

[0057] After receiving the low-bit-rate video stream and structured information, the receiving end can enlarge the resolution and enhance the image quality of the low-bit-rate video stream based on the structured information.

[0058] Figure 4 is an application block diagram of a narrowband transmission system provided in an embodiment of the present application, such as Figure 4 As shown, in one embodiment of the present application, the sending end is a sending end video transcoding device, and the receiving end is a receiving end video enhancement device. The sending end video transcoding device receives the IPC video stream and sends the video stream to the receiving end video enhancement device through a current limiting communication device.

[0059] In some embodiments, step 310 specifically includes: Step 3101, based on the structured information, training an image generator based on a generative adversarial network SRGAN to obtain a trained image generator; Step 3102: Based on the trained image generator and the deep learning-based super-resolution reconstruction technology, the low-bitrate video stream is amplified in resolution and enhanced in image quality.

[0060] In one embodiment of the present application, the receiving end receives a complete low-resolution video stream, decodes it through the built-in video decoder (VDEC) function of Hi3531DV200, and generates a D1 resolution picture. At this time, the resolution is enlarged through VPSS, but the picture is relatively blurred. At this time, due to the loss of many pixels due to low-resolution compression at the sending end, it is impossible to achieve better detail supplementation by only using traditional interpolation processing.

[0061] This application adopts super-resolution reconstruction technology based on deep learning. Through training with a large amount of real data, inference and filling of details are performed to make blurred images clearer. The super-resolution reconstruction technology of images refers to restoring a given low-resolution image into a corresponding high-resolution image through a specific algorithm.

[0062] In one embodiment of the present application, the media processing software platform (MPP) of HiSilicon 3531DV200 supports models such as TensorFlow and ResNet, and can quantize and prune the trained model for the embedded platform, remove components in the model that have little effect on the results and redundant parameters in the model, achieve lightweight transplantation, and then use algorithms to enhance the image. The super-resolution reconstruction method based on deep learning aims to restore the low-resolution real-scene degraded image to a high-resolution reconstructed image to the greatest extent possible while ensuring a certain degree of real-time performance.

[0063] Since the scenes facing the surveillance video are not fixed, there is mostly no prior knowledge when performing super-resolution, so the algorithm enhancement often has limited effect. In order to better improve the clarity of super-resolution enhancement, this application uses the generative adversarial network SRGAN to generate realistic high-resolution images. After the SRGAN-based image generator module generates images of the original resolution size, a discriminator is used to identify the authenticity. The most critical loss function of the adversarial network is the content loss. It is necessary to calculate the difference between the generated image and the real image in the feature space to identify the authenticity.

[0064] In order to train the SRGAN-based image generator, the content loss can be represented based on the Euclidean distance between the structured information sent by the sender and the reconstructed structured information. This method can be used to train the generator through real monitoring images, and at the same time, the real-time and universality of the image changes can be guaranteed. The structured information of the real image and the compressed image are transmitted to the receiving end in parallel, which can ensure that when the image is super-resolved and the details are enhanced, the image super-resolution reconstruction effect is improved and the computational complexity is reduced, thereby improving the real-time performance of real-time video stream processing.

[0065] In some embodiments, step 3101 specifically includes: Step 31011, initialize the image generator; Step 31012, initializing the discriminator based on the structured information; Step 31013, using the low bit rate video stream as input of the picture generator to generate an original resolution image; Step 31014, extracting the second structural information of the dynamic target in the original resolution image, and calculating the Euclidean distance between the second structural information and the structural information; Step 31015, based on the Euclidean distance, the network weights of the image generator and the discriminator are updated alternately, and the reconstruction capability of the image generator is gradually improved by competing with each other; Step 31016, obtain the image generator after the model converges as the trained image generator.

[0066] Figure 5 This is a flow chart of the training steps of the SRGAN-based image generator provided in the embodiment of the present application, such as Figure 5 As shown, in one embodiment of the present application, the image generator training steps are as follows: Step 1: Initialize the generator at the receiving end video enhancement device; Step 2: Initialize the discriminator according to the structural feature information data of the original image received in real time; Step 3: Take the received low-resolution video stream as the input of the generator to generate the original resolution image.

[0067] Step 4: Calculate the Euclidean distance of the image with the original resolution to determine its authenticity.

[0068] Step 5: Alternately update the network weights of the generator and the discriminator, and gradually improve the reconstruction ability of the generator by competing with each other.

[0069] Step 6: Get the generator after the model converges.

[0070] Figure 6 is a video stream data transmission processing flow diagram provided by an embodiment of the present application, such as Figure 6 As shown, in one embodiment of the present application, the transmitting end includes an original picture acquisition module, a structured extraction module, a constant low bit rate stream generation module and a low-pass bit rate control module, and the receiving end includes a video stream decoding module and an SR reconstruction module, wherein: The original picture acquisition module completes the extraction and decoding of the encoded high-resolution video stream to obtain the high-resolution original picture; The structured extraction module extracts the structured information of the target in the monitoring image, including the target semantics, location, and contour features; The constant low bit rate stream generation module completes the quality reduction processing of high-resolution images and uses CBR mode to encode and output low bit rate video streams; The low-pass bit rate control module diagnoses the packet loss rate of the current constant bit rate video stream in real time, dynamically adjusts the downsampling resolution, and adjusts the frame rate and QP quantization parameters of the encoded output; The video stream decoding module completes the decoding function of the received low-resolution video stream with a constant bit rate and submits the picture to the SR reconstruction module; The SR reconstruction module realizes resolution amplification and image quality enhancement of the decoded image.

[0071] Figure 7 is one of the structural schematic diagrams of a video narrowband transmission device under bandwidth-limited conditions provided by an embodiment of the present application, such as Figure 7 As shown, the device includes an acquisition module 710, an extraction module 720 and a sending module 730, wherein: The acquisition module 710 is used to acquire the target video stream and decode the target video stream to obtain a high-resolution original picture video stream; Extraction module 720, used to extract structured information of dynamic objects in the original picture video stream; The sending module 730 is used to obtain the packet loss rate of the original picture video stream, and when the packet loss rate is greater than 0, perform downgrading transcoding on the original picture video stream to generate a low-bitrate video stream, and send the downgraded transcoded original picture video stream and structured information to the receiving end, otherwise the original picture video stream and structured information are directly sent to the receiving end.

[0072] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0073] Figure 8 FIG. 2 is a second structural diagram of a video narrowband transmission device under bandwidth-constrained conditions provided in an embodiment of the present application. Figure 8 As shown, the device includes a receiving module 810 and an up-sampling module 820, wherein: The receiving module 810 is used to receive the original picture video stream or the low bit rate video stream and the structured information sent by the sending end; The up-sampling module 820 is used to enlarge the resolution and enhance the image quality of the low-bitrate video stream based on the structured information.

[0074] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0075] Based on the method in the above embodiment, Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, an embodiment of the present application provides an electronic device, which may include: a processor (processor) 910, a communication interface (CommunicationsInterface) 920, a memory (memory) 930 and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call the logic instructions in the memory 930 to execute the video narrowband transmission method under bandwidth-limited conditions in the above embodiment.

[0076] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present application can be essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the video narrowband transmission method under bandwidth-limited conditions described in various embodiments of the present application.

[0077] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the video narrowband transmission method under bandwidth-constrained conditions in the above embodiment.

[0078] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the video narrowband transmission method under bandwidth-constrained conditions in the above embodiment.

[0079] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0080] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0081] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0082] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0083] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for transmitting video in narrowband under bandwidth-constrained conditions, characterized in that: The method is applied to the sending end and includes: Obtaining a target video stream, and decoding the target video stream to obtain a high-resolution original picture video stream; Extracting structural information of dynamic objects in the original video stream; Obtain the packet loss rate of the original picture video stream, perform downgrading transcoding processing on the original picture video stream when the packet loss rate is greater than 0, generate a low-bitrate video stream, and send the low-bitrate video stream after the downgrading transcoding processing and the structured information to the receiving end, otherwise directly send the original picture video stream and the structured information to the receiving end.

2. The method for transmitting video in narrowband under bandwidth-constrained conditions according to claim 1, characterized in that: The downgrading and transcoding the original picture video stream includes: The original picture video stream is scaled by 1 / 2, the frame rate and quantization parameters are adjusted, and the video is re-encoded based on a constant bit rate CBR compression mode to generate a video stream with a target bit rate.

3. The method for transmitting video in narrowband under bandwidth-constrained conditions according to claim 2, characterized in that: The method further comprises: Obtaining a packet loss rate of the low bit rate video stream during transmission; Based on the packet loss rate, a target resolution and an encoding output frame rate in a downgraded transcoding process are adjusted.

4. A method for transmitting video in narrowband under bandwidth-constrained conditions, characterized in that: The method is applied to the receiving end and includes: Receive the original video stream or low-bitrate video stream and structured information sent by the transmitter; The low bit rate video stream is upscaled in resolution and enhanced in image quality based on the structured information.

5. The method for transmitting video in narrowband under bandwidth-constrained conditions according to claim 4, characterized in that: The step of enlarging the resolution and enhancing the image quality of the low bit rate video stream based on the structured information includes: Based on the structured information, training an image generator based on a generative adversarial network (SRGAN) to obtain a trained image generator; Based on the trained image generator and the deep learning-based super-resolution reconstruction technology, the low-bitrate video stream is amplified in resolution and enhanced in image quality.

6. The method for transmitting video in narrowband under bandwidth-constrained conditions according to claim 5, characterized in that: The step of training a picture generator based on a generative adversarial network (SRGAN) based on the structured information includes: Initialize the image generator; Initializing a discriminator based on the structured information; Using the low bit rate video stream as input of a picture generator to generate an original resolution image; Extracting second structural information of the dynamic target in the original resolution image, and calculating the Euclidean distance between the second structural information and the structural information; Based on the Euclidean distance, the network weights of the image generator and the discriminator are updated alternately, and the reconstruction capability of the image generator is gradually improved by competing with each other; The image generator after the model converges is obtained as the trained image generator.

7. A video narrowband transmission device under bandwidth-limited conditions, characterized in that: include: An acquisition module is used to acquire a target video stream and decode the target video stream to obtain a high-resolution original picture video stream; An extraction module, used for extracting structural information of dynamic objects in the original picture video stream; The sending module is used to obtain the packet loss rate of the original picture video stream, perform downgrading transcoding processing on the original picture video stream when the packet loss rate is greater than 0, generate a low-bitrate video stream, and send the original picture video stream after the downgrading transcoding processing and the structured information to the receiving end, otherwise directly send the original picture video stream and the structured information to the receiving end.

8. A video narrowband transmission device under bandwidth-constrained conditions, characterized in that: include: A receiving module, used for receiving the original picture video stream or low bit rate video stream and structured information sent by the sending end; An upsampling module is used to enlarge the resolution and enhance the image quality of the low bit rate video stream based on the structured information.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the video narrowband transmission method under bandwidth-constrained conditions as described in any one of claims 1-6.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is enabled to execute the method for narrowband video transmission under bandwidth-constrained conditions as claimed in any one of claims 1 to 6.

Images