5G tablet computer high-definition video real-time transmission processing method and system

Abstract

The invention relates to the technical field of image communication, in particular to a 5G tablet computer high-definition video real-time transmission processing method and system, and the method comprises the steps: collecting video frames of a to-be-shot scene in real time, and obtaining each frame of image in a buffer queue in the video frames; screening feature points; dividing each frame of image into each feature block, obtaining a motion feature value of each feature block, and dividing each feature block into a dynamic feature block or a static feature block; analyzing whether a combined region of the continuous dynamic feature blocks is a dynamic feature region or not; when compression coding is carried out on each frame of image, setting a value of a quantization parameter for each feature block; and packaging and transmitting the compressed and coded image, decoding and restoring each frame of image, and displaying the image on the tablet computer in real time according to an original time sequence. The invention aims to dynamically adjust the coding strategy according to the dynamic characteristics of different areas, and can significantly reduce the code rate while ensuring the dynamic content transmission quality, thereby improving the video transmission effect.

Description

Technical Field

[0001] This application relates to the field of image communication technology, and particularly to a method and system for real-time transmission and processing of high-definition videos on 5G tablets. Background Art

[0002] Driven by the rapid development of 5G communication technology and the continuous improvement of the performance of mobile terminals, the real-time transmission of high-definition videos has gradually expanded from traditional fixed devices to portable terminals. In particular, mobile devices such as tablets are increasingly widely used in scenarios such as distance education, telemedicine, and enterprise meetings. Thanks to the high bandwidth, low latency, and high reliability provided by 5G networks, the real-time transmission of high-definition videos on devices such as tablets has become feasible and has become an important technical means to improve user experience and service quality.

[0003] However, in typical application scenarios such as remote meetings and online classes, most of the background of the video remains unchanged for a long time, and only a small part of the area changes dynamically. Traditional real-time video transmission methods usually perform complete encoding and transmission on each frame, and cannot dynamically adjust the local data compression rate, resulting in the transmission of a large amount of redundant data, which not only increases the burden on the network bandwidth, but also causes an increase in transmission delay, reducing the stability and transmission efficiency in a weak network environment. Especially on resource-constrained mobile devices, this problem is more obvious. Summary of the Invention

[0004] In view of the above, it is necessary to provide a method and system for real-time transmission and processing of high-definition videos on 5G tablets. Compared with the traditional method for real-time transmission and processing of high-definition videos on 5G tablets, according to the dynamic characteristics of different regions, the encoding strategy is dynamically adjusted, which can significantly reduce the bit rate while ensuring the transmission quality of dynamic content, thereby improving the video transmission effect: In a first aspect, an embodiment of the present application provides a method for real-time transmission and processing of high-definition videos on 5G tablets, the method including the following steps: Real-time collect video frames of the scene to be photographed, and obtain each frame image in the buffer queue in the video frames, where the image is an RGB image; Screen each feature point from the pixel points on each frame image through the pixel value differences on each channel between each pixel point on each frame image and its preset neighboring pixel points; Divide each frame image evenly into respective feature blocks, use the optical flow method to obtain the optical flow vectors of each feature point, and obtain the motion feature value of each feature block through the dispersion degree and average level of the magnitudes of the optical flow vectors of all feature points within each feature block. By comparing the optical flow vectors of each feature point within each feature block with the motion feature value, divide each feature block into a dynamic feature block or a static feature block; evaluate whether a scene mutation has occurred after each frame image through the pixel value differences of pixel points at the same positions on each channel between each frame image and a preset number of consecutive frames of images after it; merge consecutive dynamic feature blocks. When a scene mutation has occurred in the frame image where each merged region is located compared to the previous frame image, perform the division operation of dynamic feature blocks and static feature blocks on each merged region again, and divide the merged region regarded as a dynamic feature block into a dynamic feature region; When performing compression encoding on each frame image using high-efficiency video coding, set the value of the quantization parameter for each feature block belonging to a static feature block, a dynamic feature block, and a dynamic feature region respectively according to the division result of each feature block; Pack and transmit the compressed and encoded image. The receiving end decodes and restores each frame image and displays it on the tablet computer in real time according to the original time sequence.

[0005] In one embodiment, the screening of each feature point from the pixel points on each frame image includes: Calculate the mean value of the pixel values of each pixel point on all channels, and use the arithmetic mean of all the mean values of each pixel point and its preset neighboring pixel points as the feature point judgment threshold of each pixel point; Calculate the deviation value of the mean value between each pixel point and each of its preset neighboring pixel points, count the number of all the deviation values of each pixel point that are greater than the feature point judgment threshold. When the number is greater than a preset positive integer, regard each pixel point as a feature point.

[0006] In one embodiment, the method for obtaining the motion feature value is: Calculate the average value of the magnitudes of the optical flow vectors of all feature points within each feature block; The motion feature value is the sum of the average value and the dispersion degree.

[0007] In one embodiment, the process of dividing each feature block into a dynamic feature block or a static feature block is: By comparing the magnitude of the optical flow vector of each feature point within each feature block with the motion feature value, screen each motion feature point from the feature points within each feature block; when the proportion of the motion feature points within each feature block among all feature points is greater than or equal to a preset value, divide each feature block into a dynamic feature block, otherwise, divide it into a static feature block.

[0008] In one embodiment, the method for screening the motion feature points is as follows: The feature points with the magnitude of the optical flow vector in each feature block greater than the motion feature value are used as the motion feature points.

[0009] In one embodiment, the method for evaluating whether a scene mutation has occurred after each frame of image is as follows: The average pixel difference between each frame of image and its subsequent frame of image is obtained by the difference amount of the means of the pixel points at the same positions between each frame of image and its subsequent frame of image; When the normalized values of the average pixel differences calculated between each frame of image and the subsequent consecutive preset number of frames of images are all greater than the preset pixel difference threshold, it is determined that a scene mutation has occurred after each frame of image.

[0010] In one embodiment, the average pixel difference is the mean of the difference amounts corresponding to all pixel points on each frame of image.

[0011] In one embodiment, setting the values of the quantization parameters for each feature block belonging to the static feature block, the dynamic feature block, and the dynamic feature region includes: Calculate the difference value between the magnitude of the optical flow vector of each feature point in each feature block and the average value, and denote the mean of the difference values of all feature points in each feature block as the motion mean; For a feature block with 0 feature points, set the value of the quantization parameter to a preset first value; For a feature block belonging to the static feature block, calculate the product of the normalized value of the motion mean of the feature block and a preset first sensitivity coefficient, and set the value of the quantization parameter to the difference between a preset second value and the product; For a feature block belonging to the dynamic feature block and not belonging to the dynamic feature region, calculate the product value of the normalized value of the motion mean of the feature block and a preset second sensitivity coefficient, and set the value of the quantization parameter to the difference between a preset third value and the product value; For a feature block belonging to the dynamic feature region, set the value of the quantization parameter to a preset fourth value.

[0012] In one embodiment, the magnitude relationship among the preset first value, the preset second value, the preset third value, and the preset fourth value is: the preset first number > the preset second value > the preset third value > the preset fourth value.

[0013] In a second aspect, the embodiments of the present application further provide a 5G tablet high-definition video real-time transmission and processing system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the 5G tablet high-definition video real-time transmission and processing method described in any one of the above are implemented.

[0014] The present application has at least the following beneficial effects: By screening feature points based on pixel value differences, the present application can adaptively screen feature points according to the characteristics of the image, improving the reliability and accuracy of feature point screening, which is beneficial for subsequent dynamic detection based on feature points; dividing the image into feature blocks enables local analysis of the image, more precisely distinguishing static and dynamic regions in the image. Through the calculation of optical flow method and motion feature values, the feature blocks can be accurately divided into dynamic or static, enabling the subsequent encoding process to select different encoding strategies according to the dynamicity of the region, thereby optimizing the encoding efficiency and reducing the transmission of redundant data; Furthermore, by comparing the pixel value differences between each frame of the image and subsequent multiple frames of the image, global picture mutations can be detected. At the same time, through continuous comparison of multiple frames, the situation of misjudging a picture mutation due to short-term image jitter or local changes can be effectively avoided, thereby improving the accuracy of picture mutation detection; when a picture mutation occurs in each frame of the image compared to the previous frame, the merged regions of continuous dynamic regions in each frame of the image are dynamically and statically divided again, the differences between the merged regions in each frame of the image compared to the front and rear frames are analyzed, the dynamic characteristics of the merged regions are evaluated, and the merged regions with significant differences compared to both the front and rear frames are separately marked, which can perform more refined processing on the marked regions during the encoding process to ensure high-quality transmission of dynamic content; Furthermore, by dynamically adjusting the encoding strategy according to the dynamic characteristics of different regions, while ensuring the transmission quality of dynamic content, the bit rate can be significantly reduced, thereby improving the video transmission effect. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0016] Figure 1 It is a flowchart of the steps of the 5G tablet high-definition video real-time transmission and processing method provided by an embodiment of the present application; Figure 2 It is a schematic diagram of the division process of dynamic feature blocks and static feature blocks; Figure 3 Schematic diagram of the setting process for quantifying parameter values. Specific implementation manners

[0017] In the description of the embodiments of the present application, words such as "exemplary", "or", "for example", etc. are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary", "or", "for example" is intended to present relevant concepts in a specific manner.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. It should be understood that unless otherwise stated in this application, " / " means "or".

[0019] In addition, it should be noted that the terms "first" and "second" in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] The following specifically describes the specific solutions of the 5G tablet high-definition video real-time transmission processing method and system provided by this application in conjunction with the accompanying drawings.

[0021] Please refer to Figure 1 , which shows the step flowchart of the 5G tablet high-definition video real-time transmission processing method provided by an embodiment of this application. The method includes the following steps: Step 1, collect video frames of the scene to be photographed in real time, and obtain each frame image in the buffer queue of the video frames, where the image is an RGB image.

[0022] In this embodiment, taking the real-time transmission of high-definition video in the application scenarios of enterprise remote meetings and online classes as an example, the scene to be photographed is collected in real time through the built-in or external camera module of the tablet computer to obtain continuous video frames.

[0023] Perform format unification processing on the collected video frames, and uniformly convert the original video streams collected by different hardware into RGB format to ensure the compatibility and efficiency of subsequent processing.

[0024] Adjust the resolution of the video frames, and output the size of the frames according to the actual network transmission capacity and the computing resources of the tablet to achieve a balance between transmission load and display clarity.

[0025] For common problems in the captured images, such as insufficient illumination, blurred edges, and image noise, image enhancement processing is performed, specifically Gaussian filtering for denoising, edge-preserving smoothing operation, and sharpening enhancement, so as to improve the clarity and visual quality of the video. Gaussian filtering for denoising, edge-preserving smoothing operation, and sharpening enhancement are well-known technologies and will not be elaborated in this application.

[0026] For subsequent video feature difference detection and scene complexity judgment, the enhanced consecutive several frames of images are stored in the cache queue by using the frame buffer mechanism, which is used to analyze the local feature differences between adjacent frames to ensure the real-time performance and accuracy of the difference detection. Among them, the frame buffer mechanism is a well-known technology and will not be elaborated in this application.

[0027] Step 2: Analyze the local changes between adjacent frame images by the optical flow method, perform compression encoding adaptively according to the features of different regions, and pack and transmit the compressed encoded video data sequence.

[0028] In typical application scenarios such as remote meetings and online classes, there are often significant redundant features in the video content itself. Specifically, most of the backgrounds in the video remain unchanged for a long time, and only a small part of the regions will change dynamically, such as the postures and expressions of people, the movement and drawing of the cursor. In addition, the switching of PPT content will cause a global picture mutation at a certain moment. Traditional video real-time transmission methods usually perform complete encoding and transmission on each frame of image, resulting in repeated transmission of a large amount of redundant data and affecting the performance of real-time transmission.

[0029] Since the image contains rich features, such as people, PPT graphics and texts, and the conference room background, etc., when the optical flow method is used to detect the differences between image frames, it is first necessary to select feature points as tracking targets, and then construct equations through the gradient changes of the feature points in the x, y, and z three dimensions. Among them, x and y respectively represent the horizontal and vertical dimensions of a single frame image, and t represents the time dimension of different frame images. Finally, the optical flow vector of the feature points is calculated through adjacent frame images to achieve dynamic detection. However, this method is not sensitive enough to large-scale movements. Therefore, in order to more finely distinguish static regions and dynamic regions, in this embodiment, each frame of image is evenly divided into each feature block, and analyzed from local to global, so as to perform more detailed detection on the entire image.

[0030] In this embodiment, the size of the feature block is , and the size of the feature block is preset manually. The implementer can set it by himself / herself, and this application does not make special restrictions. For feature blocks with a size less than , the mirror filling method is used to fill it to . The mirror filling method is a well-known technology and will not be elaborated in this application.

[0031] Step 2.1: Screen each feature point from the pixel points on each frame of image according to the pixel value differences on each channel between each pixel point and its preset neighboring pixel points.

[0032] When the feature point detection algorithm selects feature points, it determines whether a pixel point is a feature point by comparing the pixel values between a pixel point and the pixel points in its neighborhood. The feature point detection algorithm relies on a preset fixed threshold for feature point detection, and the fixed threshold is used regardless of the regional brightness.

[0033] However, when detecting feature points for each frame of image in a video, sudden changes in the picture cause huge changes in the global pixel point features. If the same threshold is used for feature point detection for different pictures, it will cause missed detection of feature points, thus affecting the accuracy of subsequent dynamic region detection. Based on the above analysis, the threshold for evaluating whether each pixel point is a feature point in the feature point detection algorithm is obtained according to the pixel value differences on each channel between each pixel point and its preset neighboring pixel points. The specific process is as follows: For any pixel point, calculate the mean value of the pixel values of the any pixel point on all channels, and take the arithmetic mean of all the mean values of the any pixel point and its preset neighboring pixel points as the feature point judgment threshold for the any pixel point, that is, the threshold for evaluating whether the any pixel point is a feature point in the feature point detection algorithm.

[0034] In this embodiment, the pixel points within the neighborhood centered on the any pixel point are used as the preset neighboring pixel points of the any pixel point. The value of n is 4, and the value of n is preset manually. The implementer can set it by himself, and this application does not make special restrictions.

[0035] Further, it is determined whether the any pixel point is a feature point through the feature point judgment threshold. The specific process is as follows: Calculate the deviation value of the mean value between the any pixel point and each of its preset neighboring pixel points, count the number of the deviation values greater than the feature point judgment threshold among all the deviation values of the any pixel point, and when the number is greater than the preset positive integer, take the any pixel point as a feature point.

[0036] In this embodiment, a circle is drawn with the any pixel point as the center and a radius of 3, and 16 pixel points are taken on the circle, which are respectively used as each of the preset neighboring pixel points of the any pixel point.

[0037] In this embodiment, the deviation value between the mean values is the absolute value of the difference.

[0038] In this embodiment, the value of the preset number is 9, and the value of the preset number is preset manually. The implementer can set it according to the actual situation by himself, and this application does not make special restrictions.

[0039] In this embodiment, the feature point detection algorithm is the FAST (Features from Accelerated Segment Test) feature point detection algorithm. The FAST feature point detection algorithm is a well-known technology and will not be elaborated in this application.

[0040] When determining whether each pixel point is a feature point, by adaptively setting the threshold, feature points can be adaptively selected according to the characteristics of the image in different image frames, making the screening of feature points more reliable.

[0041] Step 2.2: Use the optical flow method to obtain the optical flow vectors of each feature point. Obtain the motion feature value of each feature block through the dispersion degree and average level of the modulus lengths of the optical flow vectors of all feature points in each feature block. By comparing the optical flow vectors of each feature point in each feature block with the motion feature value, each feature block is divided into a dynamic feature block or a static feature block.

[0042] Use the optical flow method to obtain the optical flow vectors of each feature point in each feature block. In this embodiment, the optical flow method is the LK (Lucas-Kanade) optical flow method. The LK optical flow method is a well-known technology and will not be elaborated in this application.

[0043] Obtain the motion feature value of each feature block through the dispersion degree and average level of the modulus lengths of the optical flow vectors of all feature points in each feature block. Specifically: Calculate the average value of the modulus lengths of the optical flow vectors of all feature points in each feature block; take the sum of the dispersion degree of the modulus lengths of the optical flow vectors of all feature points in each feature block and the average value as the motion feature value of each feature block.

[0044] In this embodiment, the dispersion degree is the variance. As other implementation manners, on the basis of being able to measure the uneven distribution degree of the modulus lengths of the optical flow vectors of all feature points in each feature block, the implementer can adopt other existing technologies, such as the standard deviation, coefficient of variation, etc. This application does not make special restrictions.

[0045] Further, regard each feature point whose modulus length of the optical flow vector in each feature block is greater than the motion feature value as each motion feature point. When the proportion of the motion feature points in each feature block among all feature points is greater than or equal to the preset value, divide the feature block into a dynamic feature block; otherwise, divide the feature block into a static feature block. The schematic diagram of the division process of the dynamic feature block and the static feature block is as Figure 2 shown.

[0046] In this embodiment, the value of the preset value is 0.3. The value of the preset value is preset manually, and the implementer can set it by himself. This application does not make special restrictions.

[0047] Step 2.3: Evaluate whether there is a sudden change in the picture after each frame of image by the pixel value differences of the pixel points at the same positions on each channel between each frame of image and the subsequent continuous preset number of frames of images.

[0048] Evaluate whether there is a sudden change in the picture after each frame of image by the pixel value differences of the pixel points at the same positions on each channel between each frame of image and the subsequent continuous preset number of frames of images. The specific process is as follows: Obtain the average pixel difference between each frame of image and its subsequent frame of image through the difference amount between the mean values of the pixel points at the same positions on each channel between each frame of image and its subsequent frame of image. The expression is: ; where represents the average pixel difference between the t-th frame of image and the (t + 1)-th frame of image; H represents the number of pixel points on a single frame of image; and respectively represent the mean values of the i-th pixel point on the (t + 1)-th frame and the t-th frame of image; represents the absolute value operation.

[0049] It should be noted that: the more likely there is a sudden change in the picture between two adjacent frames of images, the greater the pixel value difference between the pixel points on the two adjacent frames of images on each channel, and the greater the calculated average pixel difference.

[0050] To avoid errors caused by video picture jitter, calculate the average value pixel difference between each frame of image and the subsequent continuous preset L frames of images. When the normalized values of the L calculated average pixel differences are all greater than the preset pixel difference threshold, it is determined that there is a sudden change in the picture after each frame of image.

[0051] In this embodiment, the values of L and the preset pixel difference threshold are 3 and 0.5 respectively. The values of L and the preset pixel difference threshold are both preset manually, and the implementer can set them according to the actual situation. This application does not make special restrictions.

[0052] In this embodiment, the decimal scaling normalization method is used to obtain the normalized value of the average pixel difference. The decimal scaling normalization method is a well-known technology, and this application will not elaborate further.

[0053] If it is determined that there is a sudden change in the picture after any frame of image, then the adjacent next frame of image of the any frame of image has a sudden change compared with the any frame of image.

[0054] Step 2.4: Merge the continuous dynamic feature blocks. When the frame image where each merged area is located has a sudden change compared with the previous frame of image, perform the division operation of dynamic feature blocks and static feature blocks on each merged area again, and divide the merged area as the dynamic feature block into a dynamic feature area.

[0055] After the static and dynamic partitioning of the feature blocks, the continuous feature blocks on each frame of the image are merged to obtain each merged region, denoted as each preliminary ROI region. For any preliminary ROI region, if the image where the preliminary ROI region is located has a scene mutation compared to the previous frame image, then according to the method of dividing each feature block into dynamic feature blocks or static feature blocks, the any preliminary ROI region is processed. During the processing, only the feature blocks are replaced with the preliminary ROI regions. If the any preliminary ROI region can still be divided into dynamic feature blocks, the any preliminary ROI region is denoted as a dynamic feature region. During encoding, incremental encoding is performed on the dynamic feature regions to ensure that high-quality transmission can be maintained when the dynamic feature regions are compressed, encoded, and transmitted.

[0056] Step 2.5, when using High Efficiency Video Coding to perform compression encoding on each frame of the image, according to the partitioning results of each feature block, the value of the quantization parameter is set for each feature block belonging to the static feature blocks, dynamic feature blocks, and dynamic feature regions respectively.

[0057] After classifying the feature blocks on each frame of the image, use the High Efficiency Video Coding method for compression encoding, and different degrees of compression are achieved by adjusting the value of the quantization parameter QP in the High Efficiency Video Coding.

[0058] The value range of the quantization parameter QP is [0, 51]. The larger the value of the quantization parameter QP, the better the compression effect, but the higher the loss. A higher compression degree is used for the static region to improve the transmission efficiency, and a lower compression degree is used for the dynamic region to ensure the quality of the picture during transmission.

[0059] High Efficiency Video Coding supports setting the QP value for the Coding Unit (encoding unit) separately. When using High Efficiency Video Coding to perform compression encoding on an image, set the size of the encoding unit to . According to the partitioning results of each feature block, the value of the quantization parameter is set for each feature block belonging to the static feature blocks, dynamic feature blocks, and dynamic feature regions respectively, specifically: ; where represents the value of the quantization parameter of the feature block f; N1, N2, N3, N4 respectively represent the preset first value, preset second value, preset third value, and preset fourth value; represents the feature block f; A1 represents the set of feature blocks with 0 feature points; A2 represents the set of feature blocks belonging to the static feature blocks; A3 represents the set of feature blocks belonging to the dynamic feature blocks and not belonging to the dynamic feature regions; A4 represents the set of feature blocks belonging to the dynamic feature regions; ε1, ε2 respectively represent the preset first sensitivity coefficient and preset second sensitivity coefficient; Denote the magnitude of the optical flow vector of the k-th feature point within the feature block f; Denote the average value of the magnitudes of the optical flow vectors of all feature points within the feature block f; K denotes the number of feature points within the feature block f; Denote the absolute value operation; norm( ) denotes the normalization operation. Among them, the magnitude relationship of N1, N2, N3, and N4 is: N1 > N2 > N3 > N4. It is used to reflect the complexity of the motion of the feature points within the feature block f. The larger the value, the greater the motion change within the feature block f. On the contrary, it indicates that the motion change within the feature block f is smaller. ε1 and ε2 are used to limit the sensitivity to the motion change within the feature block f. Denote as the motion mean value.

[0060] In this embodiment, the values of N1, N2, N3, and N4 are 45, 38, 22, and 15 respectively, and the values of ε1 and ε2 are 3 and 2 respectively. The values of N1, N2, N3, N4, ε1, and ε2 can be preset manually. The implementer can set them according to the actual situation, and this application does not make special restrictions.

[0061] In this embodiment, the Min - Max normalization method is used to implement the normalization operation. The Min - Max normalization method is a well - known technology, and this application will not elaborate further.

[0062] By adaptively adjusting the values of the quantization parameters, it is possible to dynamically adjust the resource allocation before video data transmission, optimize the video transmission efficiency and quality. In areas where feature points cannot be detected, a high compression ratio is used to further reduce data redundancy. At the same time, in order to ensure that the details of the dynamic feature area are retained during transmission, a low coding rate is additionally used for incremental coding of the dynamic feature area to ensure its picture quality. The schematic diagram of the setting process of the quantization parameter values is as Figure 3 shown.

[0063] According to the adaptive quantization parameters, in scenarios such as remote meetings and online classes with a large number of static pictures, the bit rate can be significantly reduced and high - quality transmission of the dynamic area can be ensured. This method is especially suitable for real - time video transmission of mobile devices such as 5G tablets.

[0064] Through the values of the quantization parameters of each feature block, high - efficiency video coding is used to compress and encode each frame of the image to obtain a video data sequence. Then, the video data sequence is encapsulated into a network extraction layer unit, and the video data sequence is packaged into the MP4 format and transmitted through the HTTP protocol. The packaging and transmission of the video data sequence are well - known technologies, and this application will not elaborate on the specific process.

[0065] Step 3, the receiving end decodes the received compressed image, restores each frame of the image, and displays it on the tablet in real - time according to the original time sequence.

[0066] The receiving end receives the video data packets transmitted by the sending end in real time through the 5G communication module. To ensure the continuity and stability of video playback, first, the received video frame data is stored and sorted through the receiving buffer module. The receiving buffer module has a frame reordering function and a data integrity detection mechanism, which can correct and compensate for packet loss or frame order disorder. For example, timestamps, frame numbers, etc. are used to ensure the correct reconstruction of the data stream.

[0067] After caching, the received video data is restored. For static regions, the content of the current frame is directly reconstructed by referring to the previous frame image; for dynamically changing regions, differential information is used for local image update, so as to completely restore the content of the current frame. The decoding process supports parallel decoding and asynchronous buffering to ensure real-time decoding under limited computing resources.

[0068] The decoded video frames are played and displayed on the tablet computer in the original time sequence.

[0069] Based on the same inventive concept as the above method, the embodiment of the present application also provides a 5G tablet high-definition video real-time transmission processing system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above 5G tablet high-definition video real-time transmission processing methods.

[0070] In summary, the present application can adaptively screen feature points according to the features of the image by screening feature points through pixel value differences, improving the reliability and accuracy of feature point screening, which is beneficial to subsequent dynamic detection based on feature points; dividing the image into feature blocks can perform local analysis on the image, more finely distinguish static and dynamic regions in the image, and through the calculation of optical flow method and motion feature values, the feature blocks can be accurately divided into dynamic or static, so that different coding strategies can be selected according to the dynamicity of the region in the subsequent coding process, thereby optimizing the coding efficiency and reducing the transmission of redundant data; Furthermore, by comparing the pixel value differences between each frame image and subsequent multiple frame images, global picture mutations can be detected. At the same time, through continuous multi-frame comparison, the situation of misjudging a picture mutation due to short-term image jitter or local changes can be effectively avoided, thereby improving the accuracy of picture mutation detection; when a picture mutation occurs in each frame image compared with the previous frame image, the merged regions of continuous dynamic regions in each frame image are dynamically and statically divided again, the differences between the merged regions in each frame image and the front and rear frame images are analyzed, the dynamic characteristics of the merged regions are evaluated, and the merged regions with large differences compared with both the front and rear frames are marked separately, so that more refined processing can be performed on the marked regions during the coding process to ensure the high-quality transmission of dynamic content; Furthermore, by dynamically adjusting the encoding strategy according to the dynamic characteristics of different regions, it is possible to significantly reduce the bit rate while ensuring the transmission quality of dynamic content, thereby improving the video transmission effect.

[0071] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the block may occur in a different order than noted in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. In the description corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description. Sometimes, there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. Each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0072] For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and without departing from the basic features of the present application, the present application can be implemented in other specific forms. Therefore, from any point of view, the above embodiments of the present application should be regarded as exemplary and non-limiting.

Claims

1. 5G Tablet PC High-Definition Video Real-Time Transmission and Processing Method, characterized in that, The method includes the following steps: Collect video frames of the scene to be photographed in real time, and obtain each frame image in the buffer queue in the video frames, where the image is an RGB image; Screen each feature point from the pixel points on each frame image through the pixel value differences on each channel between each pixel point on each frame image and its preset neighboring pixel points; Evenly divide each frame image into each feature block, use the optical flow method to obtain the optical flow vectors of each feature point, obtain the motion feature value of each feature block through the dispersion degree and average level of the modulus lengths of the optical flow vectors of all feature points in each feature block, and divide each feature block into a dynamic feature block or a static feature block by comparing the optical flow vectors of each feature point in each feature block with the motion feature value; evaluate whether a scene mutation has occurred after each frame image through the pixel value differences on each channel of the pixel points at the same positions between each frame image and the subsequent continuous preset number of frame images; merge consecutive dynamic feature blocks, and when the frame image where each merged area is located has a scene mutation compared with the previous frame image, perform the division operation of dynamic feature blocks and static feature blocks on each merged area again, and divide the merged area as a dynamic feature block into a dynamic feature area; When using high-efficiency video coding to compress and encode each frame image, set the value of the quantization parameter for each feature block belonging to the static feature block, dynamic feature block, and dynamic feature area respectively according to the division result of each feature block; Package and transmit the compressed and encoded image, and the receiving end decodes and restores each frame image and displays it on the tablet computer in real time according to the original time sequence.

2. The high-definition video real-time transmission and processing method for a 5G tablet computer according to claim 1, characterized in that, The screening of each feature point from the pixel points on each frame image includes: Calculate the average value of the pixel values of each pixel point on all channels, and use the arithmetic mean of all the average values of each pixel point and its preset neighboring pixel points as the feature point judgment threshold of each pixel point; Calculate the deviation value of the average value between each pixel point and its preset each neighboring pixel point, count the number of all the deviation values of each pixel point that are greater than the feature point judgment threshold, and when the number is greater than the preset positive integer, use each pixel point as a feature point.

3. The high-definition video real-time transmission processing method for a 5G tablet computer according to claim 1, characterized in that, The method for obtaining the motion feature value is: Calculate the average value of the modulus lengths of the optical flow vectors of all feature points in each feature block; The motion feature value is the sum of the average value and the dispersion degree.

4. The high-definition video real-time transmission processing method for a 5G tablet computer according to claim 1, wherein The process of dividing each feature block into a dynamic feature block or a static feature block is: Screen each motion feature point from the feature points in each feature block by comparing the modulus length of the optical flow vector of each feature point in each feature block with the motion feature value; when the proportion of the motion feature points in each feature block among all the feature points is greater than or equal to the preset value, divide each feature block into a dynamic feature block, otherwise, divide it into a static feature block.

5. The high-definition video real-time transmission processing method for a 5G tablet computer according to claim 4, wherein The screening method of the motion feature point is: use the feature point with the modulus length of the optical flow vector in each feature block greater than the motion feature value as the motion feature point.

6. The high-definition video real-time transmission processing method of a 5G tablet computer according to claim 2, wherein, The method for evaluating whether a scene mutation has occurred after each frame image is: Obtain the average pixel difference between each frame image and the subsequent frame image through the difference amount of the average value of the pixel points at the same positions between each frame image and the subsequent frame image; When the normalized values of the average pixel differences calculated between each frame image and the subsequent consecutive preset number of frame images are all greater than the preset pixel difference threshold, it is determined that a scene mutation has occurred after each frame image.

7. The high-definition video real-time transmission processing method for a 5G tablet computer according to claim 6, wherein The average pixel difference is the mean value of the difference amounts corresponding to all pixel points on each frame image.

8. The 5G tablet computer high-definition video real-time transmission processing method according to claim 3, wherein, The step of respectively setting the values of the quantization parameters for each feature block belonging to the static feature block, the dynamic feature block, and the dynamic feature region includes: Calculating the difference value between the magnitude of the optical flow vector of each feature point in each feature block and the average value, and denoting the mean value of the difference values of all feature points in each feature block as the motion mean; For a feature block with the number of feature points being 0, setting the value of the quantization parameter to a preset first value; For a feature block belonging to the static feature block, calculating the product of the normalized value of the motion mean of the feature block and a preset first sensitivity coefficient, and setting the value of the quantization parameter to the difference between a preset second value and the product; For a feature block belonging to the dynamic feature block and not belonging to the dynamic feature region, calculating the product value of the normalized value of the motion mean of the feature block and a preset second sensitivity coefficient, and setting the value of the quantization parameter to the difference between a preset third value and the product value; For a feature block belonging to the dynamic feature region, setting the value of the quantization parameter to a preset fourth value.

9. The real-time transmission and processing method of high-definition video for a 5G tablet computer according to claim 8, wherein, The magnitude relationship among the preset first value, the preset second value, the preset third value, and the preset fourth value is: the preset first value > the preset second value > the preset third value > the preset fourth value. 10.5G tablet computer high-definition video real-time transmission processing system, including a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the 5G tablet high-definition video real-time transmission processing method described in any one of claims 1-9.

Images