Screen content coding method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of video coding, and provides a screen content coding method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring a to-be-coded screen video image, and extracting key frames of each picture group of the screen video image; edge detection is carried out on the key frame, and the complexity of the key frame is determined according to the detected number of edge pixel points of the key frame; and determining a coding rate according to the complexity, and carrying out coding processing on the screen video image based on the coding rate to obtain a coding stream. By calculating the complexity of the key frame, the calculation amount of the complexity is reduced, the number of edge pixel points is used as the complexity variable of the key frame, the encoding code stream is adaptively determined according to the complexity, the edge information of the video image is reserved, the compression quality of the encoded video is ensured, the encoding efficiency of the screen content is improved, and the user experience is improved. Efficient coding is realized, and the method is suitable for scenes with relatively high time delay requirements.

Description

Technical Field

[0001] The present application relates to the field of video coding technology, and in particular to a screen content coding method, apparatus, device, storage medium and program product. Background Art

[0002] With the rapid development of internet technology, video services have become the largest type of service in terms of internet traffic. Due to their large file size, high bandwidth usage, and high latency requirements, video files pose significant challenges to application developers and service providers in terms of storage, transmission, and bandwidth. Therefore, breakthroughs in video codec technology are crucial for video transmission and application. Currently, encoding technologies such as x264 remain the most widely used due to their rich community ecosystem support, compatibility with various software and hardware, and excellent compression ratios.

[0003] Traditional x264 encoding technology is often used to compress and process natural video content captured by cameras. Many solutions have also been optimized for it, and all can improve encoding quality to a certain extent. However, the complex computational iteration process inevitably sacrifices a certain degree of encoding efficiency, especially for special scenarios such as screen content. In the field of screen content encoding, existing encoding schemes mainly include calculating the intra-frame complexity and inter-frame complexity in each GOP (Group of Pictures) and its corresponding scene complexity, and then using an encoding strategy that matches the complexity to encode each GOP in the video file, thereby improving transcoding efficiency. Dictionary encoding-based methods can also better encode screen text content, effectively improving the compression quality of text areas in screen content.

[0004] However, in the existing screen content encoding method, the method of calculating the intra-frame complexity, inter-frame complexity and the corresponding scene complexity of the GOP requires the intra-frame complexity and inter-frame complexity of each frame to be calculated separately. When the video resources are relatively large, the time delay requirements are high, and the network resources are poor, it is difficult to meet the requirements of efficient video encoding; the introduction of the dictionary encoding method, the construction and update of the dictionary is a complex process, and a certain amount of time complexity will be sacrificed in the calculation process. It is also not suitable for scenarios with high network delay requirements. Summary of the Invention

[0005] The embodiments of the present application provide a screen content encoding method, apparatus, device, storage medium and program product to solve the technical problem that the existing screen content encoding method has low encoding efficiency and is not suitable for scenarios with high latency requirements.

[0006] In a first aspect, an embodiment of the present application provides a screen content encoding method, comprising:

[0007] Acquire a screen video image to be encoded, and extract key frames of each picture group of the screen video image;

[0008] Performing edge detection on the key frame, and determining the complexity of the key frame according to the number of edge pixels of the key frame detected;

[0009] An encoding bit rate is determined according to the complexity, and encoding processing is performed on the screen video image based on the encoding bit rate to obtain an encoding bit stream.

[0010] In one embodiment, performing edge detection on the key frame and determining the complexity of the key frame according to the number of edge pixels of the key frame detected includes:

[0011] Based on a preset edge detection operator, edge gradient detection is performed on the key frame to obtain a gradient value of the key frame; the edge detection operator includes a horizontal convolution factor and a vertical convolution factor; the gradient value includes a horizontal gradient value and a vertical gradient value;

[0012] Calculating the gradient magnitude and gradient direction of the key frame according to the horizontal gradient value and the vertical gradient value;

[0013] Traversing the pixel points of the key frame based on the gradient magnitude and the gradient direction, and obtaining edge pixel points of the key frame using a non-maximum suppression algorithm;

[0014] The complexity of the key frame is determined according to the number of the edge pixels.

[0015] In one embodiment, encoding the screen video image based on the encoding bit rate to obtain an encoded bit stream includes:

[0016] Obtaining a coding quantization parameter currently used for encoding, and determining an estimated bit rate according to the complexity of a first frame in the screen video image; the complexity of the first frame is the complexity of the first key frame in the screen video image;

[0017] Calculating the encoding byte size corresponding to the screen video image encoding according to the encoding quantization parameter;

[0018] If the encoded byte size exceeds a preset byte threshold corresponding to the peak bit rate, iteratively adjusting the quantization encoding parameter until the encoded byte size is less than or equal to the byte threshold, thereby obtaining a target quantization parameter for encoding;

[0019] Determining a high-bitrate frame in the screen video image based on the target quantization parameter, and performing frame drop processing on the high-bitrate frame;

[0020] Based on the target quantization parameter, the screen video image after the frame loss process is encoded to obtain an encoded code stream.

[0021] In one embodiment, before encoding the screen video image based on the encoding bit rate to obtain the encoded bit stream, the method further includes:

[0022] Obtaining a timestamp difference between associated timestamps of any two adjacent frames in the screen video image;

[0023] Performing dynamic frame motion detection on the screen video image based on the timestamp difference to determine whether there is a first target frame that needs to be dropped;

[0024] If it exists, the first target frame is overwritten with the second target frame, and it is determined whether the first target frame is a key frame; the second target frame is the previous frame of the first target frame;

[0025] If the first target frame is not a key frame, a frame drop process is performed on the first target frame.

[0026] In one embodiment, before extracting the key frames of each picture group of the screen video image, the method further includes:

[0027] Performing image quality detection and format conversion on the screen video image to convert the screen video image into a preset format; wherein the image quality detection includes a freeze rate detection and a frame rate detection;

[0028] performing bilinear scaling processing on the screen video image in the preset format, and determining whether to perform filtering processing on the screen video image according to the image quality detection result;

[0029] If it is determined according to the image quality detection result that the screen video image is to be filtered, Gaussian blur filtering is performed on the screen video image in the preset format.

[0030] In one embodiment, after encoding the screen video image based on the encoding bit rate to obtain an encoded bit stream, the method further includes:

[0031] The coded code stream is divided into blocks, and based on a preset concurrent number, a preset image quality enhancement operator is called to perform image quality enhancement processing on each block obtained by the division; the image quality enhancement operator includes a Laplace operator.

[0032] In a second aspect, an embodiment of the present application provides a screen content encoding device, comprising:

[0033] An image acquisition module is used to obtain the screen video image to be encoded and extract the key frames of each picture group of the screen video image;

[0034] a complexity calculation module, configured to perform edge detection on the key frame and determine the complexity of the key frame based on the number of edge pixels of the key frame detected;

[0035] The encoding module is used to determine an encoding rate according to the complexity, and to encode the screen video image based on the encoding rate to obtain an encoding stream.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory storing a computer program, wherein when the processor executes the program, the steps of the screen content encoding method described in the first aspect are implemented.

[0037] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the screen content encoding method described in the first aspect.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the screen content encoding method described in the first aspect.

[0039] The screen content encoding method, apparatus, device, storage medium, and program product provided in the embodiments of the present application extract key frames from each screen group of a screen video image, perform edge detection, and only perform complexity calculations on the key frames, thereby reducing the amount of complexity calculations. The encoding bit rate is determined based on the calculated complexity, and the encoding bit rate is adaptively controlled to encode the screen video image and obtain an encoded bitstream. By calculating the complexity of the key frames, the amount of complexity calculations is reduced. The number of edge pixels is used as the complexity variable of the key frames, and the encoding bitstream is adaptively determined based on the complexity, preserving the edge information of the video image, which is beneficial to ensuring the compression quality of the encoded video, while also improving the encoding efficiency of the screen content, achieving efficient encoding, and being suitable for scenarios with high latency requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 1 is a flow chart of a method for encoding screen content provided in an embodiment of the present application;

[0042] Figure 2 This is a schematic diagram of the call platform architecture provided in an embodiment of the present application;

[0043] Figure 3 This is a schematic diagram of the screen content encoding process provided by an embodiment of the present application;

[0044] Figure 4 This is a schematic diagram of the preprocessing process provided in the embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of the complexity calculation process provided by an embodiment of the present application;

[0046] Figure 6 This is a schematic diagram of the frame rate control process provided by an embodiment of the present application;

[0047] Figure 7 This is a schematic diagram of the bit rate control process provided by an embodiment of the present application;

[0048] Figure 8 This is a schematic diagram of the post-processing process provided by the embodiment of the present application;

[0049] Figure 9 is a structural diagram of a screen content encoding device provided in an embodiment of the present application;

[0050] Figure 10 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions in this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0052] It should be noted that, in the description of the present invention, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, the phrase "comprises a..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising the elements. Terms such as "upper" and "lower" indicate positions or relationships based on those shown in the accompanying drawings and are intended solely to facilitate the description of the present invention and simplify the description. They are not intended to indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation, and are therefore not to be construed as limitations on the present invention. Unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be broadly construed, for example, to mean fixed, removable, or integral; mechanical or electrical; direct or indirect through an intermediary; or internal communication between two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0053] The terms "first," "second," and so forth, used herein are used to distinguish similar objects, not to describe a specific order or precedence. It should be understood that such terms are interchangeable where appropriate, allowing embodiments of the present invention to be implemented in an order other than that illustrated or described herein. Furthermore, the terms "first," "second," and so forth generally distinguish objects of a single type, and do not limit the number of objects. For example, the first object may be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the connected objects.

[0054] Unlike traditional camera footage, screen content comes from sources like screenshots, recordings, and shared screens. It also contains a large number of moving or stationary graphics and text, as well as blocks or areas with identical numbers or letters within an image sequence. Screen content encoding encodes the captured screen content information. With the in-depth development of 5G technology, demand for video technology has become more stringent. Compared to traditional customer service systems that only provide voice and text interaction, the demand for multimedia intelligent interaction is increasing. To create a new digital experience portal and promote the application of multimedia intelligent interaction and integrated online operations, customer service systems are evolving from traditional "auditory" to "face-to-face visual" interaction, enabling more efficient and intuitive solutions to complex customer service issues across a wider range of scenarios. Digital services require more effective technical support. As a key support platform for customer service systems, call platforms provide favorable technical support and assurance for intelligent customer service systems. To provide better service quality, services such as video calling, touchscreen interaction, and dynamic screen page content push have become key features of call platforms. Call platforms have high requirements for latency and quality, and also have multiple scenarios such as same-screen sliding, screen sharing, and screenshot push. In order to ensure the quality and reliability of video transmission on call platforms, it is necessary to seek a more efficient encoding solution.

[0055] Based on this, this application provides a screen content encoding method that can be applied to the x264 screen in the call platform, focusing on the application characteristics of the call platform. By improving x264, it can achieve good applications in multiple scenarios such as video calls, same-screen collaboration, touch screen interaction, dynamic page content push, etc. in the call platform.

[0056] Specifically, Figure 1 This is a flow chart of the screen content encoding method provided by this application. Figure 1 The screen content encoding method provided in the embodiment of the present application may include:

[0057] Step 100: obtaining a screen video image to be encoded, and extracting key frames of each picture group of the screen video image;

[0058] Step 200: performing edge detection on the key frame, and determining the complexity of the key frame based on the number of edge pixels of the key frame detected;

[0059] Step 300: Determine an encoding rate according to the complexity, and perform encoding processing on the screen video image based on the encoding rate to obtain an encoding stream.

[0060] First, the screen video image to be encoded is obtained. This screen video image can be from a screenshot, screen recording, or screen sharing. Based on the obtained screen video image, key frames are extracted from each group of pictures (GOP). The screen video image includes one or more key frames, and each group of pictures can also include one or more key frames. For each GOP, the key frame can basically contain all the information in the GOP.

[0061] Edge detection is performed on each extracted keyframe. Edge pixels are obtained from each keyframe through edge detection, and the number of edge pixels in each keyframe is counted. Based on the counted number of edge pixels in each keyframe, the complexity corresponding to the keyframe is determined. Because keyframes within a GOP can contain complete image information, in this embodiment, complexity calculation is performed on a GOP-by-GOP basis, with complexity calculation performed for each keyframe within the GOP.

[0062] Preferably, each GOP may include one or more key frames. When a GOP includes multiple key frames, the sum of the complexities of the multiple key frames may be used as the complexity of the GOP, or the average value or weighted average value of the complexities of the multiple key frames may be used as the complexity of the GOP.

[0063] Furthermore, in scenarios such as dynamic screen page content display, touch screen interaction, and same-screen collaboration in the call platform system, the screen content is mostly text information with strong edge information. Obtaining the edge information of the screen content through edge detection is conducive to retaining the key feature information of the video image, thereby ensuring the compression quality of the encoded video.

[0064] Preferably, the corresponding encoding bit rate is determined based on the calculated complexity, and the screen video image is encoded based on the encoding bit rate to obtain an encoding stream. The number of edge points in the video key frame is used as a complexity variable, and the encoding stream is further determined based on the complexity, and the encoding bit rate is adaptively controlled.

[0065] In one embodiment, referring to Figure 2 The overall architecture of the call platform shown in FIG. 1 and the screen content encoding method provided in the embodiment of the present application are specifically applied to the video encoding module in the call platform. Figure 2Customers and agents interact with each other through a call platform, which primarily includes an access session control module, a service navigation processing module, a manual call processing module, and an interconnected session control module. The service navigation processing module interacts bidirectionally with the self-service business processing module, assisting customers in completing business processes on their own. Preferably, with the assistance of a video encoding module, customers can use the self-service business processing module to perform video editing, video synthesis, and video encoding. In the manual call processing module, service requests such as voice calls, video calls, on-screen conversations, touchscreen interactions, screenshot sharing, and screen sharing can be triggered, thereby triggering the video encoding module to encode the screen content.

[0066] Preferably, the video encoding module is activated for business scenarios requiring video encoding of screen content, primarily including self-service transactions and manual video calls. When determining whether to activate screen content encoding, the call platform controls the activation of the video encoding module and detects the screen content based on the business type. When the screen content encoding function is required, the video encoding module is invoked to encode the screen content.

[0067] In this embodiment, key frames from each group of screen video images are extracted. Edge detection is then performed, and complexity calculations are performed only on the key frames, reducing the complexity computations. The encoding bitrate is then determined based on the calculated complexity, and the encoding bitrate is adaptively controlled to encode the screen video image and generate an encoded bitstream. By calculating the complexity of the key frames, the complexity computations are reduced. The number of edge pixels is used as a complexity variable for the key frames, and the encoding bitstream is adaptively determined based on the complexity. This preserves the edge information of the video image, helps ensure the compression quality of the encoded video, and improves the encoding efficiency of the screen content, achieving efficient encoding. This approach is suitable for scenarios with high latency requirements.

[0068] Preferably, Figure 2 Based on the call platform architecture shown, further reference Figure 3The screen content encoding process shown in the figure performs screen content detection on the screen video image to be processed. When it is determined that screen content encoding needs to be initiated, the video encoding module is called. First, the input video frames are preprocessed. This preprocessing mainly includes standardizing the video frame format and frame size, and filtering to eliminate noise and blur. Complexity calculation is then performed on the video key frames. The core algorithms for complexity calculation include edge detection. Frame rate control is then used to dynamically adjust the video frames and control the frame rate through frame drop and frame filling. Bit rate control is also used to adaptively control the encoding bit rate of the video encoding module based on the complexity calculation results. Finally, post-encoding processing is used to enhance the image quality of the encoded bitstream, improving the contrast of image details and text edges, and enhancing the visual effect, ultimately resulting in an optimized encoded bitstream.

[0069] Therefore, before performing complexity calculation on a video image, it is necessary to pre-process the video image to retain only valuable edge information, and then perform complexity calculation on it. This can not only retain the image's feature information to the greatest extent possible, but also minimize computational complexity. Based on this, in step 100, before extracting the key frames of each GOP of the screen video image, the following steps may also be performed:

[0070] Step 001: Perform image quality detection and format conversion on the screen video image to convert the screen video image into a preset format; wherein the image quality detection includes a freeze rate detection and a frame rate detection;

[0071] Step 002: performing bilinear scaling on the screen video image in the preset format, and determining whether to perform filtering on the screen video image based on the image quality detection result;

[0072] Step 003: If it is determined according to the image quality detection result that the screen video image is to be filtered, Gaussian blur filtering is performed on the screen video image in the preset format.

[0073] Reference Figure 4 The preprocessing process shown, in order to preprocess the video more flexibly and not waste too much computing resources, performs image quality detection before video encoding to identify and judge the image quality of the video and determine whether filtering processing needs to be enabled. Image quality detection is controlled through two aspects. First, if the agent finds that the video quality is poor during the video service process, the service side can issue an instruction to enable image quality optimization and start preprocessing and filtering operations on the video image; second, by automatically detecting the freeze rate and frame rate of the video frame, the video quality is judged, and then it is determined whether preprocessing and filtering operations need to be enabled. Preferably, if the service side issues an instruction to enable image quality optimization, there is no need to repeat the video quality detection.

[0074] Specifically, in the process of preprocessing the screen video image to be processed, the screen video image is first subjected to image quality detection and format conversion, and the screen video image is converted into a preset format, wherein the image quality detection includes stuttering rate detection and frame rate detection. Preferably, the preset format includes but is not limited to the YUV format. In the YUV format, "Y" represents brightness, that is, grayscale value, and "U" and "V" represent the difference between the blue channel and the brightness, and the difference between the red channel and the brightness, respectively. The following description takes the YUV format as an example of the preset format.

[0075] Bilinear scaling is performed on the YUV format video image obtained after format conversion. Based on the image quality detection results, it is determined whether preprocessing filtering is required, that is, whether filtering is performed on the screen video image after bilinear scaling. If filtering is not required, the preprocessed screen video image is obtained. If filtering is required, Gaussian blur filtering is performed on the YUV format screen video image after bilinear scaling to obtain the preprocessed screen video image.

[0076] To ensure computational uniformity, the input non-YUV video frames are first converted to YUV format, and the image is bilinearly scaled. Optionally, bilinear scaling employs bilinear interpolation, which predicts unknown points using known points. This operation aims to modify the image size. Since the input video frames may have different resolutions due to different devices, they are first scaled, for example, to a uniform size of 320×240. Scaling does not affect the information contained in the image, but it can reduce redundant information by reducing the image size, thereby reducing computational complexity. For scenarios requiring preprocessing filter optimization, a Gaussian blur filter is provided. This filter can filter common Gaussian noise and blur, eliminating interfering information in the video frame before subsequent calculations. This reduces computational complexity while also better preserving the video frame's information. Furthermore, the preprocessing of screen video images is highly scalable, allowing for extended applications such as sharpening and de-jittering.

[0077] Optimally, for scenarios such as dynamic screen page content display, touchscreen interaction, and same-screen collaboration in the call platform system, where the screen content, which is primarily textual, has strong edge information, an edge detection algorithm is used to obtain the edge information of the screen content. The number of edge points is then calculated as a complexity variable to achieve dynamic bitrate control. Since the key frames in a GOP can contain complete image information, the complexity calculation is performed on the key frames in each GOP separately.

[0078] Specifically, in step 200, edge detection is performed on the key frame, and the complexity of the key frame is determined based on the number of edge pixels of the key frame detected. The following steps may also be included:

[0079] Step 210: Perform edge gradient detection on the key frame based on a preset edge detection operator to obtain a gradient value of the key frame; the edge detection operator includes a horizontal convolution factor and a vertical convolution factor; the gradient value includes a horizontal gradient value and a vertical gradient value;

[0080] Step 220, calculating the gradient magnitude and gradient direction of the key frame according to the horizontal gradient value and the vertical gradient value;

[0081] Step 230, traversing the pixel points of the key frame based on the gradient magnitude and the gradient direction, and obtaining edge pixel points of the key frame using a non-maximum suppression algorithm;

[0082] Step 240: Determine the complexity of the key frame according to the number of edge pixels.

[0083] When calculating the complexity of a keyframe, edge gradient detection is performed on the keyframe based on a preset edge detection operator to obtain the gradient value of the keyframe. The edge detection operator includes a horizontal convolution factor and a vertical convolution factor. Edge gradient detection of the keyframe, i.e., planar convolution of the keyframe, includes a horizontal convolution based on the horizontal convolution factor and a vertical convolution based on the vertical convolution factor. The gradient value obtained by edge gradient detection includes a horizontal horizontal gradient value and a vertical vertical gradient value.

[0084] Furthermore, according to the horizontal gradient value and the vertical gradient value, the total gradient magnitude and gradient direction of the key frame are calculated. Based on the gradient magnitude and gradient direction, each pixel point of the key frame is traversed, and the edge pixel points of the key frame are obtained by using the non-maximum suppression algorithm. The number of edge pixel points is counted, and the complexity of the key frame is determined based on the counted number of edge pixel points. The number of edge pixel points is used as the complexity variable of the key frame.

[0085] In one embodiment, referring to Figure 5 The complexity calculation process shown in the figure calculates the number of edge pixels in the video keyframe as a complexity variable, and then adaptively controls the encoding bitrate based on the complexity. Due to the format conversion of the screen video image during preprocessing, the Y component of the YUV format video image can fully represent the image's edge details. Therefore, only the grayscale image of the Y component is calculated during the complexity calculation, further reducing the complexity calculation amount.

[0086] Specifically, taking the Sobel operator of the first-order derivative as an edge detection operator as an example, the edge gradient detection is performed on the pre-processed video frame by the Sobel operator. For example, the Sobel operator contains two sets of 3×3 matrices, namely the horizontal template and the vertical template. By performing planar convolution with the video image, the horizontal brightness difference approximation and the vertical brightness difference approximation can be obtained respectively. The horizontal convolution factor G of the Soble operator is x and the convolution factor G in the vertical direction y They are:

[0087]

[0088] If A represents the feature vector corresponding to the preprocessed video frame, the gradient calculation method is shown in the following formula 2:

[0089]

[0090] Based on formula 2, when performing gradient calculation, first calculate the horizontal gradient G x and the vertical gradient G y , as follows:

[0091] G x =[f(x+1),(y-1)+2*f(x+1,y)+f(x+1,y+1)-f(x-

[0092] 1,y-1)+2*f(x-1,y)+f(x-1,y+1)]; (3)

[0093] G y =[f(x-1),(y-1)+2*f(x,y-1)+f(x+1,y-1)-f(x-

[0094] 1,y+1)+2*f(x,y+1)+f(x+1,y+1)]; (4)

[0095] (x, y) represents the coordinates of the pixel in the video frame, according to the calculated horizontal gradient G x and the vertical gradient G y , the total gradient size G and gradient direction θ are:

[0096]

[0097] Preferably, the purpose of non-maximum suppression is to find the local maximum of the pixel point, and set the grayscale value corresponding to the non-maximum point to a specific characteristic value such as 0, so that most of the non-edge pixels can be eliminated. The principle of non-maximum suppression is to compare the amplitude of the two pixels before and after the plane at a specific angle such as 45° along the gradient direction of the pixel point. If the pixel point is larger than the two pixels before and after, it is retained. If the pixel point is smaller than at least one of the two pixels before and after, its pixel value is set to 0. Non-maximum suppression can further eliminate non-edge noise and refine the edges.

[0098] Specifically, for an image with a given pixel's gradient magnitude and gradient direction, the algorithm traverses each pixel to find the gradient magnitudes of its two adjacent pixels in the same gradient direction as the current pixel. It then determines whether the current pixel's gradient magnitude is greater than or equal to the gradient magnitudes of its two adjacent pixels, and stores the result in a preset array. If the current pixel's gradient magnitude is greater than or equal to the gradient magnitudes of its adjacent pixels, the pixel is considered to have the local maximum gradient magnitude; otherwise, the pixel is considered a non-edge pixel and its pixel value is set to 0. After non-maximum suppression, the gradient magnitudes are classified into three categories: strong edge, weak edge, and non-edge. Strong edge refers to pixels with a gradient magnitude greater than or equal to a preset high threshold, while non-edge refers to pixels with a gradient magnitude less than a preset low threshold. Pixels with gradient magnitudes between these two categories are classified based on the surrounding pixels. That is, if there is a strong edge pixel among the adjacent pixels, the pixel is classified as a weak edge; otherwise, it is classified as a non-edge. The gradient magnitude of a pixel is determined based on its gradient magnitude.

[0099] In the specific implementation, a double loop traverses the entire video image, processing each pixel according to the above process. A preset number of adjacent pixels surrounding each pixel are identified. If a pixel has a strong edge neighbor, the edge is determined to be a weak edge. Finally, the strong edge pixel is connected to its adjacent weak edge pixels. Using a dual-threshold edge pixel determination method combining high and low thresholds, edge pixels are identified and counted, and normalized to a range of values to represent complexity.

[0100] Preferably, in order to solve the problem that the H264 encoder cannot dynamically control the frame rate, the frame rate of the encoded output video stream is dynamically controlled according to the available bit rate, mainly through the frame loss strategy identification and frame loss strategy execution, to achieve frame rate control during video encoding. In step 300, before encoding the screen video image according to the determined encoding code stream to obtain the encoding code stream, it can also include:

[0101] Step 301, obtaining the timestamp difference between the associated timestamps of any two adjacent frames in the screen video image;

[0102] Step 302: Perform dynamic frame motion detection on the screen video image based on the timestamp difference to determine whether there is a first target frame that needs to be dropped.

[0103] Step 303: If it exists, the first target frame is overwritten with the second target frame, and it is determined whether the first target frame is a key frame; the second target frame is the previous frame of the first target frame;

[0104] Step 304: If the first target frame is not a key frame, perform frame drop processing on the first target frame.

[0105] The frame rate control of the video frames is achieved based on the dynamic frame motion detection and timestamp association detection of the video frames. Specifically, the timestamp difference between the associated timestamps of any two adjacent video frames in the screen video image is first obtained, and the screen video image is subjected to dynamic frame motion detection based on the timestamp difference to determine whether there is a first target frame that needs to be dropped. If there is a first target frame, the second target frame is used to cover the first target frame, and it is determined whether the first target frame is a key frame. If it is a key frame, the key frame information needs to be saved. If it is not a key frame, the frame dropping strategy is executed to perform frame dropping processing on the first target frame. Among them, the second target frame used to cover the first target frame is the previous video frame of the first target frame.

[0106] In one embodiment, referring to Figure 6 In the frame rate control process shown, frame drop detection is a precursor to executing the frame drop strategy. This allows for dynamic detection of whether the current input video frame requires frame drop processing. Frame drop detection is implemented by detecting the timestamps associated with adjacent frames and dynamic frame motion detection. In scenarios such as touchscreen scrolling, rapid screen scrolling can exacerbate video motion blur. To address this scenario, the difference in the true absolute timestamps between adjacent frames is calculated by detecting the timestamps associated with adjacent frames. If the time difference between adjacent frames exceeds a certain value, the video screen is considered to be moving too quickly, and the frame drop strategy needs to be implemented.

[0107] Adjacent frame timestamp detection is effective for identifying frames with high sliding speeds, but it cannot accurately detect motion differences between video frames. To more accurately detect video frame motion information, dynamic frame motion detection can instantly detect scenes with intense motion in the current video frame and implement a frame drop strategy. The principles of dynamic frame motion detection include, but are not limited to, calculating the pixel difference between the YUV data of two video frames through a motion block matching algorithm, then weighting and normalizing these pixel differences, and finally determining the magnitude of motion based on a set threshold.

[0108] A typical frame-dropping strategy is to directly discard detected video frames of poor quality. However, this operation can affect the smoothness of the video, especially in poor network environments. To address this issue, this embodiment uses the information of the previous frame to supplement the video frames that need to be dropped when the current video frame is discarded. This does not affect the smoothness of the video, but also ensures image continuity and visual clarity.

[0109] Furthermore, because a key frame contains complete information about the image in its GOP, when executing the frame drop strategy, it is also necessary to determine whether it is a key frame. If it is a key frame, the frame drop operation is not performed on it, and its information, such as the number of bits used by the key frame, is saved. If it is a non-key frame, it is dropped according to the frame drop strategy.

[0110] Preferably, in order to solve the problem of excessive instantaneous bitrate fluctuations caused by intense motion and scene switching in low bandwidth conditions, peak bitrate control is used to suppress the instantaneous bitrate peak, thereby avoiding the phenomenon of increased network congestion and image freeze caused by excessive instantaneous bitrate. In step 300, based on the determined encoding bitrate, the screen video image is encoded to obtain an encoded bitstream, which specifically includes:

[0111] Step 310: Obtain the encoding quantization parameter currently used for encoding, and determine an estimated bit rate based on the complexity of the first frame in the screen video image; the complexity of the first frame is the complexity of the first key frame in the screen video image;

[0112] Step 320, calculating the encoding byte size corresponding to the screen video image encoding according to the encoding quantization parameter;

[0113] Step 330: If the encoded byte size exceeds a preset byte threshold corresponding to the peak bit rate, iteratively adjust the quantization encoding parameter until the encoded byte size is less than or equal to the byte threshold, thereby obtaining a target quantization parameter for encoding.

[0114] Step 340: determining a high-bitrate frame in the screen video image based on the target quantization parameter, and performing frame drop processing on the high-bitrate frame;

[0115] Step 350 : Based on the target quantization parameter, the screen video image after the frame loss process is encoded to obtain an encoded bit stream.

[0116] Gets the quantization parameter (QP) currently used for encoding. This parameter is the key parameter used in image or video data compression, and represents the quantization parameter (QP). The QP value reflects the degree of compression of spatial details in the image. A lower QP value preserves more image detail, while a higher QP value loses more detail, resulting in reduced video clarity. There is a negative correlation between QP and bitrate, meaning that as the QP value increases, the required bitrate decreases accordingly.

[0117] An estimated bit rate is determined based on the complexity of the first frame in the screen video image, wherein the complexity of the first frame is the complexity of the first key frame. The encoding byte size corresponding to the screen video image encoding is calculated based on the QP value currently used for encoding, thereby determining an estimated bit rate. Furthermore, the encoding byte size corresponding to the screen video image encoding is calculated based on the QP value currently used for encoding. If it exceeds the byte threshold corresponding to the peak bit rate, the QP value is iteratively adjusted until the encoding byte size corresponding to the screen video image is less than or equal to the preset byte threshold, thereby obtaining a target QP value for encoding. Based on the target QP value, a high-bitrate frame in the screen video image is determined, and a frame loss process is performed on the high-bitrate frame. Then, the screen video image after the frame loss process is encoded based on the target QP value to obtain an encoded bitstream.

[0118] In one embodiment, referring to Figure 7 The rate control process shown in FIG. 1 is to control the rate in the video encoding process by peak rate control and encoding bit reallocation. Specifically, Figure 7 As shown in the figure, an estimated bitrate is first estimated based on the complexity of the first frame. The QP value is then used to estimate the number of bytes required for encoding. This is then compared to the target byte count through continuous backpropagation, and the target value is finally approximated iteratively to obtain the estimated target QP value. The expected bitrate is calculated by positively correlating the complexity of the first keyframe, obtained through complexity calculation, with the current bitrate. This means the expected bitrate is dynamically estimated based on the complexity of the first frame.

[0119] First, the current QP value is used to estimate the required encoding byte size. The cumulative sum of the encoded bytes in the queue over a period is calculated, and a check is performed to determine whether the queue has fully completed a period. For example, the period duration is set to 1 second. If the period has not fully completed, the current estimated byte count is placed directly after the last valid data in the queue. If the period has fully completed, the data is shifted forward, the oldest data is discarded, and the current frame's data is placed at the end of the queue. Next, the desired bitrate is calculated. If the sum of the current cumulative byte count and the current estimated value exceeds the peak bitrate, the QP value is increased and iterated until it approaches the peak bitrate. The QP value calculated at this bitrate is then used as the target QP for the current frame. The maximum number of bytes encoded for a single frame cannot exceed a certain percentage of the peak bitrate. Otherwise, if a frame occupies too many bytes, the bytes available for other frames in the same period will be reduced. Coding bit reallocation controls the distribution of the available encoding bitrate within a single frame to avoid exhausting the available bits and ensure that more coding bits are reserved for complex scenes.

[0120] Preferably, in the process of implementing the encoding bit reallocation, for non-preset frames, such as initialization frames other than the first 5 frames, dynamic motion detection is used to determine whether the current picture is in violent motion and calculate the appropriate bit rate, so as to determine whether it is necessary to reset the bit rate and maximum bit rate for the current picture. If it is necessary to reset the bit rate and maximum bit rate, the VBV (Video Buffering Verifier) parameters in the encoder parameters are readjusted and set. The macroblock encoding QP value is calculated based on the reset VBV parameters for subsequent encoding. Whether frame drop processing is required is determined based on the statistical high-bitrate frames. Usually, when the picture changes violently, simply reducing the bit rate will seriously affect the image quality, so frame drop processing is performed on it.

[0121] Preferably, in order to solve the problems of blurred images after encoding due to the unclear input video stream image itself, soft images due to the low-pass filtering nature of the scaling algorithm, and image quality degradation due to the loss of some high-frequency components during the encoding process, the contrast of the encoded image details and text edges is improved by post-encoding image quality enhancement processing, thereby making the blurred image clear. That is, the screen content encoding method provided by the embodiment of the present application also includes post-processing. After obtaining the encoded code stream, the encoded code stream can also be subjected to image quality enhancement processing. Specifically, in step 300, after encoding the screen video image based on the determined encoding bit rate and obtaining the encoded code stream, it can also include:

[0122] Step 400 , the coded stream is divided into blocks, and based on a preset concurrent number, a preset image quality enhancement operator is called to perform image quality enhancement processing on each block obtained by the division; the image quality enhancement operator includes a Laplacian operator.

[0123] Post-processing of the coded bitstream is implemented based on multiple preset image quality enhancement operators. Multiple image quality enhancement operators can realize parallel processing of the coded bitstream. Specifically, the coded bitstream is first divided into blocks. Based on the preset concurrency number, the same number of image quality enhancement operators as the concurrency number are called, and image quality enhancement processing is performed on each divided block to obtain the final coded bitstream. The image quality enhancement operators include but are not limited to the Laplace operator.

[0124] In one embodiment, referring to Figure 8 The post-processing flow shown in the figure determines whether the temporary memory is smaller than the memory size required by the current input YUV video frame. If so, the previous memory is released and then the memory is reallocated. According to the pre-set number of concurrency, the input YUV format video frame is divided into blocks to facilitate subsequent concurrent execution. The corresponding number of image quality enhancement operators are called concurrently to perform image quality enhancement transformation processing on each divided block. After the transformation operation is completed in all blocks, the completed data, except for the first and last rows, is overwritten and transferred to the YUV memory. The YUV memory data is restored to obtain the image quality enhanced encoded bitstream cached in the YUV.

[0125] The image quality enhancement operator is the core of the post-encoding image quality enhancement process. It mainly performs a sharpening operation on the encoded video frame by applying the Laplacian operator to eliminate edge blur and other phenomena in the encoded video frame. The operating principle of the Laplacian operator is to reduce image blur by enhancing the high-frequency components of the image, enhance image details and contours, and enhance grayscale contrast, thereby making the image clearer. Common Laplacian operators are the 4-neighborhood Laplacian operator and the 8-neighborhood Laplacian operator. The Laplacian operator is rotationally invariant, thus meeting the requirements for image edge sharpening in different directions. However, the 4-neighborhood Laplacian operator has a finer boundary, which can lead to unclear sharpening boundaries for images containing more detailed information such as screen text. The 8-neighborhood Laplacian operator significantly increases the computational cost. Based on this, in this embodiment, the absolute value of the difference between the center element and the sum of its neighborhood values 4 times, that is, the absolute value of the center element and the sum of the elements in its neighborhood minus 5 times, is used as the sharpening operator kernel to form the Laplacian operator as the image quality enhancement operator. The operator kernel is as follows:

[0126] 0 -1 0 -1 5 -1 0 -1 0

[0127] The transformation operation corresponding to the operator kernel is shown in the following formula 6:

[0128]

[0129] After optimizing the encoded video image using this operator, the edge details of the text in the image are improved. Compared with before processing, the text part will be clearer, reducing the problem of blurred encoded images caused by network bandwidth limitations.

[0130] In this embodiment, the video frame quality is optimized and pre-processed before screen content encoding, which solves the problems of blurred edges and low clarity of the original video frames caused by video acquisition equipment, transmission network and environmental factors, and avoids the influence of video source quality on the encoding effect.

[0131] Furthermore, a keyframe complexity calculation method based on GOPs and adaptively controlling the peak bitrate based on the complexity of the first frame addresses the issue of initial frame blur by calculating the motion complexity of sub-planes and the static complexity of the current image. Furthermore, an adaptive frame rate control strategy based on screen content coding ensures the ability to perceive scenes with intense motion through the dual perception of frame-associated timestamps and dynamic motion detection. Adaptive frame rate control using a frame drop-and-fill strategy effectively improves encoding efficiency and quality.

[0132] Furthermore, the adaptive bitrate control strategy based on screen content encoding suppresses excessive instantaneous encoding bitrates during encoding, and comprehensively plans the available bitrates to solve the problem of insufficient bitrate allocation in complex scenarios. For the problem of blurred images caused by scrolling in scenarios such as touch-screen interaction or screen sharing, the quality of the current frame is improved by reallocating the bits of the current frame. The image quality enhancement processing based on screen content encoding compensates for the blurred, soft, and smooth image quality after encoding. For the problem of blurred text edges, the sharpening operation can improve the clarity of text edges and text areas without increasing bits.

[0133] The screen content encoding device provided in an embodiment of the present application is described below. The screen content encoding device described below and the screen content encoding method described above can refer to each other.

[0134] Reference Figure 9 , the screen content encoding device provided by the embodiment of the present application includes:

[0135] The image acquisition module 10 is used to obtain the screen video image to be encoded and extract the key frames of each picture group of the screen video image;

[0136] The complexity calculation module 20 is used to perform edge detection on the key frame and determine the complexity of the key frame according to the number of edge pixels of the key frame detected;

[0137] The encoding module 30 is configured to determine an encoding rate according to the complexity, and perform encoding processing on the screen video image based on the encoding rate to obtain an encoded code stream.

[0138] In one embodiment, the complexity calculation module 20 is further configured to:

[0139] Using a preset edge detection operator, edge gradient detection is performed on the key frame to obtain a gradient value of the key frame; the edge detection operator includes a horizontal convolution factor and a vertical convolution factor; the gradient value includes a horizontal gradient value and a vertical gradient value;

[0140] Calculating the gradient magnitude and gradient direction of the key frame according to the horizontal gradient value and the vertical gradient value;

[0141] Traversing the pixel points of the key frame based on the gradient magnitude and the gradient direction, and obtaining edge pixel points of the key frame using a non-maximum suppression algorithm;

[0142] The complexity of the key frame is determined according to the number of the edge pixels.

[0143] In one embodiment, the encoding module 30 is further configured to:

[0144] Obtaining a coding quantization parameter currently used for encoding, and determining an estimated bit rate according to the complexity of a first frame in the screen video image; the complexity of the first frame is the complexity of the first key frame in the screen video image;

[0145] Calculating the encoding byte size corresponding to the screen video image encoding according to the encoding quantization parameter;

[0146] If the encoded byte size exceeds a preset byte threshold corresponding to the peak bit rate, iteratively adjusting the quantization encoding parameter until the encoded byte size is less than or equal to the byte threshold, thereby obtaining a target quantization parameter for encoding;

[0147] Determining a high-bitrate frame in the screen video image based on the target quantization parameter, and performing frame drop processing on the high-bitrate frame;

[0148] Based on the target quantization parameter, the screen video image after the frame loss process is encoded to obtain an encoded code stream.

[0149] In one embodiment, the video content encoding apparatus further includes a frame rate control module configured to:

[0150] Obtaining a timestamp difference between associated timestamps of any two adjacent frames in the screen video image;

[0151] Performing dynamic frame motion detection on the screen video image based on the timestamp difference to determine whether there is a first target frame that needs to be dropped;

[0152] If it exists, the first target frame is overwritten with the second target frame, and it is determined whether the first target frame is a key frame; the second target frame is the previous frame of the first target frame;

[0153] If the first target frame is not a key frame, a frame drop process is performed on the first target frame.

[0154] In one embodiment, the video content encoding apparatus further includes a pre-processing module configured to:

[0155] Performing image quality detection and format conversion on the screen video image to convert the screen video image into a preset format; wherein the image quality detection includes a freeze rate detection and a frame rate detection;

[0156] performing bilinear scaling processing on the screen video image in the preset format, and determining whether to perform filtering processing on the screen video image according to the image quality detection result;

[0157] If it is determined according to the image quality detection result that the screen video image is to be filtered, Gaussian blur filtering is performed on the screen video image in the preset format.

[0158] In one embodiment, the video content encoding apparatus further includes a post-processing module configured to:

[0159] The coded code stream is divided into blocks, and based on a preset concurrent number, a preset image quality enhancement operator is called to perform image quality enhancement processing on each block obtained by the division; the image quality enhancement operator includes a Laplace operator.

[0160] Figure 10 An example of a physical structure diagram of an electronic device is shown below. Figure 10 As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communication interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 may call a computer program in the memory 1030 to execute the steps of the screen content encoding method, for example, including:

[0161] Acquire a screen video image to be encoded, and extract key frames of each picture group of the screen video image;

[0162] Performing edge detection on the key frame, and determining the complexity of the key frame according to the number of edge pixels of the key frame detected;

[0163] An encoding bit rate is determined according to the complexity, and encoding processing is performed on the screen video image based on the encoding bit rate to obtain an encoding bit stream.

[0164] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0165] On the other hand, embodiments of the present application further provide a computer program product, comprising a computer program. The computer program may be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the screen content encoding method provided in the above embodiments, for example, including:

[0166] Acquire a screen video image to be encoded, and extract key frames of each picture group of the screen video image;

[0167] Performing edge detection on the key frame, and determining the complexity of the key frame according to the number of edge pixels of the key frame detected;

[0168] An encoding bit rate is determined according to the complexity, and encoding processing is performed on the screen video image based on the encoding bit rate to obtain an encoding bit stream.

[0169] On the other hand, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, wherein the computer program is configured to cause a processor to execute the steps of the screen content encoding method provided in the above embodiments, for example, including:

[0170] Acquire a screen video image to be encoded, and extract key frames of each picture group of the screen video image;

[0171] Performing edge detection on the key frame, and determining the complexity of the key frame according to the number of edge pixels of the key frame detected;

[0172] An encoding bit rate is determined according to the complexity, and encoding processing is performed on the screen video image based on the encoding bit rate to obtain an encoding bit stream.

[0173] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NANDFLASH), solid-state drives (SSDs)), etc.

[0174] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0175] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A screen content encoding method, characterized in that: include: Acquire a screen video image to be encoded, and extract key frames of each picture group of the screen video image; Performing edge detection on the key frame, and determining the complexity of the key frame according to the number of edge pixels of the key frame detected; An encoding bit rate is determined according to the complexity, and encoding processing is performed on the screen video image based on the encoding bit rate to obtain an encoding bit stream.

2. The screen content encoding method according to claim 1, wherein: The performing edge detection on the key frame and determining the complexity of the key frame according to the number of edge pixels of the key frame detected includes: Based on a preset edge detection operator, edge gradient detection is performed on the key frame to obtain a gradient value of the key frame; the edge detection operator includes a horizontal convolution factor and a vertical convolution factor; the gradient value includes a horizontal gradient value and a vertical gradient value; Calculating the gradient magnitude and gradient direction of the key frame according to the horizontal gradient value and the vertical gradient value; Traversing the pixel points of the key frame based on the gradient magnitude and the gradient direction, and obtaining edge pixel points of the key frame using a non-maximum suppression algorithm; The complexity of the key frame is determined according to the number of the edge pixels.

3. The screen content encoding method according to claim 1, characterized in that: The encoding process of the screen video image based on the encoding bit rate to obtain an encoded bit stream includes: Obtaining a coding quantization parameter currently used for encoding, and determining an estimated bit rate according to the complexity of a first frame in the screen video image; the complexity of the first frame is the complexity of the first key frame in the screen video image; Calculating the encoding byte size corresponding to the screen video image encoding according to the encoding quantization parameter; If the encoded byte size exceeds a byte threshold corresponding to a preset peak bit rate, iteratively adjusting the quantization encoding parameter until the encoded byte size is less than or equal to the byte threshold, thereby obtaining a target quantization parameter for encoding; Determining a high-bitrate frame in the screen video image based on the target quantization parameter, and performing frame drop processing on the high-bitrate frame; Based on the target quantization parameter, the screen video image after the frame loss process is encoded to obtain an encoded code stream.

4. The screen content encoding method according to claim 1, wherein: Before encoding the screen video image based on the encoding bit rate to obtain an encoded bit stream, the method further includes: Obtaining a timestamp difference between associated timestamps of any two adjacent frames in the screen video image; Performing dynamic frame motion detection on the screen video image based on the timestamp difference to determine whether there is a first target frame that needs to be dropped; If it exists, the first target frame is overwritten with the second target frame, and it is determined whether the first target frame is a key frame; the second target frame is the previous frame of the first target frame; If the first target frame is not a key frame, a frame drop process is performed on the first target frame.

5. The screen content encoding method according to claim 1, wherein: Before extracting the key frames of each picture group of the screen video image, the method further includes: Performing image quality detection and format conversion on the screen video image to convert the screen video image into a preset format; wherein the image quality detection includes a freeze rate detection and a frame rate detection; performing bilinear scaling processing on the screen video image in the preset format, and determining whether to perform filtering processing on the screen video image according to the image quality detection result; If it is determined according to the image quality detection result that the screen video image is to be filtered, Gaussian blur filtering is performed on the screen video image in the preset format.

6. The screen content encoding method according to claim 1, characterized in that: After encoding the screen video image based on the encoding bit rate to obtain an encoded bit stream, the method further includes: The coded code stream is divided into blocks, and based on a preset concurrent number, a preset image quality enhancement operator is called to perform image quality enhancement processing on each block obtained by the division; the image quality enhancement operator includes a Laplace operator.

7. A screen content encoding device, characterized in that: include: An image acquisition module is used to obtain the screen video image to be encoded and extract the key frames of each picture group of the screen video image; a complexity calculation module, configured to perform edge detection on the key frame and determine the complexity of the key frame based on the number of edge pixels of the key frame detected; The encoding module is used to determine an encoding rate according to the complexity, and to encode the screen video image based on the encoding rate to obtain an encoding stream.

8. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of the screen content encoding method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the screen content encoding method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the screen content encoding method according to any one of claims 1 to 6 are implemented.

Images