Video processing method, device, server and storage medium

Abstract

The present disclosure relates to a video processing method, device, server and storage medium. The method includes: the method is applied to a graphics processor, including: receiving an input code stream, decoding the input code stream to obtain a first video code stream; performing video blur processing on the first video code stream to obtain a second video code stream, wherein the video blur processing on the first video code stream is completed under the control of a central processing unit; encoding the second video code stream to obtain an output code stream. The embodiments of the present disclosure can greatly improve the efficiency of video processing.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video processing method, device, server and storage medium. Background Art

[0002] In the field of image processing technology, Gaussian blur is the process of blurring an image by a Gaussian function. The blurred image can be used to produce effects such as bloom, depth of field, heat waves or frosted glass. In the related art, when Gaussian blurring is performed on a video, the GPU (Graphics Processing Unit) is first used to decode the video to be processed, the CPU (Central Processing Unit) is used to perform Gaussian blurring on the decoded video, and then the GPU is used to encode the blurred video. However, the speed of processing the video in the related art is slow. Summary of the invention

[0003] The present disclosure provides a video processing method, device, electronic device and storage medium to at least solve the problem of slow video processing speed in the related art. The technical solution of the present disclosure is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a video processing method is provided, including:

[0005] Receiving an input code stream, and decoding the input code stream to obtain a first video code stream;

[0006] Performing video blur processing on the first video code stream to obtain a second video code stream, wherein the video blur processing on the first video code stream is completed under the control of a central processing unit;

[0007] The second video code stream is encoded to obtain an output code stream.

[0008] In a possible implementation, the performing video blur processing on the first video code stream to obtain the second video code stream, wherein the performing video blur processing on the first video code stream is completed under the control of a central processing unit, and includes:

[0009] A command queue sent by a central processing unit is received, and according to each command in the command queue, a corresponding block image is smoothed to obtain a second video code stream, wherein the block image is obtained by segmenting the video image of the first video code stream.

[0010] In a possible implementation, the image processor includes a plurality of kernel processing objects, and the command queue includes a correspondence between pixels to be processed of the block image and the kernel processing objects;

[0011] The receiving command queue sent by the central processing unit and performing smoothing processing on the corresponding block image according to each command in the command queue to obtain a second video code stream includes:

[0012] According to the correspondence between the pixels to be processed of the block image and the kernel processing object, the kernel processing object is triggered to perform smoothing processing on the corresponding pixels to be processed to obtain a second video code stream.

[0013] In a possible implementation, triggering the kernel processing object to perform smoothing on the corresponding pixels to be processed according to the correspondence between the pixels to be processed of the block image and the kernel processing object includes:

[0014] According to the correspondence between the pixels to be processed of the block image and the kernel processing object, triggering the kernel processing object to obtain the corresponding pixels to be processed from the corresponding kernel buffer, wherein the total number of pixels of the block image is greater than the number of the pixels to be processed;

[0015] The kernel processing object is triggered to perform smoothing processing on the pixels to be processed to obtain a second video code stream.

[0016] In a possible implementation, triggering the kernel processing object to perform smoothing on the pixels to be processed to obtain a second video stream includes:

[0017] Triggering the kernel processing object to perform smoothing on the pixel to be processed to obtain a processed pixel;

[0018] According to the coordinate information of the pixel to be processed, storing the processed pixel in the corresponding kernel memory;

[0019] The processed pixels are obtained from the kernel memory to obtain a second video code stream.

[0020] In a possible implementation, the command queue includes a correspondence between an image data type and a processing method; the command queue sent by the receiving central processor is received, and according to each command of the command queue, a corresponding block image is smoothed to obtain a second video code stream, including:

[0021] Acquire the data type of the block image;

[0022] According to the correspondence between the data type and the processing method of the image, the block image is processed according to the processing method corresponding to the data type to obtain a second video code stream.

[0023] In a possible implementation, the processing of the divided images in a processing manner corresponding to the data type to obtain a second video stream includes:

[0024] If the data type is chrominance data, the block image is smoothed to obtain a second video code stream.

[0025] In a possible implementation, the performing video blur processing on the first video code stream to obtain the second video code stream, wherein the performing video blur processing on the first video code stream is completed under the control of a central processing unit, and includes:

[0026] Obtaining a Gaussian sample matrix from a preconfigured memory object, wherein the Gaussian sample matrix is ​​generated by a central processor according to the obtained variance parameter of the Gaussian function;

[0027] According to the Gaussian sample matrix, Gaussian blur processing is performed on the first video stream to obtain a second video stream.

[0028] In a possible implementation manner, the size of the Gaussian sample matrix is ​​3 to 6 times the variance of the Gaussian function.

[0029] According to a second aspect of an embodiment of the present disclosure, there is provided a video processing device, including:

[0030] A graphics processor, configured to receive an input code stream, decode the input code stream to obtain a first video code stream, perform video blur processing on the first video code stream to obtain a second video code stream, and encode the second video code stream to obtain an output code stream;

[0031] The central processing unit is used to control the graphics processor to execute the step of performing video blur processing on the second video code stream.

[0032] In a possible implementation, the graphics processor, when executing the step of performing video blur processing on the first video code stream to obtain the second video code stream, includes:

[0033] The graphics processor receives a command queue sent by the central processor, and according to each command in the command queue, performs smoothing on the corresponding block image to obtain a second video code stream, wherein the block image is obtained by segmenting the video image of the first video code stream.

[0034] In a possible implementation, the image processor includes a plurality of kernel processing objects, and the command queue includes a correspondence between pixels to be processed of the block image and the kernel processing objects.

[0035] The graphics processor executes the command queue sent by the receiving central processor, and according to each command of the command queue, performs smoothing processing on the corresponding block image to obtain a second video code stream, the steps include:

[0036] According to the correspondence between the pixels to be processed of the block image and the kernel processing object, the kernel processing object is triggered to perform smoothing processing on the corresponding pixels to be processed to obtain a second video code stream.

[0037] In a possible implementation, the step of triggering the kernel processing object to perform smoothing processing on the corresponding pixels to be processed according to the correspondence between the pixels to be processed of the block image and the kernel processing object to obtain the second video code stream includes:

[0038] According to the correspondence between the pixels to be processed of the block image and the kernel processing object, triggering the kernel processing object to obtain the corresponding pixels to be processed from the corresponding kernel buffer, wherein the total number of pixels of the block image is greater than the number of the pixels to be processed;

[0039] The kernel processing object performs smoothing processing on the pixels to be processed to obtain a second video code stream.

[0040] In a possible implementation, the graphics processor, when executing the step of triggering the kernel processing object to perform smoothing on the pixels to be processed to obtain the second video code stream, includes:

[0041] Triggering the kernel processing object to perform smoothing on the pixel to be processed to obtain a processed pixel;

[0042] According to the coordinate information of the pixel to be processed, storing the processed pixel in the corresponding kernel memory;

[0043] The processed pixels are obtained from the kernel memory to obtain a second video code stream.

[0044] In a possible implementation, the command queue includes a correspondence between an image data type and a processing method, and the graphics processor executes the command queue sent by the receiving central processor, and according to each command of the command queue, performs smoothing processing on the corresponding block image to obtain a second video code stream, and the steps include:

[0045] Acquire the data type of the block image;

[0046] According to the correspondence between the data type and the processing method of the image, the block image is processed according to the processing method corresponding to the data type to obtain a second video code stream.

[0047] In a possible implementation, the graphics processor executes the correspondence between the data type and the processing method of the image and processes the block image in a processing method corresponding to the data type to obtain a second video stream, the steps comprising:

[0048] If the data type is chrominance data, the block image is smoothed to obtain a second video code stream.

[0049] In a possible implementation, the graphics processor performs video blur processing on the first video stream to obtain the second video stream, and the steps include:

[0050] Obtaining a Gaussian sample matrix from a preconfigured memory object, wherein the Gaussian sample matrix is ​​generated by a central processor according to the obtained variance parameter of the Gaussian function;

[0051] According to the Gaussian sample matrix, Gaussian blur processing is performed on the first video stream to obtain a second video stream.

[0052] In a possible implementation manner, the size of the Gaussian sample matrix is ​​3 to 6 times the variance of the Gaussian function.

[0053] According to a third aspect of an embodiment of the present disclosure, a server is provided, comprising:

[0054] CPU and GPU;

[0055] a memory for storing instructions executable by the processor;

[0056] Wherein, the graphics processor is configured to execute the instructions to perform the video processing method described in any one of the embodiments of the present disclosure.

[0057] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a graphics processor of a server, the server is enabled to execute the video processing method described in any one of the embodiments of the present disclosure.

[0058] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, and when the instructions are executed by a graphics processor of a server, the server is enabled to execute the video processing method described in any one of the embodiments of the present disclosure.

[0059] The technical solution provided by the embodiment of the present disclosure brings at least the following beneficial effects: In the above-mentioned video processing method, the first decoded video code stream is subjected to Gaussian blur processing by the computing unit of the GPU. The decoding, blur processing, and encoding of the video code stream are all completed on the same GPU. Compared with the traditional method of performing Gaussian blur processing with the help of the CPU, the embodiment of the present disclosure omits the step of transmitting video data between the GPU and the CPU, and due to the hardware advantage of the GPU in graphics processing, the rate of the GPU performing Gaussian blur processing is also greatly improved compared with the CPU. Therefore, the embodiment of the present disclosure can greatly improve the efficiency of video processing.

[0060] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0062] Figure 1 The figure is a flowchart of a video processing method according to an exemplary embodiment.

[0063] Figure 2 The figure is a flowchart of a video processing method according to an exemplary embodiment.

[0064] Figure 3 It is a schematic diagram showing a block image according to an exemplary embodiment.

[0065] Figure 4 is a schematic diagram showing a Gaussian sample matrix before normalization according to an exemplary embodiment.

[0066] Figure 5 is a schematic diagram of a Gaussian sample matrix according to an exemplary embodiment.

[0067] Figure 6 is a schematic block diagram of a video processing device according to an exemplary embodiment.

[0068] Figure 7 It is a block diagram of a server according to an exemplary embodiment. DETAILED DESCRIPTION

[0069] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0070] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0071] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0072] Figure 1 is a flowchart of a video processing method according to an exemplary embodiment. Figure 1 As shown, the decoder in the GPU decodes the coded video. Video coding is a method for compressing the video during video transmission. The coded video in the embodiment of the present disclosure may include MPEG coded video, H.264 coded video, and H.265 coded video. The decoder decodes the coded video to obtain the original bitstream of the image. The computing unit in the GPU performs video blur processing on the original bitstream, accepts the bitstream control of the CPU when performing the video blur processing, and receives the video blur processing parameters, processing methods, processing timing, etc. transmitted by the CPU. The computing unit may include multiple kernel processing objects, and the multiple kernel processing objects may process the blocks of each frame of the video bitstream in parallel. After the processing is completed, the encoder in the GPU encodes the original bitstream into the blurred coded video. The CPU performs relevant configuration tasks before sending relevant control commands to the GPU. In one example, it can be implemented with the help of the OpenCL standard. It includes constructing a context, which includes the operating environment of the CPU and GPU, GPU devices, memory management, and command queues; constructing a kernel object, which is an object for processing on the kernel processing object; constructing a command queue, which is used to control the specific commands executed by the kernel processing object; and constructing a kernel memory, which is used to store the video data required for the kernel processing object to calculate.

[0073] Figure 2 is a flowchart of a video processing method according to an exemplary embodiment. Figure 2 As shown, the method is used for a graphics processor and includes the following steps.

[0074] Step S201: receiving an input code stream, and decoding the input code stream to obtain a first video code stream.

[0075] In the embodiment of the present disclosure, the graphics processor may include a graphics processor having a computing unit, such as Intel SG1. The input code stream may include a code stream compressed by video encoding, such as an MPEG video code stream, an H.264 video code stream, and an H.265 video code stream. The input code stream may be decoded by an operation inverse to the encoding to obtain a first video code stream. The format of the first video code stream includes but is not limited to YUV420P, RGB, CMYK, Lab, etc.

[0076] Step S203, performing video blur processing on the first video code stream to obtain a second video code stream, wherein the video blur processing on the first video code stream is completed under the control of a central processing unit.

[0077] In the embodiment of the present disclosure, the video blur processing may include combining the values ​​of all pixels near the pixel in the video through a preset convolution algorithm, and resetting the size of the pixel, wherein the convolution algorithm may include Gaussian blur, bilateral filtering, etc. The video blur processing may also include median filtering, mean filtering, etc. for the image pixels in the video. The image processor performs video blur processing on the first video stream under the control of the central processing unit. In one example, the central processing unit may provide an API interface to transmit video blur processing parameters, processing methods, processing timing, etc. to manage the program running on the GPU. In another example, taking Gaussian blur as an example, the central processing unit pre-configures parameters such as Gaussian sample matrix, Gaussian blur steps, Gaussian blur frames, and Gaussian blur kernel size for the graphics processor to call during video blur processing. And construct a context, which includes the operating environment of the CPU and GPU, GPU devices, memory management, and command queues; construct a kernel object for objects processed on the kernel processing object; construct a command queue for controlling the specific commands executed by the kernel processing object; and construct a kernel memory for storing video data required for kernel processing object calculation. The CPU executes the commands in the command queue in sequence, with the commands at the head being executed first. The CPU also writes the processing method for image blur into the command queue. After the GPU processes a frame of image, it sends an interrupt command to the CPU, and the CPU controls the GPU to continue processing the next frame of image or end processing.

[0078] Step S205: encode the second video code stream to obtain an output code stream.

[0079] In the embodiment of the present disclosure, the image processor encodes the second video code stream, wherein the encoding method includes but is not limited to MPEG video encoding, H.264 video encoding, and H.265 video encoding, etc. By encoding the second video code stream, an output code stream is obtained.

[0080] In the above-mentioned video processing method, the first decoded video code stream is subjected to Gaussian blur processing by the computing unit of the GPU. The decoding, blur processing, and encoding of the video code stream are all completed on the same GPU. Compared with the traditional method of performing Gaussian blur processing with the help of the CPU, the embodiment of the present disclosure omits the step of transmitting video data between the GPU and the CPU, and due to the hardware advantage of the GPU in graphics processing, the rate of the GPU performing Gaussian blur processing is also greatly improved compared with the CPU. Therefore, the embodiment of the present disclosure can greatly improve the efficiency of video processing.

[0081] In a possible implementation, the performing video blur processing on the first video code stream by the graphics processor to obtain the second video code stream, wherein the performing video blur processing on the first video code stream by the graphics processor is completed under the control of the central processing unit, and includes:

[0082] A command queue sent by a central processing unit is received, and according to each command in the command queue, a corresponding block image is smoothed to obtain a second video code stream, wherein the block image is obtained by segmenting the video image of the first video code stream.

[0083] In the disclosed embodiment, the command queue is used to store commands that need to be executed on the graphics processor, and multiple command queues allow the program in the graphics processor to execute multiple unrelated instructions without synchronization. In one example, the CPU can create a command queue using the OpenCL standard command queue function, for example;

[0084] cl_command_queue clCreateCommandQueue(cl_comtext context,

[0085] cl_device_id device,

[0086] cl_command_queue_properties properties,

[0087] cl_int *errcode_ret)

[0088] Among them, the first parameter indicates the context of the command queue, the second parameter indicates the device (GPU, etc.) to which the command queue is connected, the third parameter indicates a series of properties of the command queue, including setting the execution order of commands, etc., and the fourth parameter returns an error code. In one example, the trigger time, blur processing method, convolution parameters, etc. of the graphics processor can be set in the command queue properties so that the graphics processor performs convolution processing on the first video stream according to the command queue. In the embodiment of the present disclosure, the block image is obtained by segmenting each frame image of the first video stream. For example, the size of a frame image is 256×256, which can be divided into 256 16×16 blocks.

[0089] The graphics processor of the embodiment of the present disclosure receives a command queue sent by the central processing unit. Under the control of the command queue, the block images in the video code stream can be executed in parallel and in order, thereby improving the processing efficiency of the blur processing.

[0090] In a possible implementation, the image processor includes a plurality of kernel processing objects, and the command queue includes a correspondence between pixels to be processed of the block image and the kernel processing objects;

[0091] The receiving command queue sent by the central processing unit and performing smoothing processing on the corresponding block image according to each command in the command queue to obtain a second video code stream includes:

[0092] According to the correspondence between the pixels to be processed of the block image and the kernel processing object, the kernel processing object is triggered to perform smoothing processing on the corresponding pixels to be processed to obtain a second video code stream.

[0093] In the disclosed embodiment, the image processor includes a plurality of kernel processing objects, each of which has a unique hardware ID. The correspondence between the to-be-processed pixels of the block image and the kernel processing objects can be obtained in the following manner: the computing unit of the graphics processing (including all the kernel processing objects) can be regarded as a two-dimensional workspace, and the workspace can be divided into a plurality of workgroups according to the task attributes. Each workgroup corresponds to the block image, and each kernel processing object in the workgroup is responsible for processing a to-be-processed pixel in the block image. In one example, the workgroup can be encoded, such as group_id. Different kernel processing objects determine which workgroup they belong to by obtaining group_id, and determine which output pixel value they calculate by obtaining pixel identification information local_id. For example, for an image with a size of 1280×720, each workgroup includes 16×16 kernel processing objects, and processes a block image of size 16×16, then the above image is divided into (1280 / 16)×(720 / 16) block images. The workgroup range corresponding to the block image can include from (0,0) to (80-1,45-1). The pixel identification information of the work group ranges from (0,0) to (16-1,16-1).

[0094] In the disclosed embodiment, by establishing a correspondence between the pixels to be processed of the block image and the kernel processing objects, the corresponding pixels of the corresponding block image processed by each kernel processing object can be determined. Each kernel processing object divides the work and cooperates in parallel to ensure the accuracy and efficiency of processing.

[0095] Figure 3 FIG. 1 is a schematic diagram showing a block image according to an exemplary embodiment. Figure 3 As shown, triggering the kernel processing object to perform smoothing on the corresponding pixels to be processed according to the correspondence between the pixels to be processed of the block image and the kernel processing object includes:

[0096] According to the correspondence between the pixels to be processed of the block image and the kernel processing object, triggering the kernel processing object to obtain the corresponding pixels to be processed from the corresponding kernel buffer, wherein the total number of pixels of the block image is greater than the number of the pixels to be processed;

[0097] The kernel processing object is triggered to perform smoothing processing on the pixels to be processed to obtain a second video code stream.

[0098] In the embodiment of the present disclosure, each kernel processing object in the image processor processes corresponding pixels of the corresponding block image. Figure 3As shown, the size of the image to be processed is 176×112, corresponding to a workspace of 176×112. The workspace is divided into workgroups of 16×16, so each workgroup includes 16×16 kernel processing objects. The 16×16 kernel processing objects perform convolution blur processing on the pixels of the workgroup 302. The total number of pixels of the obtained block image is greater than the number of pixels to be processed (i.e., the number of pixels in the workgroup). In one example, the block image 301 is cached in the kernel buffer, and the size of the block image 301 is 32×32. Not all pixels in the block image 301 are processed. The kernel processing object processes the pixels in the workgroup 302, and when processing, it needs to use the pixels in the block image 301 except the workgroup 302. In one example, the block image can be obtained in the following manner: with the workgroup as the center, expand the size of 1 / 2 of the workgroup to the surroundings. For example, the kernel size is 16×16 and the block image size is 32×32. In this way, correct calculation can be guaranteed even when the blurring convolution kernel is at its maximum (the blurring convolution kernel is at its maximum and is the same as the size of the working group).

[0099] In the disclosed embodiment, the total number of pixels of the block image stored in the kernel memory is greater than the number of pixels to be processed. Therefore, when the kernel processing object performs convolution processing on the pixels at the edge of the working group, it is not necessary to perform padding processing on the edge of the block image, and the convolution processing can be completed by using the pixel data of the original image to be processed, so the convolution result is more accurate and reliable.

[0100] In a possible implementation, triggering the kernel processing object to perform smoothing on the pixels to be processed to obtain a second video stream includes:

[0101] Triggering the kernel processing object to perform smoothing on the pixel to be processed to obtain a processed pixel;

[0102] According to the coordinate information of the pixel to be processed, the processed pixel is stored in the corresponding kernel memory; the processed pixel is obtained from the kernel memory to obtain the second video code stream

[0103] In the disclosed embodiment, the work group corresponds to the block image. In one example, the work group can be encoded, such as group_id. Different kernel processing objects determine which work group they belong to by obtaining group_id, and determine which output pixel value they calculate by obtaining pixel identification information local_id. The group_id and local_id can also represent the coordinate information of the block image, and according to the coordinate information, the calculation result of the kernel processing object is stored in the kernel memory.

[0104] In the disclosed embodiment, the block images after convolution processing are stored in the corresponding kernel memory according to the coordinate information of the block images, which is beneficial for obtaining the complete second video code stream directly from the kernel memory according to the coordinate information during the subsequent image encoding.

[0105] In a possible implementation, the command queue includes a correspondence between an image data type and a processing method; the command queue sent by the receiving central processor is received, and according to each command of the command queue, a corresponding block image is smoothed to obtain a second video code stream, including:

[0106] Acquire the data type of the block image;

[0107] According to the correspondence between the data type and the processing method of the image, the block image is processed according to the processing method corresponding to the data type to obtain a second video code stream.

[0108] In the disclosed embodiment, the data type includes the color channel type in the color space. For example, for an RGB image, the data type can be divided into R (red) type, G (green) type, and B (blue) type. For YUV, the data type can be divided into chrominance data (U type or V type) and brightness data (Y type). In one example, if the data type is chrominance data, convolution processing is performed on each frame of the image to obtain a second video stream. In another example, if the data type is brightness data, the brightness data is directly stored in a memory without convolution processing.

[0109] In the embodiment of the present disclosure, for different data types, each frame of the image is processed in a processing manner corresponding to the data type. For example, for chromaticity data, since it contains more texture information, processing the chromaticity data has a greater impact on the blurring result. However, since the influence of brightness data on texture information is relatively small, brightness data is not processed, which is conducive to improving the processing efficiency of blurring.

[0110] In a possible implementation, the performing video blur processing on the first video code stream to obtain the second video code stream, wherein the performing video blur processing on the first video code stream is completed under the control of a central processing unit, and includes:

[0111] Obtaining a Gaussian sample matrix from a preconfigured memory object, wherein the Gaussian sample matrix is ​​generated by a central processor according to the obtained variance parameter of the Gaussian function;

[0112] According to the Gaussian sample matrix, Gaussian blur processing is performed on the first video stream to obtain a second video stream.

[0113] In the disclosed embodiment, Gaussian blur is a method of applying normal distribution to image processing. Since normal distribution is a "bell" shaped curve, the closer to the center, the larger the value; the farther away from the center, the smaller the value. When calculating the Gaussian sample matrix, the "center point" of the normal distribution curve bell can be used as the origin, and other points can be assigned weights according to their positions on the normal curve to obtain a Gaussian sample matrix. In one example, the Gaussian sample matrix can be obtained by sampling the variance of the Gaussian function at the center point of each pixel and then normalizing it. For example, the variance of the Gaussian function is set to 1.5, and the Gaussian sample matrix with a blur radius of 1 before normalization is as follows Figure 4 As shown by Figure 4 It can be seen that the center point of the Gaussian sample matrix has the largest weight, the upper and lower values ​​are symmetrical and equal, and the left and right values ​​are symmetrical and equal. Figure 5 As shown, due to Figure 4 The sum of the weights of the 9 points in is equal to 0.4787147. The weights need to be normalized so that the sum of the weights of the 9 points is equal to 1. Therefore, Figure 4 The 9 values ​​in need to be divided by 0.4787147 respectively. The final Gaussian sample matrix is ​​obtained. By convolving the image with the Gaussian sample matrix, the new value of each pixel can be obtained as the weighted average of the pixel neighborhood.

[0114] In an embodiment of the present disclosure, the method for performing video blur processing on the first video code stream can be used for Gaussian blur, and the blur processing process includes determining a Gaussian sample matrix through a variance value of a Gaussian function, and convolving the first video code stream using the Gaussian sample matrix. The present disclosure can improve the video processing efficiency of Gaussian blur.

[0115] In one possible implementation, the Gaussian function of each point on the blurred image is not zero, which means that the entire image needs to be included in the calculation of each pixel. In fact, in the process of sampling the Gaussian function, pixels with a distance greater than a preset multiple of the variance have a sufficiently small effect, so the contribution of pixels outside this range can be ignored. Therefore, in one example, the size of the Gaussian sample matrix is ​​3 to 6 times the variance of the Gaussian function.

[0116] The disclosed embodiment does not determine the Gaussian sample matrix based on all pixels, but determines the Gaussian sample matrix based on pixel points with 3 to 4 times the variance outside the center point, thereby improving processing efficiency while ensuring the accuracy of the calculation results.

[0117] It should be understood that, although the various steps in the flow chart are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0118] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0119] Figure 6 is a schematic block diagram of a video processing device according to an exemplary embodiment. Figure 6 , the device comprises:

[0120] The graphics processor 601 is used to receive an input code stream, decode the input code stream to obtain a first video code stream, perform video blur processing on the first video code stream to obtain a second video code stream, and encode the second video code stream to obtain an output code stream.

[0121] The central processing unit 602 is used to control the graphics processor to execute the step of performing video blur processing on the second video code stream.

[0122] In a possible implementation, in a possible implementation, the graphics processor, when executing the step of performing video blur processing on the first video code stream to obtain the second video code stream, includes:

[0123] The graphics processor receives a command queue sent by the central processor, and according to each command in the command queue, performs smoothing on the corresponding block image to obtain a second video code stream, wherein the block image is obtained by segmenting the video image of the first video code stream.

[0124] In a possible implementation, the image processor includes a plurality of kernel processing objects, and the command queue includes a correspondence between pixels to be processed of the block image and the kernel processing objects.

[0125] The graphics processor executes the command queue sent by the receiving central processor, and according to each command of the command queue, performs smoothing processing on the corresponding block image to obtain a second video code stream, the steps include:

[0126] According to the correspondence between the pixels to be processed of the block image and the kernel processing object, the kernel processing object is triggered to perform smoothing processing on the corresponding pixels to be processed to obtain a second video code stream.

[0127] In a possible implementation, the step of triggering the kernel processing object to perform smoothing processing on the corresponding pixels to be processed according to the correspondence between the pixels to be processed of the block image and the kernel processing object to obtain the second video code stream includes:

[0128] According to the correspondence between the pixels to be processed of the block image and the kernel processing object, triggering the kernel processing object to obtain the corresponding pixels to be processed from the corresponding kernel buffer, wherein the total number of pixels of the block image is greater than the number of the pixels to be processed;

[0129] The kernel processing object performs smoothing processing on the pixels to be processed to obtain a second video code stream.

[0130] In a possible implementation, the graphics processor, when executing the step of triggering the kernel processing object to perform smoothing on the pixels to be processed to obtain the second video code stream, includes:

[0131] Triggering the kernel processing object to perform smoothing on the pixel to be processed to obtain a processed pixel;

[0132] According to the coordinate information of the pixel to be processed, storing the processed pixel in the corresponding kernel memory;

[0133] The processed pixels are obtained from the kernel memory to obtain a second video code stream.

[0134] In a possible implementation, the command queue includes a correspondence between an image data type and a processing method, and the graphics processor executes the command queue sent by the receiving central processor, and according to each command of the command queue, performs smoothing processing on the corresponding block image to obtain a second video code stream, and the steps include:

[0135] Acquire the data type of the block image;

[0136] According to the correspondence between the data type and the processing method of the image, the block image is processed according to the processing method corresponding to the data type to obtain a second video code stream.

[0137] In a possible implementation, the graphics processor executes the correspondence between the data type and the processing method of the image and processes the block image in a processing method corresponding to the data type to obtain a second video stream, the steps comprising:

[0138] If the data type is chrominance data, the block image is smoothed to obtain a second video code stream.

[0139] In a possible implementation, the graphics processor performs video blur processing on the first video stream to obtain the second video stream, and the steps include:

[0140] Obtaining a Gaussian sample matrix from a preconfigured memory object, wherein the Gaussian sample matrix is ​​generated by a central processor according to the obtained variance parameter of the Gaussian function;

[0141] According to the Gaussian sample matrix, Gaussian blur processing is performed on the first video stream to obtain a second video stream.

[0142] In a possible implementation manner, the size of the Gaussian sample matrix is ​​3 to 6 times the variance of the Gaussian function.

[0143] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0144] Figure 7 7 is a block diagram of a server 700 for video processing according to an exemplary embodiment. For example, the electronic device 700 may be a server. Figure 7 The server 700 includes a processing component 720, which further includes one or more processors, and a memory resource represented by a memory 722 for storing instructions, such as applications, that can be executed by the processing component 720. The application stored in the memory 722 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 720 is configured to execute instructions to perform the above method.

[0145] The server 700 may further include a power supply component 724 configured to perform power management of the server 700, a wired or wireless network interface 726 configured to connect the electronic device 700 to a network, and an input / output (I / O) interface 728. The electronic device 700 may operate based on an operating system stored in the memory 722, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.

[0146] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 722 including instructions, and the above instructions can be executed by a processor of the server 700 to complete the above method. The storage medium can be a computer-readable storage medium, for example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0147] In an exemplary embodiment, a computer program product is also provided. The computer program product includes instructions, and the instructions can be executed by a processor of the server 700 to implement the above method.

[0148] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0149] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0150] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that: The method is applied to a graphics processor, comprising: Receiving an input code stream, and decoding the input code stream to obtain a first video code stream; The first video code stream is subjected to video blur processing to obtain a second video code stream, wherein the video blur processing of the first video code stream is completed under the control of a central processing unit; the video blur processing of the first video code stream to obtain the second video code stream includes: according to the correspondence between the pixels to be processed of the block image and the kernel processing object, triggering the kernel processing object to obtain the corresponding pixels to be processed from the corresponding kernel buffer; triggering the kernel processing object to perform smoothing processing on the pixels to be processed to obtain the second video code stream, wherein the total number of pixels of the block image is greater than the number of the pixels to be processed, and the block image is obtained by segmenting the video image of the first video code stream; the graphics processor includes a plurality of kernel processing objects; wherein the kernel processing object performs smoothing processing on the pixels to be processed, including: if the data type of the block image is chromaticity data, performing convolution processing on the chromaticity data; if the data type of the block image is luminance data, storing the luminance data in a memory; The second video code stream is encoded to obtain an output code stream; wherein the decoding of the input code stream, the video blur processing of the first video code stream, and the encoding of the second video code stream are all implemented by the graphics processor.

2. The method according to claim 1, characterized in that The performing video blur processing on the first video code stream to obtain a second video code stream, wherein the performing video blur processing on the first video code stream is completed under the control of a central processing unit, and includes: A command queue sent by a central processing unit is received, and according to each command in the command queue, a corresponding block image is smoothed to obtain a second video code stream.

3. The method according to claim 2, characterized in that The command queue includes the correspondence between the pixels to be processed of the block image and the kernel processing objects.

4. The method according to claim 1, characterized in that Triggering the kernel processing object to perform smoothing on the pixels to be processed to obtain a second video stream, including: Triggering the kernel processing object to perform smoothing on the pixel to be processed to obtain a processed pixel; According to the coordinate information of the pixel to be processed, storing the processed pixel in the corresponding kernel memory; The processed pixels are obtained from the kernel memory to obtain a second video code stream.

5. The method according to claim 2, characterized in that: The command queue includes the corresponding relationship between the image data type and the processing method; The receiving command queue sent by the central processing unit and performing smoothing processing on the corresponding block image according to each command of the command queue to obtain a second video code stream includes: Acquire the data type of the block image; According to the correspondence between the data type and the processing method of the image, the block image is processed according to the processing method corresponding to the data type to obtain a second video code stream.

6. The method according to claim 5, characterized in that The step of processing the divided images in a processing manner corresponding to the data type according to the correspondence between the data type and the processing manner of the image to obtain a second video code stream includes: If the data type is chrominance data, the block image is smoothed to obtain a second video code stream.

7. The method according to any one of claims 1 to 6, characterized in that The performing video blur processing on the first video code stream to obtain a second video code stream, wherein the performing video blur processing on the first video code stream is completed under the control of a central processing unit, and includes: Obtaining a Gaussian sample matrix from a preconfigured memory object, wherein the Gaussian sample matrix is ​​generated by a central processor according to the obtained variance parameter of the Gaussian function; According to the Gaussian sample matrix, Gaussian blur processing is performed on the first video stream to obtain a second video stream.

8. The method according to claim 7, characterized in that The size of the Gaussian sample matrix is ​​3 to 6 times the variance of the Gaussian function.

9. A video processing device, characterized in that: include: A graphics processor, configured to receive an input code stream, decode the input code stream to obtain a first video code stream, perform video blur processing on the first video code stream to obtain a second video code stream, and encode the second video code stream to obtain an output code stream; The video blurring process is performed on the first video code stream to obtain the second video code stream, including: triggering the kernel processing object to obtain the corresponding pixel to be processed from the corresponding kernel buffer according to the corresponding relationship between the pixel to be processed of the block image and the kernel processing object; triggering the kernel processing object to perform smoothing process on the pixel to be processed to obtain the second video code stream, wherein the total number of pixels of the block image is greater than the number of the pixel to be processed, and the block image is obtained by segmenting the video image of the first video code stream; the graphics processor includes a plurality of kernel processing objects; wherein the kernel processing object performs smoothing process on the pixel to be processed, including: if the data type of the block image is chromaticity data, performing convolution process on the chromaticity data; if the data type of the block image is luminance data, storing the luminance data in a memory; The central processing unit is used to control the graphics processor to execute the step of performing video blur processing on the second video code stream.

10. The device according to claim 9, characterized in that The step of performing video blur processing on the first video code stream to obtain the second video code stream comprises: The graphics processor receives the command queue sent by the central processor, and performs smoothing processing on the corresponding block image according to each command in the command queue to obtain a second video code stream.

11. The device according to claim 10, characterized in that The command queue includes the correspondence between the pixels to be processed of the block image and the kernel processing objects.

12. The device according to claim 9, characterized in that The step of triggering the kernel processing object to perform smoothing on the pixels to be processed to obtain a second video code stream comprises: Triggering the kernel processing object to perform smoothing on the pixel to be processed to obtain a processed pixel; According to the coordinate information of the pixel to be processed, storing the processed pixel in the corresponding kernel memory; The processed pixels are obtained from the kernel memory to obtain a second video code stream.

13. The device according to claim 10, characterized in that The command queue includes a correspondence between the image data type and the processing method. The graphics processor executes the command queue sent by the receiving central processor, and according to each command of the command queue, performs smoothing processing on the corresponding block image to obtain a second video code stream. The steps include: Acquire the data type of the block image; According to the correspondence between the data type and the processing method of the image, the block image is processed according to the processing method corresponding to the data type to obtain a second video code stream.

14. The device according to claim 13, characterized in that The graphics processor processes the divided image in accordance with the processing method corresponding to the data type to obtain a second video stream, the steps comprising: If the data type is chrominance data, the block image is smoothed to obtain a second video code stream.

15. The device according to any one of claims 9 to 14, characterized in that The graphics processor performs video blur processing on the first video code stream to obtain a second video code stream, and the steps include: Obtaining a Gaussian sample matrix from a preconfigured memory object, wherein the Gaussian sample matrix is ​​generated by a central processor according to the obtained variance parameter of the Gaussian function; According to the Gaussian sample matrix, Gaussian blur processing is performed on the first video stream to obtain a second video stream.

16. The device according to claim 15, characterized in that The size of the Gaussian sample matrix is ​​3 to 6 times the variance of the Gaussian function.

17. A server, characterized in that: include: CPU and GPU; a memory for storing instructions executable by the processor; The graphics processor is configured to execute the instructions to implement the video processing method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a graphics processor of a server, the server is enabled to execute the video processing method according to any one of claims 1 to 8.

19. A computer program product, comprising instructions, characterized in that: When the instruction is executed by the graphics processor of the server, the server is enabled to execute the video processing method according to any one of claims 1 to 8.

Images