Video processing method and device, electronic equipment and computer storage medium

Abstract

The application discloses a video processing method and device, electronic equipment and computer storage medium, and the method comprises the steps of obtaining a first image frame obtained by image frame interlaced sampling of a target video, and a target image frame currently processed in the first image frame; determining a reference image frame adjacent to the target image frame in the first image frame; determining state information corresponding to a pixel point in the target image frame based on the target image frame and the reference image frame; performing recovery processing on the pixel point to be recovered in the target image frame based on the state information to obtain a processed image frame of the target image frame; obtaining the processed image frame obtained after the recovery processing of each first image frame to obtain a processed video corresponding to the target video. The application can realize video de-interlacing processing only through adjacent image frames, reduces resource consumption cost, and improves de-interlacing display effect.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a video processing method, apparatus, electronic device, and computer storage medium. Background Technology

[0002] Existing deinterlacing algorithms mainly include non-motion compensation algorithms and motion compensation algorithms. Non-motion compensation algorithms are limited by image distortion, blurring, and jitter. These algorithms are based on copying, line averaging, and edge-based interpolation, resulting in low accuracy and poor deinterlacing display. Motion compensation algorithms can deliver better visual effects, but they require more hardware resources, increasing implementation costs. Furthermore, the selection of the block matching criterion threshold directly affects the display effect. A smaller threshold, with strict control over the quality of matched blocks, results in relatively good matching, but noise can cause mismatches, consuming more resources without achieving the desired block matching effect. Conversely, a larger threshold may result in similar blocks replacing the original matched blocks. Summary of the Invention

[0003] This application aims to at least address one of the technical problems existing in the related art. To this end, embodiments of this application provide a video processing method, apparatus, electronic device, and computer storage medium that can reduce resource consumption costs and improve deinterlacing display effects.

[0004] In a first aspect, embodiments of this application provide a video processing method, including:

[0005] Acquire the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame;

[0006] Identify a reference image frame in the first image frame that is adjacent to the target image frame;

[0007] Based on the target image frame and the reference image frame, determine the state information corresponding to the pixels in the target image frame;

[0008] Based on the state information, the pixels to be recovered in the target image frame are restored to obtain the processed image frame of the target image frame;

[0009] The processed image frame obtained after performing the restoration process on each of the first image frames is obtained to obtain the processed video corresponding to the target video.

[0010] Secondly, embodiments of this application provide a video processing apparatus, including:

[0011] The first acquisition module is used to acquire the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame;

[0012] The determining module is used to determine a reference image frame in the first image frame that is adjacent to the target image frame;

[0013] The detection module is used to determine the state information corresponding to the pixels in the target image frame based on the target image frame and the reference image frame;

[0014] The processing module is used to perform recovery processing on the pixels to be recovered in the target image frame based on the state information, so as to obtain the processed image frame of the target image frame;

[0015] The second acquisition module is used to acquire the processed image frame obtained after the restoration process of each of the first image frames, and to obtain the processed video corresponding to the target video.

[0016] Thirdly, embodiments of this application also provide an electronic device, including a memory storing multiple computer programs; a processor loads the computer programs from the memory to execute any of the video processing methods provided in embodiments of this application.

[0017] Fourthly, embodiments of this application also provide a computer-readable storage medium storing multiple computer programs adapted for loading by a processor to execute any of the video processing methods provided in embodiments of this application.

[0018] Fifthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements any of the video processing methods provided in embodiments of this application.

[0019] This application embodiment only requires adjacent image frames to achieve video deinterlacing, eliminating the need for block matching and thus reducing resource consumption costs. Simultaneously, by determining the state information of pixels in the target image frame using adjacent image frames, missing pixels in the target image frame can be restored based on this state information. This eliminates the need for interpolation algorithms, avoiding the problem of low accuracy in interpolated images, and also eliminates the need to set a matching criterion threshold for block matching, avoiding the impact of the matching criterion threshold on the display effect, thereby improving the deinterlacing display effect. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the video processing method provided in the embodiments of this application;

[0022] Figure 2 This is one of the schematic diagrams of pixels provided in the embodiments of this application;

[0023] Figure 3 This is a second schematic diagram of the pixels provided in the embodiments of this application;

[0024] Figure 4 This is a schematic diagram of the structure of the video processing device provided in the embodiments of this application;

[0025] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. At the same time, in the description of the embodiments of this application, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0027] This application provides a video processing method, apparatus, electronic device, and computer storage medium. Specifically, this application will be described from the perspective of a video processing apparatus, which can be integrated into an electronic device, meaning the video processing method of this application can be executed by the electronic device. Optionally, the electronic device includes a terminal device. The terminal device can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, game console, smart interactive whiteboard, or personal computer (PC), etc. Optionally, the electronic device includes a server, which can be an independent server, a server network, or a server cluster, including but not limited to computers, network hosts, single network servers, sets of network servers, or cloud servers composed of servers. The cloud server consists of a large number of computers or network servers based on cloud computing.

[0028] It should be noted that the order of description in the following embodiments is not intended to limit the preferred order of embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the figures.

[0029] Existing deinterlacing algorithms mainly include non-motion restoration algorithms and motion restoration algorithms. Non-motion restoration algorithms are limited by image distortion, blurring, and jitter. These algorithms are based on copying, line averaging, and edge-based interpolation, resulting in low accuracy of the difference image and poor deinterlacing display. Motion restoration algorithms can deliver better visual effects, but they require more hardware resources, increasing implementation costs. Furthermore, the selection of the matching threshold for block matching directly affects the display effect. A smaller threshold, with strict control over the quality of matched blocks, results in relatively good matching, but noise can cause mismatches, consuming more resources without achieving the desired block matching effect. Conversely, a larger threshold may result in similar blocks replacing the original matched blocks.

[0030] To address the problems in existing technologies, the inventors of this application propose a video processing method using adjacent image frames. This method involves processing the current processing frame (target image frame) and its adjacent reference image frames within an image frame obtained by interlacing sampling. This achieves video deinterlacing without requiring block matching, thus reducing resource consumption costs. Furthermore, by determining the state information of pixels in the target image frame using adjacent image frames, missing pixels in the target image frame can be restored based on this state information. This eliminates the need for interpolation algorithms, avoiding the problem of low accuracy in interpolated images, and also eliminates the need to set a matching criterion threshold for block matching, avoiding the impact of the matching criterion threshold on the display effect, thereby improving the deinterlacing display effect.

[0031] Optionally, this application uses a video processing device as the execution subject for illustrative purposes. The following detailed description is provided in conjunction with the accompanying drawings.

[0032] refer to Figure 1 , Figure 1 This is a flowchart illustrating the video processing method provided in the embodiments of this application. The specific flow of the video processing method provided in the embodiments of this application can be as follows: steps 10 to 50, including:

[0033] Step 10: Obtain the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame.

[0034] Optionally, the video processing device receives the target video, which may be transmitted from the user terminal to the video processing device, obtained by the video processing device from a cloud multimedia database, or dragged into the video processing device by the user from the multimedia database.

[0035] Furthermore, the video processing device performs interlaced sampling of the image frames in the target video to obtain a first image frame after interlaced sampling. The first image frame includes a preset number of image frames, which is set according to actual conditions, for example, the preset number is 4, 6, 8, 10, etc.

[0036] Furthermore, the video processing device acquires the currently processed image frame in the first image frame and determines the image frame as the target image frame in the first image frame.

[0037] Step 20: Determine the reference image frame adjacent to the target image frame in the first image frame.

[0038] Optionally, the video processing device acquires a predetermined number of image frames adjacent to the target image frame in the first image frame, and determines the predetermined number of adjacent image frames as reference image frames of the target image frame.

[0039] For example, if the quantity is set to 3, the first image frame includes 8 image frames, namely image frame 1, image frame 2, image frame 3, image frame 4, image frame 5, image frame 6, image frame 7 and image frame 8. The image frame being processed is image frame 3. Therefore, image frame 3 is the target image frame in the first image frame. Thus, the reference image frames for image frame 3 can be image frame 1, image frame 2 and image frame 4, image frame 3 can also be image frame 2, image frame 4 and image frame 5, and image frame 4, image frame 5 and image frame 6.

[0040] It should be noted that in the first image frame of this application embodiment, the pixel interlacing sampling method is the same for image frames separated by one image frame. This can also be understood as the pixel interlacing sampling method being the same for image frames with the same odd and even fields. Continuing with the above embodiment, the pixel interlacing sampling method is the same for image frame 1 and image frame 3, and the pixel interlacing sampling method is the same for image frame 2 and image frame 4.

[0041] Step 30: Based on the target image frame and the reference image frame, determine the state information corresponding to the pixels in the target image frame.

[0042] Optionally, the video processing device determines frame difference data based on the target image frame and the reference image frame. Frame difference data can be understood as the pixel difference between pixels in the image frame.

[0043] Furthermore, the video processing device determines the state information corresponding to the pixels in the target image frame based on the frame difference data. The state information indicates whether the pixels in the target image frame are in motion, that is, whether the pixels in the target image are in a stationary state (non-motion state) or in a motion state, as described in steps 301 to 303.

[0044] Step 40: Based on the state information, perform recovery processing on the pixels to be recovered in the target image frame to obtain the processed image frame of the target image frame.

[0045] Furthermore, the video processing device performs recovery processing on the pixels to be recovered in the target image frame according to the status information to obtain the processed image frame of the target image frame. The pixels to be recovered are the missing pixels after interlacing sampling. Therefore, it can be understood that the video processing device performs recovery processing on the missing pixels in the target image frame according to the status information to obtain the processed image frame of the target image frame, as described in steps 401 to 403.

[0046] Step 50: Obtain the processed image frame after restoration processing of each first image frame, and obtain the processed video corresponding to the target video.

[0047] Optionally, the video processing device acquires all the first image frames and performs restoration processing to obtain the processed image frames, and then merges all the processed image frames to obtain the processed video corresponding to the target video.

[0048] This application embodiment only requires adjacent image frames to achieve video deinterlacing, eliminating the need for block matching and thus reducing resource consumption costs. Simultaneously, by determining the state information of pixels in the target image frame using adjacent image frames, missing pixels in the target image frame can be restored based on this state information. This eliminates the need for interpolation algorithms, avoiding the problem of low accuracy in interpolated images, and also eliminates the need to set a matching criterion threshold for block matching, avoiding the impact of the matching criterion threshold on the display effect, thereby improving the deinterlacing display effect.

[0049] In an optional embodiment, steps 301 to 302 are described as follows:

[0050] Step 301: Based on the target image frame and the reference image frame, obtain at least two frame difference data;

[0051] Step 302: If at least two frame difference data satisfy the corresponding preset conditions, then determine that the state information of the pixel in the target image frame is in a static state.

[0052] Step 303: If at least one of the two frame difference data does not meet the corresponding preset condition, then the state information of the pixel in the target image frame is determined to be in motion state.

[0053] Optionally, the video processing device acquires a first reference image frame in the reference image frame that has the same parity field as the target image frame, and performs frame difference calculation on the target image frame and the first reference image frame. At the same time, it performs frame difference calculation on the image frames with the same parity field in the remaining second reference image frames of the reference image frame to obtain at least two frame difference data.

[0054] For example, the first image frame includes image frame 1, image frame 2, image frame 3 and image frame 4, and the target image frame is image frame 2. Therefore, the frame difference between image frame 2 and image frame 4 is calculated, and the frame difference between image frame 1 and image frame 3 is calculated to obtain two frame difference data.

[0055] For example, if the first image frame includes image frame 1, image frame 2, image frame 3, image frame 4, image frame 5, and image frame 6, and the target image frame is image frame 2, then frame difference calculations are performed on image frames 2, 4, and 6, and on image frames 1, 3, and 5, resulting in two frame difference data points. Alternatively, if the first image frame includes image frames 1, 2, 3, 4, 5, 6, 7, and 8, and the target image frame is image frame 2, then frame difference calculations are performed on image frames 2 and 4, 1 and 3, 5 and 7, and 6 and 8, resulting in four frame difference data points.

[0056] Furthermore, taking the first image frame as an example, which includes image frame 1, image frame 2, image frame 3, and image frame 4, image frame 1, image frame 2, image frame 3, and image frame 4 are respectively image frames f n-1 Image frame f n Image frame f n+1 and image frame f n+2 The target image frame is image frame 2. The process of calculating the frame difference data is as follows:

[0057] Optionally, the video processing device will process the image frames f n pixels With image frame f n+2 pixels Frame difference processing is performed to obtain the first frame difference data. Therefore, the first frame difference data can represent... Furthermore, the video processing device will process the image frames f n--1 pixels With image frame f n+1 pixels Frame difference processing is performed to obtain the second frame difference data. Therefore, the second frame difference data represents...

[0058] Optionally, the video processing device processes the frame difference data P. n,n+2 and frame difference data P n-1,n+1 In this embodiment, dilation processing is performed using a preset dilation template. For example, a 3×3 dilation template is used to dilate the frame difference data P. n,n+2 and frame difference data P n-1,n+1 Dilation is performed to obtain frame difference data P. n,n+2 Dilated frame difference data D n,n+2 and frame difference data P n-1,n+1 Dilated frame difference data D n-1,n+1 In this embodiment, the purpose of dilation processing is to better preserve image details and reduce the impact of brightness variations on motion estimation.

[0059] Furthermore, the video processing device processes the dilated frame difference data D n,n+2 And dilated frame difference data D n-1,n+1 In this embodiment, erosion processing is performed using a preset erosion template. For example, a 3×3 erosion template is used to erode at least two dilated frame difference data to obtain dilated frame difference data D. n,n+2 E-frame difference data after corrosion n,n+2 And dilated frame difference data D n-1,n+1 E-frame difference data after corrosion n-1,n+1In this embodiment, the purpose of erosion processing on the dilated image is to suppress the influence of noise. The frame difference data after dilation and erosion can effectively preserve edge details and suppress noise to a certain extent. Therefore, the specific process of motion estimation is as follows:

[0060] Optionally, the video processing device acquires the eroded frame difference data E. n,n+2 First brightness and post-erosion frame difference data E n-1,n+1 The second brightness. If it is determined that the first brightness is less than or equal to the threshold Thre, and the second brightness is less than or equal to the threshold Thre, the video processing device determines that the pixels in the target image frame are in a stationary state. If it is determined that the first brightness is greater than the threshold Thre, and / or the second brightness is greater than the threshold Thre, the video processing device determines that the pixels in the target image frame are in a moving state, as shown in Table 1.

[0061] Table 1. Exercise Estimation Judgment Table

[0062]

[0063] This application embodiment only requires the frame difference data of adjacent image frames to achieve video deinterlacing, without the need for block matching, thus reducing resource consumption costs. Simultaneously, by determining the state information of pixels in the target image frame using the frame difference data, missing pixels in the target image frame can be restored based on this state information. This eliminates the need for interpolation algorithms, avoiding the problem of low accuracy in interpolated images, and also eliminates the need to set a matching criterion threshold for block matching, avoiding the impact of the matching criterion threshold on the display effect, thereby improving the deinterlacing display effect.

[0064] In an optional embodiment, steps 401 to 403 are described as follows:

[0065] Step 401: Obtain the first pixel in the target image frame;

[0066] Step 402: Based on the target position of the first pixel and the state information, determine the second pixel in the reference image frame;

[0067] Step 403: Based on the state information and the second pixel, perform recovery processing on the first pixel to obtain the processed image frame of the target image frame.

[0068] Optionally, the video processing device acquires a first pixel in the target image frame, where the first pixel is any pixel in the target image frame that is missing a pixel.

[0069] Furthermore, the video processing device determines the second pixel in the reference image frame based on the target position and state information of the first pixel. Specifically: if the state information indicates that the pixel in the target image frame is stationary, then the reference image frame is determined to include at least the previous and next frames of the target image frame, and the obtained second pixel is the pixel located at the target position in the reference image frame. If the state information indicates that the pixel in the target image frame is in motion, then the reference image frame is determined to include at least the previous and next frames of the target image frame, and the second pixel includes the pixel located at the target position in the reference image frame, as well as pixels in adjacent positions to the target position in the reference image frame.

[0070] Furthermore, the video processing device performs recovery processing on the first pixel based on the status information and the second pixel to obtain the processed image frame of the target image frame, specifically as follows:

[0071] If the state information indicates that the pixels in the target image frame are stationary, then the average value of the pixels located at the target position in the previous and next frames of the target image frame is calculated, and the average value is used to restore and fill the first pixel. If the state information indicates that the pixels in the target image frame are in motion, then a weighted calculation is performed based on the pixel values ​​of the pixels located at the target position in the previous and next frames of the target image frame, as well as the pixel values ​​of the pixels at neighboring positions in the reference image frame. The weighted calculation result is used to restore and fill the first pixel, as described in steps 4031 to 4035.

[0072] This application embodiment can restore missing pixels in the target image frame based on the status information, without the need to use interpolation algorithms, thus avoiding the problem of low accuracy of interpolated images. It also does not require setting a matching criterion threshold for block matching, thus avoiding the impact of the matching criterion threshold on the display effect, thereby improving the deinterlacing display effect.

[0073] In an optional embodiment, steps 4031 to 4032 are described as follows:

[0074] Step 4031: Calculate the first target pixel value based on the pixel value of the second pixel.

[0075] Step 4032: Perform recovery processing on the first pixel based on the first target pixel value to obtain the processed image frame of the target image frame.

[0076] Optional, see reference Figure 2 , Figure 2 This is one of the schematic diagrams of pixels provided in the embodiments of this application. In this embodiment, the target image frame is... Figure 2 Image frame f in nThe target position of the first pixel includes the target column number i and the target row number j. Therefore, if the state information is a static state, the video processing device will determine the target column number and target row number in the image frame f respectively. n-1 Obtain the corresponding pixel and the image frame f n+1 The corresponding pixel is obtained from the image frame f, and the second pixel is obtained. n-1 Pixels and image frames f n+1 The number of rows and columns of pixels in the target are the same as the number of columns and rows in the target.

[0077] For example, the first pixel is f n (i,j), meaning the target column number and target row number are in the i-th row and j-th column, therefore, the second pixel is the image frame f. n-1 The pixel f in the i-th row and j-th column n-1 (i,j), image frame f n+1 The pixel f in the i-th row and j-th column n+1 (i,j).

[0078] Furthermore, the video processing device, based on the second pixel point f n-1 (i,j) and f n+1 Given the pixel value of (i,j), calculate the first target pixel value, and perform restoration processing on the first pixel based on the first target pixel value to obtain the processed image frame of the target image frame. The specific process is as follows:

[0079] According to pixel f n-1 (i,j) and pixel f n+1 The average value of the pixel values ​​at (i,j) is calculated to obtain the first target pixel value. This first target pixel value is then used to fill in and restore the first pixel, yielding the pixel value of the first pixel. The specific formula for calculating the first target pixel value is as follows:

[0080]

[0081] Among them, P n (i,j) represents the first target pixel value, P n-1 (i,j) represents pixel f n-1 The pixel value of (i,j), P n+1 (i,j) represents pixel f n+1 The pixel value of (i,j).

[0082] In this embodiment of the application, when the state information is in a static state, the missing pixels in the target image frame are restored according to the pixel values ​​of the pixels in the adjacent image frames of the target image frame, making the video playback process smoother and improving the deinterlacing display effect.

[0083] In an optional embodiment, steps 4033 to 4035 are described as follows:

[0084] Step 4033: Obtain the third pixel in the target image frame based on the target location. The third pixel is the pixel adjacent to the first pixel.

[0085] Step 4034: Calculate the second target pixel value based on the pixel value corresponding to the second pixel point, the pixel value corresponding to the third pixel point, and the corresponding weights;

[0086] Step 4035: Perform recovery processing on the first pixel based on the second target pixel value to obtain the processed image frame of the target image frame.

[0087] Optional, see reference Figure 3 , Figure 3 This is the second schematic diagram of the pixels provided in the embodiments of this application. In this embodiment, the target image frame is... Figure 3 Image frame f in n The target position of the first pixel includes the target column number i and the target row number j.

[0088] Therefore, if the status information indicates a running state, the video processing device will determine the target column number and target row number based on the image frame f. n-1 Obtain the corresponding pixel and image frame f n+1 Obtain the corresponding pixel point from the image frame f. n-1 Pixels and image frames f n+1 The number of rows and columns of pixels in the target array is the same as the number of columns and rows in the target array. For example, the first pixel is f. n If (i,j), then obtain image frame f. n-1 The pixel f in the i-th row and j-th column n-1 (i,j), image frame f n+1 The pixel f in the i-th row and j-th column n+1 (i,j).

[0089] Furthermore, the video processing device processes the image frame f according to the target column number and target row number respectively. n-1 Obtain the corresponding pixel and image frame f n+1 Obtain the corresponding pixel point from the image frame f. n-1 Pixels and image frames f n+1 The number of columns for pixels in the target array is the same as the number of columns in the target array, and the number of rows in the target array is alternated. For example, the first pixel is f. n If (i,j), then obtain image frame f. n-1 The pixel f in the (i-2)th row and jth column n-1 (i-2,j), image frame f n-1 The pixel f in the (i+2)th row and jth columnn-1 (i+2,j), image frame f n+1 The pixel f in the (i-2)th row and jth column n+1 (i-2,j), image frame f n+1 The pixel f in the (i+2)th row and jth column n+1 (i+2,j).

[0090] Therefore, the final obtained second pixel is pixel f. n (i,j), pixel f n+1 (i,j), pixel f n-1 (i-2,j), pixel f n-1 (i+2,j), pixel f n+1 (i-2,j) and pixel f n+1 (i+2,j).

[0091] Furthermore, the video processing device processes the image frame f according to the target column number and target row number. n The third pixel of the nearest neighbor to the target location is obtained, where the number of columns of each pixel in the third pixel is the same as the number of columns in the target, and the number of rows is adjacent to the number of columns in the target. For example, the first pixel is f. n (i,j), meaning the target column and target row number is the i-th row and j-th column, therefore the obtained third pixel is, image frame f n The pixel f in the (i-1)th row and jth column n (i-1,j), image frame f n The pixel f in the (i+1)th row and jth column n (i+1,j).

[0092] Furthermore, the video processing device determines the resolution based on pixel f. n (i-1,j), pixel f n (i+1,j), pixel f n-1 (i,j), pixel f n+1 (i,j), pixel f n-1 (i-2,j), pixel f n-1 (i+2,j), pixel f n+1 (i-2,j) and pixel f n+1 The second target pixel value is obtained by weighting the pixel value at (i+2,j) and its corresponding weight. The specific formula for calculating the second target pixel value is as follows:

[0093]

[0094] Among them, P n (i,j) represents the second target pixel value, P n (i-1,j) represents pixel fn The pixel value of (i-1,j), P n (i+1,j) represents pixel f. n The pixel value of (i+1,j), P n-1 (i,j) represents pixel f n-1 The pixel value of (i,j), P n+1 (i,j) represents pixel f n+1 The pixel value of (i,j), P n-1 (i-2,j) represents pixel f n-1 The pixel value of (i-2,j), P n-1 (i-2,j) represents pixel f n-1 The pixel value of (i-2,j), P n+1 (i-2,j) represents pixel f n+1 The pixel value of (i-2,j), P n+1 (i+2,j) represents pixel f n+1 The pixel value of (i+2,j).

[0095] Furthermore, the video processing device performs recovery processing on the first pixel based on the second target pixel value to obtain the processed image frame of the target image frame.

[0096] In this embodiment of the application, when the state information is in motion, the missing pixels in the target image frame are restored according to the pixel values ​​of the pixels in the adjacent image frames of the target image frame, making the video playback process smoother and improving the deinterlacing display effect.

[0097] In an optional embodiment, after step 50, the method further includes:

[0098] Step 60: For the target pixel obtained by the recovery process in the processed image frame, determine whether the target pixel is an abnormal pixel based on the first reference pixel adjacent to the row of the target pixel in the processed image frame.

[0099] Step 70: If yes, then based on the position of the abnormal pixel in the processed image frame, determine the second reference pixel of the abnormal pixel in the adjacent image frame of the processed image frame, and obtain the difference information between the abnormal pixel and the second reference pixel.

[0100] Step 80: If the difference information meets the corresponding motion state misjudgment condition, then update the pixel value of the abnormal pixel in the processed image frame based on the pixel value corresponding to the second reference pixel.

[0101] Optionally, pixels that are in a non-static (moving) state but meet the conditions of a static state are processed. The corresponding phenomenon is the appearance of abrupt pixels. Therefore, it is necessary to perform false detection on the target pixels obtained from the recovery process in the processed image frame. The specific process is as follows:

[0102] Optionally, the video processing device acquires a first reference pixel in the processed image frame that is adjacent to the target pixel row, and determines whether the target pixel is an abnormal pixel based on the target pixel and the first reference pixel, as described in steps 601 to 603.

[0103] Furthermore, if the target pixel is determined to be an abnormal pixel, the video processing device determines a second reference pixel for the abnormal pixel in adjacent image frames based on the position of the abnormal pixel in the processed image frame. Specifically, the reference pixel is the image frame f with the target image frame as the reference pixel. n The location of the abnormal pixel includes column number i and row number j. Therefore, based on the column number and row number, the abnormal pixel is located in image frame f. n-1 and image frame f n+1 Obtain the second reference pixel, where the number of rows and columns of each pixel in the second reference pixel is the same as the number of rows and columns of the abnormal pixel. For example, the abnormal pixel is f. n (i,j), meaning the column and row number of the abnormal pixel is the i-th row and j-th column, therefore the obtained second reference pixel is, image frame f n-1 Get the pixel f in the i-th row and j-th column. n-1 (i,j), image frame f n+1 The pixel f in the i-th row and j-th column n+1 (i,j).

[0104] Furthermore, the video processing device acquires the set of pixels within a preset range for each pixel in the second reference pixel, that is, acquires pixel f. n-1 (i,j) is the set of pixels within a preset range, and pixel f n+1 (i,j) is a set of pixels within a preset range, where the preset range can be a 3*3 pixel range.

[0105] Furthermore, the video processing device determines the resolution based on pixel f. n-1 The pixel values ​​of the pixels in the pixel set (i,j) are used to calculate pixel f. n-1 The first pixel mean of the pixel set (i,j), and based on pixel f n+1 The pixel values ​​of the pixels in the pixel set (i,j) are used to calculate pixel f. n+1 The second pixel mean of the pixel set (i,j).

[0106] Furthermore, the video processing device obtains the difference information between the abnormal pixel and the second reference pixel based on the pixel value of the abnormal pixel and the first pixel mean and the second pixel mean. In this embodiment, the difference information can be understood as pixel difference, that is, the difference between the pixel value of the abnormal pixel and the first pixel mean is calculated to obtain the first pixel difference, and the difference between the pixel value of the abnormal pixel and the second pixel mean is calculated to obtain the second pixel difference.

[0107] Furthermore, if the difference information satisfies the corresponding motion state misjudgment condition, i.e., the first pixel difference is less than or equal to a preset difference, and the second pixel difference is less than or equal to a preset difference, the video processing device then determines the motion state misjudgment condition based on pixel f in the second reference pixel. n The pixel value P of (i-1,j) n (i-1,j) and pixel f n The pixel value P of (i+1,j) n (i+1,j) updates the pixel values ​​of abnormal pixels to obtain the updated pixel values. The preset difference is set according to actual conditions, specifically based on the pixel value P. n (i-1,j) and pixel value P n The mean of (i+1,j) is used to update the pixel values ​​of outlier pixels, resulting in the updated pixel values. Therefore, the formula for calculating the updated pixel values ​​is:

[0108]

[0109] Among them, P′ n (i, j) represents the updated pixel value.

[0110] Furthermore, if the difference information does not meet the corresponding motion state misjudgment condition, i.e., the first pixel difference is greater than the preset difference, or / and the second pixel difference is greater than the preset difference, the video processing device determines that the motion state of the abnormal pixel is a misjudgment and does not process the pixel value of the abnormal pixel.

[0111] This application embodiment reduces pixel identification errors and improves the accuracy of pixel recovery by performing false detection on the target pixels obtained from the recovery process in the processed image frame, thereby improving the deinterlacing display effect.

[0112] In an optional embodiment, steps 601 to 603 are described as follows:

[0113] Step 601: Based on the first pixel value of the target pixel and the second pixel values ​​of at least two first reference pixels, obtain at least two pixel differences;

[0114] Step 602: If at least two pixel differences satisfy the corresponding preset threshold, and the first pixel value exceeds the third target pixel value, then the target pixel is determined to be an abnormal pixel.

[0115] Step 603, otherwise, determine whether the target pixel is an abnormal pixel.

[0116] Optionally, the video processing device obtains at least two pixel differences based on the first pixel value of the target pixel and the second pixel values ​​of at least two first reference pixels, wherein the target pixel and the first reference pixels are row-adjacent pixels. For example, the target pixel is f n (i,j), the first reference pixel is pixel f. n (i-1,j) and pixel f n (i+1,j), therefore, at least two pixel differences, i.e., the target pixel point f, are required. n The pixel value P of (i,j) n (i,j) and pixel f n The pixel value P of (i-1,j) n The pixel difference between (i-1,j) and the target pixel f n The pixel value P of (i,j) n (i,j) and pixel f n The pixel value P of (i+1,j) n The pixel difference between (i+1,j).

[0117] Furthermore, the video processing device determines whether at least two pixel differences satisfy corresponding preset thresholds, and whether the first pixel value exceeds the third target pixel value. The preset thresholds are set based on actual conditions, and the third target pixel value is determined based on the maximum and minimum values ​​among the second pixel values. That is, it determines that at least two pixel differences are greater than the preset thresholds, and the target pixel value f... n The pixel value P of (i-1,j) n (i,j) is the maximum or minimum value, i.e., the pixel value P. n (i-1,j) and pixel value P n (i+1,j) are all greater than the pixel value P. n (i,j), or pixel value P n (i-1,j) and pixel value P n (i+1,j) are all less than the pixel value P. n (i,j).

[0118] Furthermore, if it is determined that at least two pixel differences are both greater than a preset difference, and the pixel value f n (i-1,j) and pixel value P n (i+1,j) are all greater than the pixel value P. n (i,j), or pixel value fn (i-1,j) and pixel value P n (i+1,j) are all less than the pixel value P. n If (i,j), then the target pixel is determined to be an abnormal pixel.

[0119] Furthermore, if it is determined that one of the pixel differences between at least two pixel differences is less than or equal to a preset difference, or / and the first pixel value is not the maximum or minimum value among the first pixel value and the second pixel value, then the target pixel is determined not to be an abnormal pixel.

[0120] This application embodiment performs post-processing on the target pixels recovered from the processed image frame to determine whether there are any misjudged detections of the target pixels, thereby reducing pixel recognition errors, improving the accuracy of pixel recovery, and thus improving the deinterlacing display effect.

[0121] The video processing apparatus provided in the embodiments of this application is described below. The video processing apparatus described below can be referred to in correspondence with the video processing method described above. (Refer to...) Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of the video processing apparatus provided in the embodiments of this application. The video processing apparatus may include:

[0122] The first acquisition module 401 is used to acquire the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame.

[0123] The determining module 402 is used to determine a reference image frame in the first image frame that is adjacent to the target image frame;

[0124] Detection module 403 is used to determine the state information corresponding to the pixels in the target image frame based on the target image frame and the reference image frame;

[0125] The processing module 404 is used to perform recovery processing on the pixels to be recovered in the target image frame based on the state information, so as to obtain the processed image frame of the target image frame.

[0126] The second acquisition module 405 is used to acquire the processed image frame obtained after the restoration process of each of the first image frames, and to obtain the processed video corresponding to the target video.

[0127] This application embodiment only requires adjacent image frames to achieve video deinterlacing, eliminating the need for block matching and thus reducing resource consumption costs. Simultaneously, by determining the state information of pixels in the target image frame using adjacent image frames, missing pixels in the target image frame can be restored based on this state information. This eliminates the need for interpolation algorithms, avoiding the problem of low accuracy in interpolated images, and also eliminates the need to set a matching criterion threshold for block matching, avoiding the impact of the matching criterion threshold on the display effect, thereby improving the deinterlacing display effect.

[0128] The specific embodiments of the video processing apparatus and the various embodiments of the video processing method provided in this application are basically the same, and will not be described in detail here.

[0129] Optional, such as Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call a computer program stored in the memory 530 to execute steps of a video processing method, such as:

[0130] Acquire the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame;

[0131] Identify a reference image frame in the first image frame that is adjacent to the target image frame;

[0132] Based on the target image frame and the reference image frame, determine the state information corresponding to the pixels in the target image frame;

[0133] Based on the state information, the pixels to be recovered in the target image frame are restored to obtain the processed image frame of the target image frame;

[0134] The processed image frame obtained after performing the restoration process on each of the first image frames is obtained to obtain the processed video corresponding to the target video.

[0135] Furthermore, the logical computer program in the aforementioned memory 530 can be implemented as a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several computer programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] On the other hand, embodiments of this application also provide a non-transitory computer-readable storage medium, which includes a computer program. The computer program can be stored on the non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the video processing methods provided in the above embodiments, such as including:

[0137] Acquire the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame;

[0138] Identify a reference image frame in the first image frame that is adjacent to the target image frame;

[0139] Based on the target image frame and the reference image frame, determine the state information corresponding to the pixels in the target image frame;

[0140] Based on the state information, the pixels to be recovered in the target image frame are restored to obtain the processed image frame of the target image frame;

[0141] The processed image frame obtained after performing the restoration process on each of the first image frames is obtained to obtain the processed video corresponding to the target video.

[0142] Furthermore, embodiments of this application also provide a computer product, which includes a computer program. The computer program can be stored on the computer product, and when executed by a processor, the computer can perform the steps of the video processing methods provided in the above embodiments, such as:

[0143] Acquire the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame;

[0144] Identify a reference image frame in the first image frame that is adjacent to the target image frame;

[0145] Based on the target image frame and the reference image frame, determine the state information corresponding to the pixels in the target image frame;

[0146] Based on the state information, the pixels to be recovered in the target image frame are restored to obtain the processed image frame of the target image frame;

[0147] The processed image frame obtained after performing the restoration process on each of the first image frames is obtained to obtain the processed video corresponding to the target video.

[0148] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several computer programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A video processing method, characterized in that, include: Acquire the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame; Identify a reference image frame in the first image frame that is adjacent to the target image frame; Based on the target image frame and the reference image frame, determine the state information corresponding to the pixels in the target image frame; Based on the state information, the pixels to be recovered in the target image frame are restored to obtain the processed image frame of the target image frame; The processed image frame obtained after performing the restoration process on each of the first image frames is obtained to obtain the processed video corresponding to the target video. For the target pixel obtained by recovery processing in the processed image frame, based on the first reference pixel adjacent to the row of the target pixel in the processed image frame, it is determined whether the target pixel is an abnormal pixel. If so, based on the position of the abnormal pixel in the processed image frame, a second reference pixel is determined in the adjacent image frame of the processed image frame, and the difference information between the abnormal pixel and the second reference pixel is obtained. If the difference information satisfies the corresponding motion state misjudgment condition, then the pixel value of the abnormal pixel in the processed image frame is updated based on the pixel value corresponding to the second reference pixel.

2. The video processing method according to claim 1, characterized in that, The step of determining the state information corresponding to the pixels in the target image frame based on the target image frame and the reference image frame includes: Based on the target image frame and the reference image frame, at least two frame difference data are obtained; If the at least two frame difference data satisfy the corresponding preset conditions, then the state information of the pixel in the target image frame is determined to be in a static state. If at least one of the at least two frame difference data does not meet the corresponding preset condition, then the state information of the pixel in the target image frame is determined to be in motion state.

3. The video processing method according to claim 1, characterized in that, The step of restoring the pixels to be restored in the target image frame based on the state information to obtain the processed image frame of the target image frame includes: Obtain the first pixel in the target image frame; Based on the target position of the first pixel and the state information, the second pixel in the reference image frame is determined; Based on the state information and the second pixel, the first pixel is restored to obtain the processed image frame of the target image frame.

4. The video processing method according to claim 3, characterized in that, The reference image frame includes the frame before and the frame after the target image frame. The state information of the pixels in the target image frame is in a static state. The second pixel includes the pixel in the reference image frame located at the target position. The step of restoring the first pixel based on the state information and the second pixel to obtain the processed image frame of the target image frame includes: Calculate the first target pixel value based on the pixel value of the second pixel; The first pixel is restored based on the first target pixel value to obtain the processed image frame of the target image frame.

5. The video processing method according to claim 3, characterized in that, The reference image frame includes the frame before and the frame after the target image frame. The state information of the pixels in the target image frame is the motion state. The second pixel includes the pixel in the reference image frame located at the target position, and the pixels at the adjacent positions of the target position. The step of restoring the first pixel based on the state information and the second pixel to obtain the processed image frame of the target image frame includes: The third pixel in the target image frame is obtained based on the target location, and the third pixel is a pixel adjacent to the first pixel. The second target pixel value is calculated based on the pixel value corresponding to the second pixel point, the pixel value corresponding to the third pixel point, and the corresponding weights. The first pixel is restored based on the second target pixel value to obtain the processed image frame of the target image frame.

6. The video processing method according to claim 1, characterized in that, Determining whether the target pixel is an abnormal pixel based on the first reference pixel adjacent to the row of the target pixel in the processed image frame includes: Based on the first pixel value of the target pixel and the second pixel values ​​of at least two first reference pixels, at least two pixel differences are obtained. If the difference between at least two pixels satisfies the corresponding preset threshold, and the first pixel value exceeds the third target pixel value, then the target pixel is determined to be an abnormal pixel; the third target pixel value is determined based on the maximum and minimum values ​​among the second pixel values. Otherwise, determine whether the target pixel is an abnormal pixel.

7. A video processing apparatus, characterized in that, include: The first acquisition module is used to acquire the first image frame obtained after performing interlaced sampling on the target video, and the target image frame currently being processed in the first image frame; The determining module is used to determine a reference image frame in the first image frame that is adjacent to the target image frame; The detection module is used to determine the state information corresponding to the pixels in the target image frame based on the target image frame and the reference image frame; The processing module is used to perform recovery processing on the pixels to be recovered in the target image frame based on the state information, so as to obtain the processed image frame of the target image frame; The second acquisition module is used to acquire the processed image frame obtained after each of the first image frames has undergone the restoration process, and to obtain the processed video corresponding to the target video. The video processing device is also used for: For the target pixel obtained by recovery processing in the processed image frame, based on the first reference pixel adjacent to the target pixel in the processed image frame, it is determined whether the target pixel is an abnormal pixel. If so, based on the position of the abnormal pixel in the processed image frame, a second reference pixel of the abnormal pixel is determined in the adjacent image frame of the processed image frame, and the difference information between the abnormal pixel and the second reference pixel is obtained. If the difference information satisfies the corresponding motion state misjudgment condition, the pixel value of the abnormal pixel in the processed image frame is updated based on the pixel value corresponding to the second reference pixel.

8. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing multiple computer programs; the processor loads the computer programs from the memory to perform the video processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer programs adapted for loading by a processor to perform the video processing method as described in any one of claims 1 to 6.

Images