Video processing methods, devices, storage media, electronic devices and chips

Abstract

This disclosure relates to a video processing method, apparatus, storage medium, electronic device, and chip. The method includes: firstly, acquiring first field information and second field information of an interlaced video frame; then, determining the difference region and non-difference region between the first field information and the second field information; then, based on the difference region and non-difference region, determining the missing field information corresponding to the retention of target field information, wherein the target field information is one of the first field information and the second field information; and then synthesizing a progressive video frame based on the missing field information and the target field information. By applying the technical solution of this disclosure, video can be effectively deinterlaced, and progressive video frames can be synthesized by combining the retained single-field information with the missing field information, resulting in a clearer image, improved image quality, reduced jagged edges and blurring, and enhanced user viewing experience.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and in particular to a video processing method, apparatus, storage medium, electronic device and chip. Background Technology

[0002] Deinterlacing is a video processing technique used to improve video quality, especially for traditional interlaced video. Each frame of interlaced video contains two fields (half-frames), and each field contains either the odd-numbered or even-numbered lines of that frame, hence the names odd field and even field, also known as the upper field and lower field, respectively. Summary of the Invention

[0003] This disclosure provides a video processing method, apparatus, storage medium, electronic device, and chip, with the main objective of improving the technical problem that current related technologies result in low image clarity and easy image blurring after video deinterlacing.

[0004] According to a first aspect of the present disclosure, a video processing method is provided, comprising:

[0005] Obtain the first and second field information of interlaced video frames;

[0006] Determine the regions of difference and non-difference between the first field information and the second field information;

[0007] Based on the difference region and the non-difference region, the missing field information corresponding to the retention of the target field information is determined, wherein the target field information is one of the first field information and the second field information;

[0008] The missing field information and the target field information are used to synthesize progressive video frames.

[0009] Optionally, determining the difference region and non-difference region between the first field information and the second field information includes:

[0010] Determine the confidence level of the difference between the first field information and the second field information;

[0011] Based on the difference confidence level, the difference region and the non-difference region are determined.

[0012] Optionally, determining the confidence level of the difference between the first field information and the second field information includes:

[0013] The first field information and the second field information are merged according to the RGB channels to obtain a multi-channel image;

[0014] A confidence map is generated from the multi-channel image using a convolutional neural network model. The resolution of the confidence map is the same as that of the first field information and the second field information. The confidence map is used to determine the probability that there is a content difference between the first field information and the second field information at each pixel location.

[0015] Optionally, determining the difference region and the non-difference region based on the difference confidence level includes:

[0016] Based on the confidence map, the difference region and the non-difference region are determined.

[0017] Optionally, determining the difference region and the non-difference region based on the confidence map includes:

[0018] The pixel positions in the confidence map with a probability of content difference greater than a preset threshold are merged to obtain the difference region;

[0019] The pixel positions in the confidence map with a probability of content difference less than or equal to the preset threshold are merged to obtain the non-difference region.

[0020] Optionally, based on the difference region and the non-difference region, determine the corresponding missing field information while retaining the target field information, including:

[0021] Obtain the missing field recovery results corresponding to the discrepancy regions; and,

[0022] Obtain the missing field recovery results corresponding to the non-differential regions;

[0023] The missing field recovery results corresponding to the difference regions are fused with the missing field recovery results corresponding to the non-difference regions to obtain the missing field information.

[0024] Optionally, obtaining the missing field recovery result corresponding to the difference region includes:

[0025] The field information of the first field information and the field information of the second field information at the corresponding positions in the difference region are merged according to the RGB channels to obtain the target multi-channel image;

[0026] Based on the target multi-channel image, missing field recovery processing is performed using a first semantic segmentation model to obtain the missing field recovery result corresponding to the difference region.

[0027] Optionally, obtaining the missing field recovery result corresponding to the non-differential region includes:

[0028] For each pixel location in the non-differential region, the average of the first field information and the second field information at the corresponding location is calculated as the pixel value of the missing field;

[0029] Based on the pixel values ​​of the missing field, a missing field recovery result corresponding to the non-differential region is generated.

[0030] Optionally, the step of synthesizing progressive video frames based on the missing field information and the target field information includes:

[0031] Compare the areas of the differing regions and the non-dissimilar regions;

[0032] If the area of ​​the non-differential region is greater than or equal to the area of ​​the differential region by a preset multiple, then the missing field information and the target field information are synthesized to obtain a progressive video frame.

[0033] Optionally, after comparing the areas of the differing regions and the non-disagreeing regions, the method further includes:

[0034] If the area of ​​the non-differential region is smaller than the area of ​​the differential region by the preset multiple, then the missing field information and the target field information are respectively processed by the second semantic segmentation model for image enhancement to obtain enhanced missing field information and enhanced target field information.

[0035] The enhanced missing field information and the enhanced target field information are combined to obtain progressive video frames.

[0036] According to a second aspect of the present disclosure, a video processing apparatus is provided, comprising:

[0037] The acquisition module is configured to acquire the first and second field information of interlaced video frames.

[0038] The first determining module is configured to determine the difference region and the non-difference region between the first field information and the second field information;

[0039] The second determining module is configured to determine, based on the difference region and the non-difference region, the missing field information corresponding to the retention of the target field information, wherein the target field information is one of the first field information and the second field information;

[0040] The synthesis module is configured to synthesize progressive video frames based on the missing field information and the target field information.

[0041] According to a third aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the video processing method described in the first aspect.

[0042] According to a fourth aspect of the present disclosure, an electronic device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the video processing method described in the first aspect.

[0043] According to a fifth aspect of the present disclosure, a chip is provided, including one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from a memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the video processing method described in the first aspect.

[0044] By employing the above technical solutions, this disclosure provides a video processing method, apparatus, storage medium, electronic device, and chip. Compared with existing related technologies, this disclosure uses a technical solution different from the field duplication method. Specifically, it first obtains the first field information and second field information of interlaced video frames; then determines the difference region and non-difference region between the first field information and the second field information; then, based on the difference region and non-difference region, it determines the missing field information corresponding to the retention of the target field information, wherein the target field information is one of the first field information and the second field information; and then synthesizes the missing field information and the target field information to obtain progressive video frames. By applying the technical solution of this disclosure, video can be effectively deinterlaced, and progressive video frames can be synthesized by synthesizing the retained single-field information and the missing field information. The missing field information is determined based on the difference region and non-difference region between the two fields, which can accurately represent the content difference between the two fields, thereby synthesizing a clearer picture, reducing the occurrence of image blurring, and improving the user's viewing experience.

[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0047] Figure 1 A flowchart illustrating a video processing method provided in an embodiment of this disclosure is shown.

[0048] Figure 2 A flowchart illustrating a video processing method provided in an embodiment of this disclosure is shown.

[0049] Figure 3A flowchart illustrating an example provided by an embodiment of this disclosure is shown;

[0050] Figure 4 A schematic diagram of the structure of a video processing apparatus provided in an embodiment of this disclosure is shown. Detailed Implementation

[0051] Some embodiments of this disclosure will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted. It should be noted that, without conflict, the embodiments and features in the embodiments of this disclosure can be combined with each other.

[0052] Currently, related technologies can perform video deinterlacing by copying fields. Specifically, a single field in an interlaced video frame is retained, while the other field is discarded. Then, the lines in the retained single field are copied and combined to form a new field. Finally, the new field is combined with the retained single field to form a new frame. However, this method uses only half the number of lines of image information, which leads to a significant decrease in the effective image sharpness, resulting in a blurry image.

[0053] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0054] Figure 1 This is a flowchart illustrating a video processing method according to some embodiments of the present disclosure, such as... Figure 1 As shown, this method can be applied to video processing devices or equipment, and can be configured on the terminal side, etc. The method includes the following steps.

[0055] Step 101: Obtain the first field information and the second field information of the interlaced video frame.

[0056] In some embodiments, each interlaced video frame can be divided into two half-frames, that is, into two fields. Each field contains half the number of lines of the video frame. For example, one field contains the odd-numbered lines of the frame and the other field contains the even-numbered lines of the frame. These two fields can be called the odd field and the even field, or the upper field and the lower field. In this embodiment, the information of these two fields in the interlaced video frame can be obtained, namely the first field information (or the first field image, etc.) and the second field information (or the second field image, etc.).

[0057] In one embodiment of this disclosure, each image frame may consist of multiple image blocks, each image block consisting of two or more rows of pixels, and the number of pixel rows between each pixel block may be equal. The interlaced video frame of this disclosure can be divided into two half-frames, i.e., two fields, with each field containing half the number of rows of the video frame. For example, each pixel block contains two rows of pixels, one field containing an odd number of image blocks, and the other field containing an even number of pixel blocks. That is, the smallest unit of each field in this disclosure can consist of multiple rows of pixels, and is not limited to a single row of pixels.

[0058] Step 102: Determine the difference and non-difference regions between the first and second information.

[0059] The difference region between the first and second fields of information can refer to areas in two consecutive fields (odd and even fields) of a video where the image content has changed due to object movement or other reasons. These changes can be variations in brightness, color, texture, etc. For example, a difference region is a place where the image content is significantly different between one field and the other. If an object is moving in the video, its position will differ in the two fields, forming a difference region. Additionally, changes in the light source during filming, such as switching lights on or off or clouds obscuring the sun, can also cause brightness differences between the two fields.

[0060] The indistinguishable region between the first and second fields refers to the area where there is no significant change between the two fields. The image content in these areas remains essentially consistent across both fields, without noticeable motion or change. For example, if there is a stationary background wall in a video, this area should be identical between the two fields and is considered an indistinguishable region.

[0061] Step 103: Based on the difference region and the non-difference region, determine the missing field information corresponding to the retention of the target field information.

[0062] The target field information is either the first field information or the second field information; that is, either the first field information or the second field information is retained. For example, odd-numbered field information is retained while even-numbered field information is discarded, or vice versa. When one field information is retained, the other field information will be missing. This embodiment determines the corresponding missing field information, that is, how to recover or compensate for the missing field information. Specifically, this can be determined based on difference regions and non-difference regions. For example, for difference regions, since the image content has changed, retaining only one field will lead to significant visual distortion. Therefore, interpolation methods or motion compensation methods from data learning (such as determining the motion vector in the odd field and predicting the position and value of the corresponding pixel in the even field based on the motion direction and speed) can be used to recover or compensate for the missing field information, thereby obtaining the corresponding missing field information.

[0063] Step 104: Synthesize the missing field information and the target field information to obtain progressive video frames.

[0064] Progressive video frames are a video display technology that, in contrast to interlaced scanning, contains image information for all lines in each frame. That is, all lines of each frame are captured and displayed completely at the same time, rather than being displayed separately in two fields. This technology can provide higher image quality and less motion artifacts.

[0065] For example, in order to deinterlaced video frames to obtain progressive video frames, while retaining one field information of the interlaced video frame, the retained field information can be synthesized with the corresponding determined missing field information to obtain progressive video frames.

[0066] Compared with existing related technologies, the embodiments of this disclosure can effectively perform deinterlacing processing on video, and synthesize the retained single field information and missing field information to obtain progressive video frames. The missing field information is determined based on the difference and non-difference regions between the two fields, which can accurately represent the content differences between the two fields. In this way, a clearer picture can be synthesized, improving the picture quality, reducing jagged edges and blur, and improving the user's viewing experience.

[0067] To further illustrate, as Figure 1 The specific implementation process of the method shown in this embodiment is provided as follows: Figure 2 The specific method shown includes:

[0068] Step 201: Obtain the first field information and the second field information of the interlaced video frame.

[0069] For example, obtaining odd-field and even-field information of interlaced video frames, or what may be called upper-field and lower-field information, etc.

[0070] Step 202: Determine the confidence level of the difference between the first and second information.

[0071] The confidence level of the difference between two fields can be used to determine the probability that there is a content difference between the two fields at each pixel location.

[0072] In some embodiments, a convolutional neural network model can be pre-trained to compute a confidence map corresponding to two fields. The resolution of this confidence map is the same as the resolution of the two fields. This confidence map is used to determine the probability that there is a content difference between the two fields at each pixel location. For example, a training set can be pre-constructed, such as collecting sample interlaced video frames and determining the information of the two fields in each sample interlaced video frame. A label image is generated for each interlaced video frame, indicating which regions differ between the two fields. Each pixel value in the label image is either 0 or 1, where 1 indicates a difference and 0 indicates no difference. In the model's network structure, the input layer accepts a six-channel image, such as the first three channels being the upper field image and the last three channels being the lower field image. The intermediate layers use multiple convolutional layers, pooling layers, and activation functions to extract features. The output layer uses a single convolutional layer to output a single-channel confidence map, with an output range of 0 to 1. A sigmoid activation function can be used to ensure that the output value is between 0 and 1, representing the probability that there is a content difference between the two fields at the corresponding pixel location. The model is trained based on the constructed training set and model network structure, thereby obtaining a convolutional neural network model.

[0073] Accordingly, step 202 may specifically include: first, merging the first field information and the second field information according to the RGB channels to obtain a multi-channel image; then, generating a confidence map from the multi-channel image using a convolutional neural network model, and then determining the difference confidence between the first field information and the second field information using the confidence map, wherein the resolution of the confidence map is the same as the image resolution of the first field information and the second field information, and the confidence map is used to determine the probability that there is a content difference between the first field information and the second field information at each pixel position.

[0074] For example, two field images are merged according to their RGB channels to obtain a six-channel image, which is then input into the convolutional neural network model that has been trained above for calculation. The output is a single-channel confidence map after Sigmoid activation. The resolution of this confidence map is the same as that of the two fields. This confidence map is used to determine the probability that there is a content difference between the two fields at each pixel position.

[0075] Step 203: Based on the confidence level of the difference between the first and second information, determine the difference region and the non-difference region between the first and second information.

[0076] This embodiment can accurately determine the difference region and the non-difference region between the two fields based on the probability that there is a content difference between the two fields at each pixel position.

[0077] Based on the example in step 202, step 203 may specifically include: determining the difference regions and non-difference regions between the first field information and the second field information based on the confidence map output by the convolutional neural network model. This confidence map can be used to determine the probability that the two fields have content differences at each pixel location, and thus the difference regions and non-difference regions between the two fields can be accurately determined based on this confidence map.

[0078] In some examples, based on the confidence map, the difference region and the non-difference region between the two fields can be determined. Specifically, this may include merging the pixel positions in the confidence map where the probability of content difference is greater than a preset threshold to obtain the difference region; and merging the pixel positions in the confidence map where the probability of content difference is less than or equal to a preset threshold to obtain the non-difference region.

[0079] The preset threshold can be set according to actual needs. This preset threshold can be used to determine the pixel locations in the confidence map where there are content differences. For example, if the preset threshold is 0.5, pixel locations in the confidence map with a probability of content difference greater than 0.5 will be merged to obtain the difference region between the two field information; pixel locations in the confidence map with a probability of content difference less than or equal to 0.5 will be merged to obtain the non-difference region between the two field information.

[0080] Step 204: Obtain the missing field recovery results corresponding to the differential regions, and obtain the missing field recovery results corresponding to the non-differential regions.

[0081] In some embodiments, obtaining the missing field recovery result corresponding to the non-discretionary region may specifically include: for each pixel position in the non-discretionary region, calculating the average of the first field information and the second field information at the corresponding position as the pixel value of the missing field; and then generating the missing field recovery result corresponding to the non-discretionary region based on the pixel value of the missing field.

[0082] For example, in indistinguishable regions, since the image content changes almost nothing between the two fields, the information loss is minimal even if only one field is retained. Therefore, for each pixel location in an indistinguishable region, the mean of the two fields at the corresponding location is calculated as the pixel value of the missing field. Then, based on the pixel value of the missing field, the missing field restoration result corresponding to the indistinguishable region can be generated. Since the content changes little in these regions, a good transition can be achieved by using the mean method.

[0083] In regions of difference, where image content has changed, retaining only one field leads to significant visual distortion. These regions require special processing to recover or compensate for missing information. In some embodiments, a first semantic segmentation model can be pre-trained. This model calculates the missing field recovery result for the corresponding position of each of the two fields in the region of difference based on the field information, thereby obtaining the field information recovered or compensated for in that region. For example, this first semantic segmentation model can be trained using deep learning methods based on different sample interlaced video frames.

[0084] Accordingly, obtaining the missing field recovery results corresponding to the difference region may specifically include: merging the field information of the first field information and the second field information at the corresponding positions in the difference region according to the RGB channels to obtain the target multi-channel image; and performing missing field recovery processing on the target multi-channel image through the first semantic segmentation model to obtain the missing field recovery results corresponding to the difference region.

[0085] For example, the field information of the two fields at corresponding positions in the difference region is stitched together into a six-channel image according to the RGB channels. Then, the six-channel image is input into the first semantic segmentation model for calculation, and the missing field recovery result corresponding to the difference region is output.

[0086] This disclosure employs a neural network adaptive learning processing scheme for video deinterlacing, offering greater flexibility and accuracy, and removing image noise simultaneously. By applying the technical solution of this disclosure, video content is determined, and different processing methods are used for different regions, thereby improving processing efficiency and reducing processing time and computational overhead.

[0087] Step 205: Fuse the missing field recovery results corresponding to the difference regions with the missing field recovery results corresponding to the non-difference regions to obtain the missing field information while retaining the target field information.

[0088] The target field information can be either the first field information or the second field information.

[0089] For example, the missing field recovery results corresponding to the difference regions can be fused with the missing field recovery results corresponding to the non-difference regions, or weighted fusion can be used to obtain the corresponding missing field information.

[0090] Step 206: Compare the areas of the different regions and the non-different regions.

[0091] For example, the area of ​​the differing region is added to the area of ​​the non-disagreeing region to get the total area. The percentage A of the area of ​​the differing region to the total area is obtained, and the percentage B of the area of ​​the non-disagreeing region to the total area is obtained. Then, percentage A and percentage B are compared.

[0092] Step 207a: If the area of ​​the non-difference region is greater than or equal to the area of ​​the difference region by a preset multiple, then the missing field information and the target field information are synthesized to obtain a progressive video frame.

[0093] For example, if, after area comparison, it is determined that the area of ​​the non-difference region is greater than four times the area of ​​the difference region, it means that the area of ​​the non-difference region accounts for more than 80% of the total area. It is considered that the difference between the missing field information and the current two field information is not significant, and the missing field information can be directly spliced ​​with the retained single field information to obtain line-by-line video frames.

[0094] In step 207b, which is parallel to step 207a, if the area of ​​the non-difference region is less than the area of ​​the difference region by a preset multiple, the missing field information and the target field information are respectively processed by the second semantic segmentation model to obtain the enhanced missing field information and the enhanced target field information, and the enhanced missing field information and the enhanced target field information are synthesized to obtain a progressive video frame.

[0095] In some embodiments, the second semantic segmentation model can be used to enhance field information in an image. The enhanced field information can result in better stitching and a more natural transition. For example, if, after area comparison, the area of ​​the non-dissimilar region is determined to be less than four times the area of ​​the dissimilar region, it indicates that the area of ​​the non-dissimilar region accounts for less than 80% of the total area. This suggests that the missing field information and the current two field information are significantly different. In this case, the second semantic segmentation model is used for image enhancement. The missing field information and the retained single-field information are input into the second semantic segmentation model for calculation, and the enhanced missing field information and enhanced single-field information are output. Then, the enhanced missing field information and enhanced single-field information are stitched together to obtain the final image.

[0096] In this embodiment of the disclosure, both the first semantic segmentation model and the second semantic segmentation model may include the UNet model.

[0097] Based on the above embodiments, in some examples, the application scenarios of this disclosure can be video encoding and video display. In the video encoding scenario, in order to reduce transmission bandwidth and encoding overhead, progressive video sources can be sampled according to interlaced encoding, reducing transmission bandwidth by half; during the decoding process, the missing video fields are recovered using the method of this disclosure, thereby improving encoding efficiency. In the video display scenario, the method of this disclosure is used to enhance the received interlaced video information, improving the user's viewing experience.

[0098] To illustrate the implementation process of the embodiments of this disclosure, the following application examples are given, such as... Figure 3 As shown, it includes:

[0099] Step 01: Input an interlaced video frame It ;

[0100] Step 02: Identify interlaced video frames I t Divided into two parts, according to the definition of interlaced video, the odd-numbered lines are the first half (F). uD Even-numbered plays in the second half, F down ;

[0101] Step 03: Calculate the difference confidence score between the upper and lower fields using a convolutional neural network model. This step uses a convolutional neural network with a six-channel image as input (concat(F)). up F down `concat(·)` represents a channel merging operation; the output is a single-channel confidence map after Sigmoid activation. t Its resolution is similar to F. up and F down The values ​​are the same, ranging from 0 to 1, representing the F value at that pixel position. up and F down The probability of content discrepancies;

[0102] Step 04: Use a pre-set threshold T to evaluate the confidence plot Sigmoid (Conf) t The system performs a judgment; if the difference probability is greater than T, then there is a difference between the two fields at that pixel location. The difference region between the two fields is then extracted and processed using a mask. same Identify and extract the non-differentiated regions;

[0103] Step 05: For non-differential regions (i.e., Mask) same ), using F up and F down The mean value is used as the pixel value of the missing field to obtain the restoration result of the missing field.

[0104] Step 06: For the difference region (i.e., l-Mask) same The missing field is recovered using the first semantic segmentation model, whose input is a six-channel image concat(F). up F down The output is the recovery result of the missing field.

[0105] Step 07, and The missing field image F is obtained by fusion. missing ;

[0106] Step 08: Apply the Mask from Step 04 sameIf the area of ​​this region accounts for more than 80% of the total area, it is considered that the missing field information and the current upper and lower field information are not significantly different, so F is directly output. missing With F down The stitched video frames are processed sequentially; otherwise, if the missing field information differs significantly from the current two field information, a second semantic segmentation model is used for image enhancement. The input to the second semantic segmentation model is F. missing and F down The output is the enhanced result. and The two are stitched together to form a progressive video frame.

[0107] The embodiments disclosed herein can employ different processing methods for different video content and different regions, thereby improving processing efficiency, reducing processing time and computational overhead, enhancing image quality, and suppressing noise in the video.

[0108] Figure 4 This is a block diagram of a video processing apparatus according to some embodiments of the present disclosure. (Refer to...) Figure 4 The device includes: an acquisition module 31, a first determination module 32, a second determination module 33, and a synthesis module 34.

[0109] The acquisition module 3l is configured to acquire the first field information and the second field information of interlaced video frames;

[0110] The first determining module 32 is configured to determine the difference region and the non-difference region between the first field information and the second field information;

[0111] The second determining module 33 is configured to determine, based on the difference region and the non-difference region, the missing field information corresponding to the retention of the target field information, wherein the target field information is one of the first field information and the second field information;

[0112] The synthesis module 34 is configured to synthesize progressive video frames based on the missing field information and the target field information.

[0113] In some embodiments, the first determining module 32 is specifically configured to determine the difference confidence level between the first field information and the second field information; and then determine the difference region and the non-difference region based on the difference confidence level.

[0114] In some embodiments, the first determining module 32 is further configured to merge the first field information and the second field information according to the RGB channels to obtain a multi-channel image; and generate a confidence map from the multi-channel image using a convolutional neural network model. The resolution of the confidence map is the same as the image resolution of the first field information and the second field information. The confidence map is used to determine the probability that there is a content difference between the first field information and the second field information at each pixel position.

[0115] In some embodiments, the first determining module 32 is further configured to determine the difference region and the non-difference region based on the confidence map.

[0116] In some embodiments, the first determining module 32 is further configured to merge pixel positions in the confidence map where the probability of content difference is greater than a preset threshold to obtain the difference region; and to merge pixel positions in the confidence map where the probability of content difference is less than or equal to the preset threshold to obtain the non-difference region.

[0117] In some embodiments, the second determining module 33 is specifically configured to obtain the missing field recovery result corresponding to the difference region; and obtain the missing field recovery result corresponding to the non-difference region; and fuse the missing field recovery result corresponding to the difference region with the missing field recovery result corresponding to the non-difference region to obtain the missing field information.

[0118] In some embodiments, the second determining module 33 is further configured to merge the field information of the first field information and the second field information at corresponding positions in the difference region according to the RGB channels to obtain a target multi-channel image; and perform missing field recovery processing on the target multi-channel image through a first semantic segmentation model to obtain the missing field recovery result corresponding to the difference region.

[0119] In some embodiments, the second determining module 33 is further configured to calculate the mean of the first field information and the second field information at corresponding positions for each pixel position in the non-discrepancy region, as the pixel value of the missing field; and generate the missing field recovery result corresponding to the non-discrepancy region based on the pixel value of the missing field.

[0120] In some embodiments, the synthesis module 34 is specifically configured to compare the areas of the difference region and the non-difference region; if the area of ​​the non-difference region is greater than or equal to the area of ​​the difference region by a preset multiple, the missing field information and the target field information are synthesized to obtain a progressive video frame.

[0121] In some embodiments, the synthesis module 34 is further configured to, if the area of ​​the non-differential region is less than the area of ​​the differential region by a preset multiple, perform image enhancement processing on the missing field information and the target field information respectively through a second semantic segmentation model to obtain enhanced missing field information and enhanced target field information; and synthesize the enhanced missing field information and the enhanced target field information to obtain progressive video frames.

[0122] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0123] It should be noted that other corresponding descriptions of the functional units involved in the video processing apparatus provided in this embodiment can be found in [reference needed]. Figures 1 to 2 The corresponding descriptions in [the document] will not be repeated here.

[0124] Based on the above, Figures 1 to 2 Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figures 1 to 2 The method shown.

[0125] Based on this understanding, the technical solution of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods of various implementation scenarios of this disclosure.

[0126] Based on the above, Figures 1 to 2 The method shown, and Figure 4 To achieve the above objectives, this disclosure also provides an electronic device, such as a high-definition television, computer, projector, or other terminal with a liquid crystal display, plasma display, or digital light processing display. This device includes a storage medium and a processor; the storage medium stores a computer program; and the processor executes the computer program to implement the above-described virtual device embodiments. Figures 1 to 2 The method shown.

[0127] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0128] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0129] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0130] Based on the above, Figures 1 to 2 The method shown, and Figure 4 The virtual device embodiment shown also provides a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the above-described... Figures 1 to 2 The method shown.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that this disclosure can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented using hardware. By applying the solutions of the embodiments of this disclosure, different processing methods can be adopted for different video content and different regions, thereby improving processing efficiency, reducing processing time and computational overhead, improving image quality, and suppressing noise in the video.

[0132] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0133] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method of video processing, the method comprising: The method comprises the following steps: obtaining first field information and second field information of an interlaced video frame; determining a difference region and a non-difference region between the first field information and the second field information; determining corresponding missing field information under the condition of retaining target field information, which is one of the first field information and the second field information, based on the difference region and the non-difference region; synthesizing the missing field information and the target field information to obtain a progressive video frame, comprising: if the area of the non-difference region is less than a preset multiple of the area of the difference region, performing image enhancement processing on the missing field information and the target field information through a second semantic segmentation model respectively to obtain enhanced missing field information and enhanced target field information; synthesizing the enhanced missing field information and the enhanced target field information to obtain a progressive video frame.

2. The method of claim 1, wherein, The method comprises the following steps: determining a difference region and a non-difference region between the first field information and the second field information; determining a difference confidence between the first field information and the second field information; 3. The method of claim 2, wherein, determining the difference region and the non-difference region according to the difference confidence. The method comprises the following steps: merging the first field information and the second field information according to RGB channels to obtain a six-channel image; 4. The method of claim 3, wherein, generating a confidence map from the six-channel image through a convolutional neural network model, the resolution of the confidence map being the same as the image resolution of the first field information and the second field information, and the confidence map being used to determine the probability of content difference between the first field information and the second field information at each pixel position. The method comprises the following steps:

5. The method of claim 4, wherein, determining the difference region and the non-difference region according to the confidence map. The method comprises the following steps: merging pixel positions in the confidence map where the probability of content difference is greater than a preset threshold to obtain the difference region; 6. The method of claim 1, wherein, merging pixel positions in the confidence map where the probability of content difference is less than or equal to the preset threshold to obtain the non-difference region. The method comprises the following steps: obtaining a missing field recovery result corresponding to the difference region; and obtaining a missing field recovery result corresponding to the non-difference region; 7. The method of claim 6, wherein, fusing the missing field recovery result corresponding to the difference region and the missing field recovery result corresponding to the non-difference region to obtain the missing field information. The method comprises the following steps: merging field information of the first field information and the second field information at corresponding positions in the difference region according to RGB channels to obtain a target multi-channel image; performing missing field recovery processing on the target multi-channel image through a first semantic segmentation model to obtain the missing field recovery result corresponding to the difference region.

8. The method of claim 6, wherein, The obtaining of the missing field recovery result corresponding to the non-difference region comprises: For each pixel position in the non-difference region, the mean value of the first field information and the second field information at the corresponding position is calculated as the pixel value of the missing field; Based on the pixel value of the missing field, the missing field recovery result corresponding to the non-difference region is generated.

9. The method of claim 1, wherein, The synthesizing of the missing field information and the target field information to obtain the progressive video frame comprises: The area comparison of the difference region and the non-difference region is performed; If the area of the non-difference region is greater than or equal to a preset multiple of the area of the difference region, the missing field information and the target field information are synthesized to obtain the progressive video frame.

10. A video processing apparatus, comprising: The method comprises: The obtaining module is configured to obtain first field information and second field information of the interlaced video frame; The first determining module is configured to determine a difference region and a non-difference region between the first field information and the second field information; The second determining module is configured to determine corresponding missing field information in the case of retaining target field information based on the difference region and the non-difference region, the target field information being one of the first field information and the second field information; The synthesizing module is configured to synthesize the missing field information and the target field information to obtain the progressive video frame, comprising: if the area of the non-difference region is less than a preset multiple of the area of the difference region, the missing field information and the target field information are respectively subjected to image enhancement processing through a second semantic segmentation model to obtain enhanced missing field information and enhanced target field information; The enhanced missing field information and the enhanced target field information are synthesized to obtain the progressive video frame.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 9.

12. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 9.

13. A chip, characterized by The electronic device comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, the signal comprising computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to execute the method of any one of claims 1 to 9. The electronic device comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, the signal comprising computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to execute the method of any one of claims 1 to 9.

Images