Video processing method and device, storage medium, electronic equipment and chip

Abstract

The invention relates to a video processing method and device, a storage medium, electronic equipment and a chip. The method comprises the following steps: firstly, acquiring first field information and second field information of an interlaced video frame; determining a difference area and a non-difference area between the first field information and the second field information; based on the difference area and the non-difference area, corresponding missing field information under the condition that target field information is reserved is determined, and the target field information is one of the first field information and the second field information; and performing synthesis according to the missing field information and the target field information to obtain a line-by-line video frame. Through application of the technical scheme of the invention, de-interlacing processing can be effectively carried out on the video, the reserved single-field information and the missing-field information are synthesized to obtain the progressive video frame, a picture with higher definition can be obtained through synthesis, the picture quality is improved, sawteeth and blurring are reduced, and the watching experience of a user can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video processing method, device, storage medium, electronic device and chip. Background Art

[0002] Video deinterlacing is a video processing technology used to improve video quality, especially for traditional interlaced video. Each frame of interlaced video contains two fields (i.e., half frames), and each field contains the odd or even lines of the frame, so they are called odd fields or even fields, also called upper fields or lower fields. Summary of the invention

[0003] The present disclosure provides a video processing method, device, storage medium, electronic device and chip, the main purpose of which is to improve the technical problem that the current related technology will cause the picture clarity to be low after video de-interlacing processing, and the picture is prone to blur.

[0004] According to a first aspect of an embodiment of the present disclosure, a video processing method is provided, including:

[0005] Get the first field information and the second field information of the interlaced video frame;

[0006] Determine a difference area and a non-difference area between the first field information and the second field information;

[0007] Determine, based on the difference area and the non-difference area, corresponding missing field information while retaining target field information, where the target field information is one of the first field information and the second field information;

[0008] The missing field information and the target field information are synthesized to obtain a progressive video frame.

[0009] Optionally, determining a difference area and a non-difference area between the first field information and the second field information includes:

[0010] determining a confidence level of a difference between the first field information and the second field information;

[0011] The difference area and the non-difference area are determined according to the difference confidence.

[0012] Optionally, determining a confidence level of a difference between the first field information and the second field information includes:

[0013] Merging the first field information and the second field information according to RGB channels to obtain a multi-channel image;

[0014] The multi-channel image is generated through a convolutional neural network model to obtain a confidence map, the resolution of the confidence map is the same as the image resolution of the first field information and the second field information, and the confidence map is used to determine the probability that there is a content difference between the first field information and the second field information at each pixel position.

[0015] Optionally, the determining the difference area and the non-difference area according to the difference confidence includes:

[0016] The difference area and the non-difference area are determined according to the confidence map.

[0017] Optionally, determining the difference area and the non-difference area according to the confidence map includes:

[0018] Merging pixel positions in the confidence map whose probability of having content differences is greater than a preset threshold to obtain the difference area;

[0019] The pixel positions in the confidence map whose probability of having content difference is less than or equal to the preset threshold are merged to obtain the non-difference area.

[0020] Optionally, determining, based on the difference area and the non-difference area, corresponding missing field information while retaining target field information includes:

[0021] Obtaining a missing field recovery result corresponding to the difference area; and,

[0022] Obtaining a missing field recovery result corresponding to the non-difference area;

[0023] The missing field recovery result corresponding to the difference area is merged with the missing field recovery result corresponding to the non-difference area to obtain the missing field information.

[0024] Optionally, the obtaining the missing field recovery result corresponding to the difference area includes:

[0025] Merging the first field information and the second field information at corresponding positions in the difference area according to RGB channels to obtain a target multi-channel image;

[0026] Based on the target multi-channel image, a missing field recovery process is performed through a first semantic segmentation model to obtain a missing field recovery result corresponding to the difference area.

[0027] Optionally, the obtaining the missing field recovery result corresponding to the non-difference area includes:

[0028] For each pixel position in the non-difference area, calculating the average value of the first field information and the second field information at the corresponding position as the pixel value of the missing field;

[0029] Based on the pixel value of the missing field, a missing field restoration result corresponding to the non-difference area is generated.

[0030] Optionally, synthesizing the missing field information and the target field information to obtain a progressive video frame includes:

[0031] Comparing the areas of the difference region and the non-difference region;

[0032] If the area of ​​the non-difference region is greater than or equal to a preset multiple of the area of ​​the difference region, the missing field information is synthesized with the target field information to obtain a progressive video frame.

[0033] Optionally, after comparing the areas of the difference region and the non-difference region, the method further includes:

[0034] If the area of ​​the non-difference region is smaller than the area of ​​the difference region of the preset multiple, the missing field information and the target field information are respectively subjected to image enhancement processing by a second semantic segmentation model to obtain enhanced missing field information and enhanced target field information;

[0035] The enhanced missing field information and the enhanced target field information are synthesized to obtain a progressive video frame.

[0036] According to a second aspect of an embodiment of the present disclosure, there is provided a video processing device, including:

[0037] An acquisition module is configured to acquire first field information and second field information of an interlaced video frame;

[0038] A first determining module, configured to determine a difference area and a non-difference area between the first field information and the second field information;

[0039] A second determining module is configured to determine, based on the difference area and the non-difference area, corresponding missing field information while retaining target field information, wherein the target field information is one of the first field information and the second field information;

[0040] The synthesis module is configured to synthesize the missing field information and the target field information to obtain a progressive video frame.

[0041] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the video processing method described in the first aspect is implemented.

[0042] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the video processing method described in the first aspect when executing the computer program.

[0043] According to a fifth aspect of an embodiment of the present disclosure, a chip is provided, comprising one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, wherein the signal includes a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the video processing method described in the first aspect.

[0044] By means of the above technical scheme, the present disclosure provides a video processing method, device, storage medium, electronic device and chip. Compared with the current related technologies, the present disclosure uses a technical scheme different from the copy field method, specifically first obtaining the first field information and the second field information of the interlaced video frame; then determining the difference area and the non-difference area between the first field information and the second field information; then based on the difference area and the non-difference area, determining the corresponding missing field information while retaining the target field information, wherein the target field information is one of the first field information and the second field information; then synthesizing the missing field information and the target field information to obtain a progressive video frame. By applying the technical scheme of the present disclosure, the video can be effectively deinterlaced, and the retained single field information and the missing field information are synthesized to obtain a progressive video frame. The missing field information is determined based on the difference area and the non-difference area between the two fields, which can accurately represent the content difference between the two fields, and then can be synthesized to obtain a higher definition picture, reduce the occurrence of blurred pictures, and improve the user's viewing experience.

[0045] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0047] Figure 1 A schematic diagram of a process of a video processing method provided by an embodiment of the present disclosure is shown;

[0048] Figure 2 A schematic diagram of a process of a video processing method provided by an embodiment of the present disclosure is shown;

[0049] Figure 3A schematic diagram of an example process provided by an embodiment of the present disclosure is shown;

[0050] Figure 4 A schematic structural diagram of a video processing device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0051] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, in order to enhance clarity and conciseness, descriptions of features known in the art may be omitted. It should be noted that, in the absence of conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.

[0052] At present, in the related technology, video deinterlacing can be performed by duplicating fields. Specifically, a single field in an interlaced video frame can be retained, another field can be discarded, and the lines in the retained single field can be copied and combined into a new field, and finally the new field and the retained single field can be combined into a new frame. However, this method uses only half of the line information, so the effective clarity of the picture is greatly reduced, resulting in blurred pictures.

[0053] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0054] Figure 1 is a flowchart of a video processing method according to some embodiments of the present disclosure. Figure 1 As shown, the method can be applied to a video processing device or equipment, etc., and can be configured on the terminal side, etc. The method includes the following steps.

[0055] Step 101: Obtain first field information and second field information of an interlaced video frame.

[0056] In some embodiments, each interlaced video frame may be divided into two half frames, i.e., two fields, each field containing half the number of lines of the video frame. For example, one field contains display information of odd lines of the frame, and the other field contains display information of even lines of the frame. These two fields may be referred to as odd fields and even fields, or may be referred to as upper fields and lower fields. This embodiment may obtain these two field information in the interlaced video frame, i.e., first field information (or may be referred to as first field image, etc.) and second field information (or may be referred to as second field image, etc.).

[0057] In one embodiment of the present disclosure, each image frame may be composed of a plurality of image blocks, each image block is composed of two or more rows of pixels, and the number of pixel rows between each pixel block may be equal. The interlaced video frame of the present disclosure may be divided into two half-frames based on the image block, that is, divided into two fields, each field containing half the number of rows of the video frame. Exemplarily, each pixel block contains two rows of pixels, one field contains odd-numbered image blocks, and the other field contains even-numbered pixel blocks. That is to say, the minimum unit of each field in the present disclosure may be composed of multiple rows of pixels, and is not limited to a single row of pixels.

[0058] Step 102: Determine a difference area and a non-difference area between the first field information and the second field information.

[0059] The difference area between the first field information and the second field information may refer to an area in which the image content of some parts has changed due to the movement of an object or other reasons in two consecutive fields of the video (i.e., odd fields and even fields). These changes may be changes in brightness, color, texture, etc. For example, a difference area refers to a place where the image content is significantly different in one field compared to another. If there is an object moving in the video, the position of the object in the two fields will be different, forming a difference area; in addition, if the light source changes during the shooting process, such as the switch of a light or clouds blocking the sun, this will also cause a brightness difference between the two fields.

[0060] The non-difference area between the first field information and the second field information can refer to the area without significant changes between the two fields. The image content of these areas is basically consistent in the two fields, without obvious movement or change. For example, if there is a static background wall in the video, then this area should be the same between the two fields and belong to the non-difference area.

[0061] Step 103: Based on the difference area and the non-difference area, determine the missing field information corresponding to the target field information while retaining the target field information.

[0062] Among them, the target field information is one of the first field information and the second field information, that is, the first field information or the second field information is retained, for example, the odd field information is retained and the even field information is discarded; or the even field information is retained and the odd field information is discarded. In the case of retaining one of the field information, the other field information will be missing. This embodiment determines the corresponding missing field information, that is, determines how to restore or compensate for the missing field information. Specifically, it can be determined based on the difference area and the non-difference area. For example, for the difference area, since the image content has changed, retaining only one field will cause obvious visual distortion. Therefore, the difference method or motion compensation method in data learning (such as determining the motion vector in the odd field, predicting the position and value of the corresponding pixel in the even field based on the motion direction and speed) can be used to restore or compensate for the missing field information, and then obtain the corresponding missing field information.

[0063] Step 104: synthesize the missing field information and the target field information to obtain a progressive video frame.

[0064] Progressive video frame is a video display technology. In contrast to interlaced scanning, in a progressive video frame, each frame contains image information of all lines, that is, all lines of each frame are completely captured and displayed at the same time, rather than being divided into two fields for display. This technology can provide higher image quality and less motion artifacts.

[0065] For example, in order to perform video deinterlacing processing on an interlaced video frame to obtain a progressive video frame, while retaining one field information of the interlaced video frame, the retained field information can be synthesized with the corresponding determined missing field information to obtain a progressive video frame.

[0066] Compared with the current related technologies, the embodiments of the present disclosure can effectively deinterlace the video, and synthesize the retained single field information with the missing field information to obtain a progressive video frame. The missing field information is determined based on the difference area and the non-difference area between the two fields, and can accurately represent the content difference between the two fields, and then can be synthesized to obtain a clearer picture, thereby improving the picture quality, reducing aliasing and blur, and improving the user's viewing experience.

[0067] To further illustrate the Figure 1 The specific implementation process of the method shown in this embodiment provides the following Figure 2 The specific method shown includes:

[0068] Step 201: Obtain first field information and second field information of an interlaced video frame.

[0069] For example, odd field information and even field information of an interlaced video frame are obtained, or they may be called upper field information and lower field information, etc.

[0070] Step 202: Determine the confidence level of the difference between the first field information and the second field information.

[0071] The difference confidence between the two fields' information can be used to determine the probability that the two fields have content differences at each pixel location.

[0072] In some embodiments, a convolutional neural network model can be pre-trained, and the convolutional neural network model can be used to calculate the confidence map corresponding to the two fields, the resolution of the confidence map is the same as the resolution of the two fields, and the confidence map is used to determine the probability that the two fields have content differences at each pixel position. For example, a training set is pre-constructed, such as collecting sample interlaced video frames, determining the two field information in the sample interlaced video frames, and generating a label image for each interlaced video frame, indicating which areas have differences between the two fields, such as each pixel value of the label image is 0 or 1, where 1 indicates that there is a difference and 0 indicates that there is no difference. In the model network structure, the input layer accepts six-channel images, such as the first three channels are upper field images, and the last three channels are lower field images; the middle layer uses multiple convolutional layers, pooling layers and activation functions to extract features; the output layer uses a convolutional layer to output a single-channel confidence map, the output range is 0 to 1, and the Sigmoid activation function can be used to ensure that the output value is between 0 and 1, and the output value represents the probability that the two fields have content differences at the corresponding pixel position. Model training is performed based on the constructed training set and model network structure, and then a convolutional neural network model is trained.

[0073] Correspondingly, step 202 may specifically include: first merging the first field information and the second field information according to the RGB channels to obtain a multi-channel image; then generating a confidence map by using a convolutional neural network model for the multi-channel image, and then determining the confidence of the difference between the first field information and the second field information through the confidence map, wherein the resolution of the confidence map is the same as the image resolution of the first field information and the second field information, and the confidence map is used to determine the probability that there is a content difference between the first field information and the second field information at each pixel position.

[0074] For example, the two field images are merged according to the RGB channels to obtain a six-channel image, which is input into the above-mentioned trained convolutional neural network model for calculation. The output is a single-channel confidence map after Sigmoid activation. The resolution of the confidence map is the same as that of the two fields. The confidence map is used to determine the probability that there is a content difference between the two fields at each pixel position.

[0075] Step 203: Determine a difference area and a non-difference area between the first field information and the second field information according to the difference confidence between the first field information and the second field information.

[0076] This embodiment can accurately determine the difference area and the non-difference area between the two fields according to the probability that there is a content difference between the two fields at each pixel position.

[0077] Based on the example description in step 202, correspondingly, step 203 may specifically include: determining the difference area and the non-difference area between the first field information and the second field information according to the confidence map output by the convolutional neural network model. The confidence map can be used to determine the probability that there is a content difference between the two fields at each pixel position, and then the difference area and the non-difference area between the two fields can be accurately determined based on the confidence map.

[0078] In some examples, based on the confidence map, the difference area and the non-difference area between the two field information are determined, which may specifically include: merging the pixel positions in the confidence map where the probability of content difference is greater than a preset threshold to obtain the difference area; and merging the pixel positions in the confidence map where the probability of content difference is less than or equal to the preset threshold to obtain the non-difference area.

[0079] The preset threshold can be pre-set according to actual needs, and the preset threshold can be used to determine the pixel positions with content differences in the confidence map. For example, the preset threshold is 0.5, and the pixel positions with a probability of content differences greater than 0.5 in the confidence map are merged to obtain the difference area between the two field information; the pixel positions with a probability of content differences less than or equal to 0.5 in the confidence map are merged to obtain the non-difference area between the two field information.

[0080] Step 204: Obtain the missing field recovery result corresponding to the difference area, and obtain the missing field recovery result corresponding to the non-difference area.

[0081] In some embodiments, obtaining the missing field recovery result corresponding to the non-difference area may specifically include: for each pixel position in the non-difference area, calculating the average of the first field information and the second field information at the corresponding position as the pixel value of the missing field; and then generating the missing field recovery result corresponding to the non-difference area based on the pixel value of the missing field.

[0082] For example, in the non-difference area, since the image content has almost no change between the two fields, even if only one of the fields is retained, the information loss is minimal. Therefore, for each pixel position in the non-difference area, the mean of the two fields at the corresponding position is calculated as the pixel value of the missing field. Then, based on the pixel value of the missing field, the missing field recovery result corresponding to the non-difference area can be generated. Because the content of these areas changes little, a good transition can be achieved through the mean method.

[0083] In the difference area, since the image content has changed, retaining only one field will cause obvious visual distortion. These areas need special processing to restore or compensate for the missing information. In some embodiments, a first semantic segmentation model can be pre-trained. The first semantic segmentation model can be calculated based on the field information of the corresponding positions of the two fields in the difference area to obtain the missing field recovery result corresponding to the difference area, and then restore or compensate for the missing field information for the difference area. For example, the first semantic segmentation model can be trained using a deep learning method based on different sample interlaced video frames.

[0084] Correspondingly, obtaining the missing field recovery result corresponding to the difference area may specifically include: merging the field information of the first field information and the second field information at corresponding positions in the difference area according to the RGB channels to obtain a target multi-channel image; performing missing field recovery processing based on the target multi-channel image through the first semantic segmentation model to obtain the missing field recovery result corresponding to the difference area.

[0085] For example, the field information of the corresponding positions of the two fields in the difference area are spliced ​​into a six-channel image according to the RGB channels, and then the six-channel image is input into the first semantic segmentation model for calculation, and the missing field recovery result corresponding to the difference area is output.

[0086] The disclosed embodiment uses a neural network adaptive learning processing scheme to perform video deinterlacing, which has higher flexibility and accuracy, and can remove image noise while deinterlacing. By applying the technical solution of the disclosed embodiment, the video content is determined, and different processing methods are used for different areas, which can improve processing efficiency and reduce processing time and computing overhead.

[0087] Step 205: Fuse the missing field recovery result corresponding to the difference area with the missing field recovery result corresponding to the non-difference area to obtain the corresponding missing field information while retaining the target field information.

[0088] The target field information may be one of the first field information and the second field information.

[0089] For example, the missing field recovery result corresponding to the difference area is fused with the missing field recovery result corresponding to the non-difference area, or weighted fusion is performed to obtain the corresponding missing field information.

[0090] Step 206: Compare the areas of the difference region and the non-difference region.

[0091] For example, the area of ​​the difference region and the area of ​​the non-difference region are added to obtain the total area, the ratio A of the area of ​​the difference region to the total area is obtained, and the ratio B of the non-difference region to the total area is obtained, and then the ratio A and the ratio B are compared.

[0092] Step 207a: If the area of ​​the non-difference region is greater than or equal to the area of ​​the difference region by a preset multiple, synthesize the missing field information with the target field information to obtain a progressive video frame.

[0093] For example, if after area comparison it is determined that the area of ​​the non-difference area is larger than 4 times the area of ​​the difference area, it means that the area of ​​the non-difference area accounts for more than 80% of the total area, and it is considered that the difference between the missing field information and the current two field information is not large, and the missing field information can be directly spliced ​​with the retained single field information to obtain a line-by-line video frame.

[0094] Step 207b parallel to step 207a: if the area of ​​the non-difference region is smaller than the area of ​​the difference region by a preset multiple, the missing field information and the target field information are respectively subjected to image enhancement processing through a second semantic segmentation model to obtain enhanced missing field information and enhanced target field information, and the enhanced missing field information and the enhanced target field information are synthesized to obtain a line-by-line video frame.

[0095] In some embodiments, the second semantic segmentation model can be used to enhance the image of the field information, and the enhanced field information can make the stitching effect better and the transition more natural. For example, if after area comparison, it is determined that the area of ​​the non-difference area is less than 4 times the area of ​​the difference area, it means that the area of ​​the non-difference area accounts for less than 80% of the total area, and it is considered that the difference between the missing field information and the current two field information is large. The second semantic segmentation model is used for image enhancement, and the missing field information and the retained single field information are input into the second semantic segmentation model for calculation, and the enhanced missing field information and the enhanced single field information are output, and then the enhanced missing field information and the enhanced single field information are spliced ​​to obtain.

[0096] In the embodiment of the present disclosure, both the first semantic segmentation model and the second semantic segmentation model may include a UNet model.

[0097] Based on the above embodiments, in some examples, the use scenarios of the embodiments of the present disclosure may be video encoding and video display. In the video encoding scenario, in order to reduce the transmission bandwidth and encoding overhead, the progressive video source can be sampled according to interlaced encoding, which reduces the transmission bandwidth by half; in the decoding process, the method of the embodiment of the present disclosure is used to restore the missing video field, thereby improving the encoding efficiency. In the video display scenario, the method of the embodiment of the present disclosure is used to enhance the video for the received interlaced video information to improve the user's viewing experience.

[0098] In order to illustrate the implementation process of the embodiment of the present disclosure, the following application examples are given: Figure 3 As shown, including:

[0099] Step 01: Input an interlaced video frame It ;

[0100] Step 02: Interlaced video frame I t Divided into two fields, according to the definition of interlaced video, the odd lines are the upper half F uD , even-numbered rows are the second half F down ;

[0101] Step 03: Use the convolutional neural network model to calculate the difference confidence between the upper and lower fields. This step uses a convolutional neural network with the input of the six-channel image concat(F up , F down ), concat(·) represents the channel merging operation; the output is the single channel confidence map Sigmoid (Conf t ), whose resolution is similar to F up and F down The same, the value range is 0 to 1, representing the pixel position, F up and F down The probability of content differences;

[0102] Step 04: Use the threshold T set in advance to calculate the confidence map Sigmoid (Conf t ) is used for judgment. If the difference probability is greater than T, there is a difference between the two fields at the pixel position. The difference area between the two fields is extracted and the difference is calculated by Mask. same Identify the non-differential area and perform interception;

[0103] Step 05: For the non-difference area (i.e. Mask same ), use F up and F down The mean of is taken as the pixel value of the missing field, and the restoration result of the missing field is obtained.

[0104] Step 06: Difference area (i.e. l-Mask same ), the missing field is restored using the first semantic segmentation model, the input of the first semantic segmentation model is the six-channel image concat(F up , F down ), the output is the recovery result of the missing field

[0105] Step 07: and Fusion is performed to obtain the missing field image F missing ;

[0106] Step 08: Mask in step 04 sameIf the area of ​​the region accounts for more than 80% of the total area, it is considered that the missing field information is not much different from the current upper and lower field information, so F is directly output. missing With F down The spliced ​​progressive video frames; otherwise, it is considered that the missing field information is significantly different from the current two field information, and a second semantic segmentation model is used for image enhancement. The input of the second semantic segmentation model is F missing and F down , the output is the enhanced result and Splice the two into progressive video frames.

[0107] The disclosed embodiments can adopt different processing methods for different video contents and different areas, thereby improving processing efficiency, reducing processing time and computing overhead, improving image quality, and suppressing noise in the video.

[0108] Figure 4 is a block diagram of a video processing device according to some embodiments of the present disclosure. Figure 4 The device includes: an acquisition module 31, a first determination module 32, a second determination module 33, and a synthesis module 34.

[0109] An acquisition module 31 is configured to acquire first field information and second field information of an interlaced video frame;

[0110] A first determining module 32 is configured to determine a difference area and a non-difference area between the first field information and the second field information;

[0111] A second determining module 33 is configured to determine, based on the difference area and the non-difference area, corresponding missing field information while retaining target field information, wherein the target field information is one of the first field information and the second field information;

[0112] The synthesis module 34 is configured to synthesize the missing field information and the target field information to obtain a progressive video frame.

[0113] In some embodiments, the first determination module 32 is specifically configured to determine a difference confidence between the first field information and the second field information; and then determine the difference area and the non-difference area according to the difference confidence.

[0114] In some embodiments, the first determination module 32 is further configured to merge the first field information and the second field information according to RGB channels to obtain a multi-channel image; generate a confidence map from the multi-channel image through a convolutional neural network model, and the resolution of the confidence map is the same as the image resolution of the first field information and the second field information. The confidence map is used to determine the probability that there is a content difference between the first field information and the second field information at each pixel position.

[0115] In some embodiments, the first determination module 32 is further configured to determine the difference area and the non-difference area according to the confidence map.

[0116] In some embodiments, the first determination module 32 is further specifically configured to merge the pixel positions in the confidence map where the probability of content difference is greater than a preset threshold to obtain the difference area; and merge the pixel positions in the confidence map where the probability of content difference is less than or equal to the preset threshold to obtain the non-difference area.

[0117] In some embodiments, the second determination module 33 is specifically configured to obtain the missing field recovery result corresponding to the difference area; and, obtain the missing field recovery result corresponding to the non-difference area; merge the missing field recovery result corresponding to the difference area with the missing field recovery result corresponding to the non-difference area to obtain the missing field information.

[0118] In some embodiments, the second determination module 33 is further specifically configured to merge the field information of the first field information and the second field information at corresponding positions in the difference area according to RGB channels to obtain a target multi-channel image; perform missing field recovery processing based on the target multi-channel image through a first semantic segmentation model to obtain a missing field recovery result corresponding to the difference area.

[0119] In some embodiments, the second determination module 33 is further configured to calculate, for each pixel position in the non-difference area, the mean of the first field information and the second field information at the corresponding position as the pixel value of the missing field; and generate a missing field recovery result corresponding to the non-difference area based on the pixel value of the missing field.

[0120] In some embodiments, the synthesis module 34 is specifically configured to compare the areas of the difference area and the non-difference area; if the area of ​​the non-difference area is greater than or equal to a preset multiple of the area of ​​the difference area, the missing field information is synthesized with the target field information to obtain a line-by-line video frame.

[0121] In some embodiments, the synthesis module 34 is further configured to perform image enhancement processing on the missing field information and the target field information respectively through a second semantic segmentation model if the area of ​​the non-difference area is smaller than the area of ​​the difference area of ​​the preset multiple to obtain enhanced missing field information and enhanced target field information; and synthesize the enhanced missing field information with the enhanced target field information to obtain a line-by-line video frame.

[0122] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0123] It should be noted that for other corresponding descriptions of the functional units involved in the video processing device provided in this embodiment, reference can be made to Figure 1 to Figure 2 The corresponding description in will not be repeated here.

[0124] Based on the above Figure 1 to Figure 2 The method shown in the embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figure 1 to Figure 2 The method shown.

[0125] Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present disclosure.

[0126] Based on the above Figure 1 to Figure 2 The method shown, and Figure 4 In order to achieve the above-mentioned purpose, the embodiment of the present disclosure also provides an electronic device, such as a terminal such as a high-definition television, a computer, a projector, etc. with a liquid crystal display, a plasma display, a digital light processing display, etc., the device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 to Figure 2 The method shown.

[0127] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0128] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0129] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0130] Based on the above Figure 1 to Figure 2 The method shown, and Figure 4 The virtual device embodiment shown in the embodiment further provides a chip, including one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the above-mentioned Figure 1 to Figure 2 The method shown.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present disclosure can be implemented by means of software plus a necessary general hardware platform, or by hardware. By applying the solution of the embodiment of the present disclosure, different processing methods can be used for different video contents and different areas, thereby improving processing efficiency, reducing processing time and computing overhead, and improving image quality and suppressing noise in the video.

[0132] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0133] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A video processing method, characterized in that: include: Get the first field information and the second field information of the interlaced video frame; Determine a difference area and a non-difference area between the first field information and the second field information; Determine, based on the difference area and the non-difference area, corresponding missing field information while retaining target field information, where the target field information is one of the first field information and the second field information; The missing field information and the target field information are synthesized to obtain a progressive video frame.

2. The method according to claim 1, characterized in that Determining a difference area and a non-difference area between the first field information and the second field information includes: determining a confidence level of a difference between the first field information and the second field information; The difference area and the non-difference area are determined according to the difference confidence.

3. The method according to claim 2, characterized in that The determining a confidence level of a difference between the first field information and the second field information comprises: Merging the first field information and the second field information according to RGB channels to obtain a multi-channel image; The multi-channel image is generated through a convolutional neural network model to obtain a confidence map, the resolution of the confidence map is the same as the image resolution of the first field information and the second field information, and the confidence map is used to determine the probability that there is a content difference between the first field information and the second field information at each pixel position.

4. The method according to claim 3, characterized in that: The determining the difference area and the non-difference area according to the difference confidence level includes: The difference area and the non-difference area are determined according to the confidence map.

5. The method according to claim 4, characterized in that The determining the difference area and the non-difference area according to the confidence map includes: Merging pixel positions in the confidence map whose probability of having content differences is greater than a preset threshold to obtain the difference area; The pixel positions in the confidence map whose probability of having content difference is less than or equal to the preset threshold are merged to obtain the non-difference area.

6. The method according to claim 1, characterized in that Determining, based on the difference area and the non-difference area, corresponding missing field information while retaining target field information includes: Obtaining a missing field recovery result corresponding to the difference area; and, Obtaining a missing field recovery result corresponding to the non-difference area; The missing field recovery result corresponding to the difference area is merged with the missing field recovery result corresponding to the non-difference area to obtain the missing field information.

7. The method according to claim 6, characterized in that The obtaining of the missing field recovery result corresponding to the difference area includes: Merging the first field information and the second field information at corresponding positions in the difference area according to RGB channels to obtain a target multi-channel image; Based on the target multi-channel image, a missing field recovery process is performed through a first semantic segmentation model to obtain a missing field recovery result corresponding to the difference area.

8. The method according to claim 6, characterized in that The obtaining of the missing field recovery result corresponding to the non-difference area includes: For each pixel position in the non-difference area, calculating the average value of the first field information and the second field information at the corresponding position as the pixel value of the missing field; Based on the pixel value of the missing field, a missing field restoration result corresponding to the non-difference area is generated.

9. The method according to claim 1, characterized in that: The step of synthesizing the missing field information and the target field information to obtain a progressive video frame includes: Comparing the areas of the difference region and the non-difference region; If the area of ​​the non-difference region is greater than or equal to a preset multiple of the area of ​​the difference region, the missing field information is synthesized with the target field information to obtain a progressive video frame.

10. The method according to claim 9, characterized in that After comparing the areas of the difference region and the non-difference region, the method further includes: If the area of ​​the non-difference region is smaller than the area of ​​the difference region of the preset multiple, the missing field information and the target field information are respectively subjected to image enhancement processing by a second semantic segmentation model to obtain enhanced missing field information and enhanced target field information; The enhanced missing field information and the enhanced target field information are synthesized to obtain a progressive video frame.

11. A video processing device, characterized in that: include: An acquisition module is configured to acquire first field information and second field information of an interlaced video frame; A first determining module, configured to determine a difference area and a non-difference area between the first field information and the second field information; A second determining module is configured to determine, based on the difference area and the non-difference area, corresponding missing field information while retaining target field information, wherein the target field information is one of the first field information and the second field information; The synthesis module is configured to synthesize the missing field information and the target field information to obtain a progressive video frame.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

14. A chip, characterized in that: The electronic device comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, wherein the signal includes a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 10.

Images