Interlaced video data processing method, apparatus and system, and video processing device

Abstract

Embodiments of the present application disclose a method, device and system for processing interlaced video data and a video processing apparatus. The method comprises obtaining image data of a previous field and image data of a current field in an interlaced video source, wherein the current field is located after the previous field; performing field interpolation value operation according to the current field image data to obtain field interpolation value data of a to-be-supplemented field; performing inter-field interpolation value operation according to the previous field image data and the current field image data to obtain inter-field interpolation value data of the to-be-supplemented field; performing motion estimation according to the previous field image data and the current field image data to obtain a motion weight value of the to-be-supplemented field; obtaining to-be-supplemented field image data according to the field interpolation value data, the inter-field interpolation value data and the motion weight value; and merging the current field image data and the to-be-supplemented field image data to obtain frame image data. Embodiments of the present application can reduce delay time and improve video fluency and display effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a method for processing interlaced video data, an apparatus for processing interlaced video data and a video processing device. BACKGROUND

[0002] Most of the current video sources and display devices are interlace-based, but there are still some application scenarios for de-interlacing. Interlaced video sources first appeared because of the need to save transmission bandwidth. Since an interlaced video source has only half the data amount of a progressive video source, it can achieve a transmission of twice the frame rate under the same bandwidth. Early televisions usually used interlaced scanning for display, so they were well suited to interlaced video sources. With the gradual increase in transmission bandwidth, most current displays use progressive scanning for display to improve display quality. Although they can also support interlaced scanning, interlaced scanning will cause obvious picture jitter problems, and the display quality is much worse than that of progressive scanning. However, there are some progressive video sources at present: 1) interlaced video sources left over from the past; 2) video cameras that still support interlaced video source output; and 3) in application scenarios where the transmission bandwidth is insufficient, it is still necessary to artificially downsample a progressive video to an interlaced video for transmission.

[0003] In order to make the current display device compatible with the interlaced video source, it is necessary to perform de-interlacing on the interlaced video source. The most commonly used method at present is a de-interlacing method based on motion estimation. However, this method needs to rely on the image data of the next field of the current field, that is, only after the image data of the next field is obtained, the de-interlacing method can output the interpolation result (or image data) of the current field, which leads to at least one frame of delay in the current de-interlacing method, affecting the display effect. SUMMARY

[0004] Embodiments of the present application provide a method for processing interlaced video data, an apparatus for processing interlaced video data, a system for processing interlaced video data and a video processing device, which reduce the delay time and improve the smoothness and display effect of the video.

[0005] In one aspect, the embodiment of the present application provides a method for processing interlaced video data, comprising: obtaining image data of a previous field and image data of a current field in an interlaced video source to obtain previous field image data and current field image data, wherein the current field is located after the previous field; performing field interpolation on the current field image data to obtain field interpolation data corresponding to a to-be-supplemented field; performing field interpolation on the previous field image data and the current field image data to obtain field interpolation data corresponding to the to-be-supplemented field; performing motion estimation on the previous field image data and the current field image data to obtain a motion weight corresponding to the to-be-supplemented field; obtaining to-be-supplemented field image data according to the field interpolation data, the field interpolation data, and the motion weight corresponding to the to-be-supplemented field; and merging the current field image data and the to-be-supplemented field image data to obtain frame image data.

[0006] The technical solution described above realizes the conversion of the interlaced video source into the progressive video source by performing field interpolation, field interpolation, and motion estimation on the previous field image data and the current field image data to obtain the to-be-supplemented field image data, and finally obtaining the complete frame image data of the interlaced video source, reduces the problem of at least one frame delay in the prior art that requires the image data of the next field after the current field to realize the de-interlacing of the video source, eliminates the dependence on the next field data, reduces the delay time, and improves the fluency and display effect of video playing.

[0007] In one embodiment of the present application, the motion weight corresponding to the to-be-supplemented field is obtained by performing motion estimation on the previous field image data and the current field image data, comprising: obtaining field difference data corresponding to the to-be-supplemented field according to the current field image data; obtaining field difference weight corresponding to the to-be-supplemented field according to the field difference data; obtaining field difference weight corresponding to the to-be-supplemented field according to the previous field image data and the field interpolation data; and obtaining the motion weight according to the field difference weight and the field difference weight.

[0008] In one embodiment of the present application, the field difference data corresponding to the to-be-supplemented field is obtained according to the current field image data, comprising: performing absolute difference operation on image data of a first target pixel point in a first target row of the current field image data and image data of a second target pixel point in a second target row of the current field image data to obtain field difference data of a third target pixel point corresponding to the first target pixel point and the second target pixel point in a third target row of the to-be-supplemented field, wherein the second target row is adjacent to the first target row and the second target pixel point corresponds to the first target pixel point.

[0009] In one embodiment of the present application, the field difference weight value corresponding to the to-be-supplemented field is obtained according to the field difference data, comprising: obtaining a field difference average value of the field difference data by averaging the field difference data in a first preset range; and obtaining the field difference weight value according to the field difference average value.

[0010] In one embodiment of the present application, the inter-field difference weight value corresponding to the to-be-supplemented field is obtained according to the previous field image data and the field interpolation data, comprising: obtaining inter-field difference data corresponding to the to-be-supplemented field according to the previous field image data and the field interpolation data; and obtaining the inter-field difference weight value corresponding to the to-be-supplemented field according to the inter-field difference data.

[0011] In one embodiment of the present application, the inter-field difference data corresponding to the to-be-supplemented field is obtained according to the previous field image data and the field interpolation data, comprising: performing absolute difference operation on image data of a fourth target pixel point in a fourth target row of the previous field image data and image data of a fifth target pixel point in a fifth target row of the field interpolation data to obtain inter-field difference data of a sixth target pixel point corresponding to the fourth target pixel point and the fifth target pixel point in a sixth target row corresponding to the to-be-supplemented field, wherein the fourth target row corresponds to the fifth target row and the fourth target pixel point corresponds to the fifth target pixel point.

[0012] In one embodiment of the present application, the inter-field difference weight value corresponding to the to-be-supplemented field is obtained according to the inter-field difference data, comprising: obtaining an inter-field difference average value of the inter-field difference data by averaging the inter-field difference data in a second preset range; and obtaining the inter-field difference weight value according to the inter-field difference average value.

[0013] In another aspect, the embodiment of the present application provides a kind of interlaced video data processing device, for realizing the interlaced video data processing method described above and comprising: field data acquisition module, for obtaining the continuous previous field image data and current field image data in interlaced video source, wherein the current field image data is located after the previous field image data;Field interpolation module, for carrying out field interpolation operation according to the current field image data to obtain the field interpolation data corresponding to the field to be supplemented;Inter-field interpolation module, for carrying out inter-field interpolation operation according to the previous field image data and the current field image data to obtain the inter-field interpolation data corresponding to the field to be supplemented;Motion weight determination module, for carrying out motion estimation according to the previous field image data and the current field image data to obtain motion weight;To be supplemented field data determination module, for obtaining the field image data to be supplemented according to the field interpolation data, the inter-field interpolation data and the motion weight corresponding to the field to be supplemented;And image data merging module, for merging the current field image data and the field image data to be supplemented to obtain frame image data.

[0014] The interlaced video data processing device in the above technical solution carries out field interpolation, inter-field interpolation and motion estimation according to the previous field image data and the current field image data to obtain the field image data to be supplemented, and finally obtains the complete frame image data of the interlaced video source, realizes the conversion of the interlaced video source to the progressive video source, reduces the problem of at least one frame delay in the prior art that needs to use the next field image data after the current field to realize the de-interlacing of the video source, eliminates the dependence on the next field data, reduces the delay time, and improves the fluency and display effect of video playing.

[0015] In another aspect, the embodiment of the present application provides a kind of video processing equipment, comprising; video input interface;And programmable logic device, electrically connected to the video input interface;Wherein, the programmable logic device is used to obtain the previous field image data and current field image data by the video input interface for obtaining the continuous previous field and current field in interlaced video source image data, carry out field interpolation operation according to the current field image data to obtain the field interpolation data corresponding to the field to be supplemented, carry out inter-field interpolation operation according to the previous field image data and the current field image data to obtain the inter-field interpolation data corresponding to the field to be supplemented, carry out motion estimation according to the previous field image data and the current field image data to obtain the motion weight corresponding to the field to be supplemented, obtain the field image data to be supplemented according to the field interpolation data, the inter-field interpolation data and the motion weight corresponding to the field to be supplemented, and merge the current field image data and the field image data to be supplemented to obtain frame image data, wherein the current field is located after the previous field.

[0016] The technical scheme has the following advantages or beneficial effects: the technical scheme obtains the to-be-supplemented field image data by field interpolation, inter-field interpolation and motion estimation according to the previous field image data and the current field image data, and finally obtains the complete frame image data of the interlaced video source, realizes the conversion of the interlaced video source into the progressive video source, reduces the problem of at least one frame delay existing in the prior art when the next field image data after the current field is used to realize the de-interlacing of the video source, eliminates the dependence on the next field data, reduces the delay time, and improves the fluency and display effect of video playing.

[0017] In another aspect, the embodiment of the present application provides an interlaced video data processing system, comprising a processor and a memory connected to the processor; wherein the memory stores a computer program, and the processor executes the computer program to perform the interlaced video data processing method as described above.

[0018] In another aspect, the embodiment of the present application provides a computer readable storage medium, which is a non-volatile memory and stores computer executable instructions, and the computer executable instructions are used to perform the interlaced video data processing method as described above.

[0019] The technical scheme has the following advantages or beneficial effects: the technical scheme obtains the to-be-supplemented field image data by field interpolation, inter-field interpolation and motion estimation according to the previous field image data and the current field image data, and finally obtains the complete frame image data of the interlaced video source, realizes the conversion of the interlaced video source into the progressive video source, reduces the problem of at least one frame delay existing in the prior art when the next field image data after the current field is used to realize the de-interlacing of the video source, eliminates the dependence on the next field data, reduces the delay time, and improves the fluency and display effect of video playing. In addition, the field difference average value and the inter-field difference average value are used to replace the field difference and the inter-field difference within a preset range to solve the field difference weight and the inter-field difference weight, which ensures the continuity of the interpolation result (or image data) in the preset area and improves the de-interlacing effect. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 The flowchart of the interlaced video data processing method provided by the first embodiment of the present application is shown.

[0022] Figure 2Detailed flowchart of step 15 in Figure 1 Detailed flowchart of step 152 in

[0023] Figure 3 Detailed flowchart of step 152 in Figure 2 Detailed flowchart of step 153 in

[0024] Figure 4 Detailed flowchart of step 153 in Figure 2 Detailed flowchart of step 1533 in

[0025] Figure 5 Detailed flowchart of step 1533 in Figure 4 Detailed flowchart of motion estimation in

[0026] Figure 6 Structure diagram of video processing device provided by first embodiment of the present application.

[0027] Figure 7a Process diagram of interlaced video data processing method provided by an embodiment of the present application.

[0028] Figure 7b Detailed flowchart of motion estimation in Figure 7a Detailed flowchart of motion estimation in

[0029] Figure 7c Another process diagram of interlaced video data processing method provided by an embodiment of the present application.

[0030] Figure 8 Module diagram of interlaced video data processing device provided by second embodiment of the present application.

[0031] Figure 9 Structure diagram of interlaced video data processing system provided by third embodiment of the present application.

[0032] Figure 10 Structure diagram of computer readable storage medium provided by fourth embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0034]

First Embodiment

[0035] As Figure 1As shown, the first embodiment of the present application provides a method for processing interlaced video data. The method for processing interlaced video data, for example, comprises:

[0036] S11: obtaining image data of a previous field and image data of a current field in an interlaced video source to obtain previous field image data and current field image data, wherein the current field is located after the previous field;

[0037] S12: performing field interpolation operation according to the current field image data to obtain field interpolation data corresponding to a to-be-supplemented field;

[0038] S13: performing inter-field interpolation operation according to the previous field image data and the current field image data to obtain inter-field interpolation data corresponding to the to-be-supplemented field;

[0039] S14: performing motion estimation according to the previous field image data and the current field image data to obtain a motion weight corresponding to the to-be-supplemented field;

[0040] S15: obtaining to-be-supplemented field image data according to the field interpolation data, the inter-field interpolation data and the motion weight of the to-be-supplemented field; and

[0041] S16: merging the current field image data and the to-be-supplemented field image data to obtain frame image data.

[0042] In this way, the embodiment of the present application realizes the conversion of the interlaced video source to the progressive video source by performing field interpolation, inter-field interpolation and motion estimation according to the previous field image data and the current field image data to obtain the to-be-supplemented field image data and finally obtain the complete frame image data of the interlaced video source, reduces the problem of at least one frame delay in the prior art that requires the image data of the next field after the current field to realize the de-interlacing of the video source, eliminates the dependence on the next field data, reduces the delay time, and improves the fluency and display effect of video playing.

[0043] As described above, Figure 2 As shown, step S15, for example, comprises:

[0044] S151: obtaining field difference data corresponding to the to-be-supplemented field according to the current field image data;

[0045] S152: obtaining field difference weight corresponding to the to-be-supplemented field according to the field difference data;

[0046] S153: obtaining inter-field difference weight corresponding to the to-be-supplemented field according to the previous field image data and the field interpolation data; and

[0047] S155: obtaining the motion weight value corresponding to the to-be-supplemented field according to the inter-field difference weight value and the intra-field difference weight value.

[0048] Specifically, the step S1511, for example, comprises: performing absolute difference operation on image data of a first target pixel point in a first target row of the current field image data and image data of a second target pixel point in a second target row of the current field image data to obtain intra-field difference data of a third target pixel point corresponding to the first target pixel point and the second target pixel point in a third target row corresponding to the to-be-supplemented field, wherein the second target row is adjacent to the first target row and the second target pixel point corresponds to the first target pixel point.

[0049] Therefore, the intra-field difference data corresponding to the to-be-supplemented field can be obtained according to the intra-field interpolation data of all pixel points of the current field.

[0050] Further, as shown in the step S152, for example, the step S152 comprises: Figure 3

[0051] S1521: obtaining an intra-field difference average value of the intra-field difference data by averaging the intra-field difference data in a first preset range; and

[0052] S1523: obtaining the intra-field difference weight value according to the intra-field difference average value.

[0053] The first preset range herein may, for example, be one circle of pixel points around the target pixel point, or two circles, three circles or the like of pixel points around the target pixel point, and can be determined according to actual conditions.

[0054] In addition, as shown in the step S153, for example, the step S153 comprises: Figure 4

[0055] S1531: obtaining inter-field difference data corresponding to the to-be-supplemented field according to the previous field image data and the intra-field interpolation data; and

[0056] S1533: obtaining the inter-field difference weight value corresponding to the to-be-supplemented field according to the inter-field difference data.

[0057] Specifically, the step S1531, for example, comprises: performing absolute difference operation on image data of a fourth target pixel point in a fourth target row of the previous field image data and image data of a fifth target pixel point in a fifth target row of the intra-field interpolation data to obtain inter-field difference data of a sixth target pixel point corresponding to the fourth target pixel point and the fifth target pixel point in a sixth target row corresponding to the to-be-supplemented field, wherein the fourth target row corresponds to the fifth target row and the fourth target pixel point corresponds to the fifth target pixel point.

[0058] ​​Therefore, the inter-field difference data corresponding to the to-be-supplemented field can be obtained according to the image data of all pixel points of the previous field and the field interpolation data of the corresponding to-be-supplemented field in the current field.

[0059] Further, as shown in Figure 5 S15331 comprises, for example:

[0060] S15331: averaging the inter-field difference data in a second preset range to obtain an inter-field difference average value of the inter-field difference data; and

[0061] S15333: obtaining the inter-field difference weight according to the inter-field difference average value.

[0062] The second preset range herein may, for example, be a circle of pixel points around the target pixel point, or two circles, three circles, etc. of pixel points around the target pixel point, and can be determined according to actual conditions.

[0063] To facilitate understanding of the present application, the data processing method of the present embodiment will be described in detail below. Figure 6 to Figure 7c

[0064] The interlaced video data processing method provided by the present application is suitable for, for example, a device such as a video processing device that needs to perform de-interlacing processing on an interlaced video source, so that the video processing device can be compatible with the input of interlaced video sources and progressive video sources, and display the interlaced video source on a progressive display device. Typically, as shown in Figure 6 ​As shown, the video processing device 500 comprises, for example, a video input interface 510, a programmable logic device 530, and a video output interface 550, wherein the programmable logic device 530 is connected between the video input interface 510 and the video output interface 550. The video input interface 510 and the video output interface 550 may, for example, be a standard video interface commonly available in the market, such as an HDMI interface, a DP interface, an SDI interface, a DVI interface, etc., or other interfaces capable of transmitting video data, without being limited thereto. The programmable logic device 530 may, for example, be an FPGA (Field Programmable Gate Array), which is mainly used for the programmable logic device 133 to process the image data obtained from the video source, such as gamma transformation, color gamut conversion, filtering, layer superposition, de-interlacing, etc., and output the processed video data to a subsequent device, such as a display control system or other video processing device or display device, through the video output interface 550. The model of the programmable logic device 113 may, for example, be EP4CE30F29C8N. In addition, the video processing device 500 can further comprise a microcontroller 570. The microcontroller 570 is connected to the programmable logic device 530. The microcontroller 570 may, for example, be an MCU or an ARM core-based processor, without being limited thereto. It is worth mentioning here that the components of the video processing device 500 can be arranged on the same circuit board, or can be arranged separately on different circuit boards, such as a plug-in video processing device, without being limited thereto. In addition, the video input interface 510 of the video processing device 500 can access a progressive video source or receive an interlaced video source. The video processing device 500 judges and identifies the accessed video source. When the accessed video source is identified as a progressive video source, the programmable logic device 530 of the video processing device 500 does not perform de-interlacing processing (i.e., the interlaced video data processing provided in the embodiments of the present application) on it. When the accessed video source is identified as an interlaced video source, the programmable logic device 530 of the video processing device 500 performs de-interlacing processing on the interlaced video source to obtain frame image data in progressive video source format, and then the programmable logic device 530 performs subsequent processing on the frame image data.

[0065] The method provided in the embodiments of the present application realizes the conversion of the interlaced video source into the progressive video source by performing field interpolation, inter-field interpolation, and motion estimation according to the previous field image data and the current field image data to obtain the to-be-supplemented field image data and finally obtain the complete frame image data of the interlaced video source, reduces the problem of at least one frame delay in the prior art which needs to use the next field image data after the current field to realize the de-interlacing of the video source, eliminates the dependence on the next field data, reduces the delay time, and improves the fluency and display effect of video playing.

[0066] It is worth mentioning that the interlaced video data processing method provided by the present application is applied to the programmable logic device 530 on the video processing device 500, and the implementation process is as follows.

[0067] Firstly, the video processing device 500 receives a video source through the video input interface 510, and judges whether the type of the video source is an interlaced video source. The judgment process can be achieved by reading the video source related information of the video source, such as the type of the video source, through the programmable logic device 530, and the existing technology can be used to achieve the process, and the specific process will not be repeated here.

[0068] Then, when the video source is an interlaced video source, the programmable logic device 530 obtains the image data of the previous field and the image data of the current field through the video input interface 510, wherein the current field is located after the previous field. The image data here is, for example, image data represented in RGB format, or image data represented in YUV format, and the present application is not limited to this. It is worth mentioning that for a progressive video source, one field of the video source transmits the whole frame image data, while for an interlaced video source, one field of the video source transmits half of the frame image data, that is, the image data of this field can include only odd rows, that is, 2n+1 (n is a positive integer) rows of image data, such as the first row, the third row, the fifth row, etc., or it can include only even rows, that is, 2n+2 (n is a positive integer) rows of image data, such as the second row, the fourth row, the sixth row, etc. When the image data of the field includes odd rows of image data, it can be called an odd field; when the image data of the field includes even rows of image data, it can be called an even field. The transmission characteristic of the interlaced video source is that an odd field, an even field, an odd field, and so on are transmitted alternately. When the current field is an odd field, the previous field and the next field are both even fields, and when the current field is an even field, the previous field and the next field are both odd fields. Here, the image data of the previous field and the current field is obtained, that is, the image data of an odd field and an even field is obtained. When displaying, the whole frame image data needs to be displayed for each field, so for an odd field, the even rows of image data in the frame image data, that is, the even field data corresponding to the odd field, need to be supplemented, that is, the field to be supplemented is an even field; for an even field, the field to be supplemented is an odd field. When the current field and the field to be supplemented are merged, the whole frame image data is obtained, thereby realizing the de-interlacing processing. For example, the resolution of the whole frame image data of the progressive video source is 64 (rows) x 64 (columns), so the resolution of the odd field of the interlaced video source is 32 (rows) x 64 (columns), and the resolution of the even field is 32 (rows) x 64 (columns).

[0069] Next, as shown in FIG. 5, the programmable logic device 530 performs the de-interlacing processing on the image data of the previous field and the image data of the current field through the video input interface 510, and obtains the image data of the previous field and the image data of the current field after the de-interlacing processing. Figure 7a and 7cAs shown, the programmable logic device 530 performs field interpolation operation according to the current field image data to obtain the field interpolation data corresponding to the to-be-supplemented field. The field interpolation operation can be implemented by using the field interpolation method in the prior art, such as the field copy method, the vertical direction interpolation method, and the gradient direction interpolation method. Specifically, the field copy method generally refers to copying the image data of a target row in the current field to obtain the field interpolation data corresponding to a row in the to-be-supplemented field. The vertical direction interpolation method and the gradient direction interpolation method refer to using the image data of the adjacent upper and lower rows in the current field to perform interpolation operation to obtain the interpolation result corresponding to the missing field (i.e., the to-be-supplemented field). Since the row data of the missing field (to-be-supplemented field) and the row data of the current field are arranged in an interlaced manner, the image data of each pixel point of any two rows in the current field can be used to perform interpolation operation to obtain the interpolation data corresponding to a row in the to-be-supplemented field, that is, the field interpolation data corresponding to the to-be-supplemented field. In short, an interpolation point (pixel point) is taken from the upper row in the current field, and an interpolation point is taken from the lower row adjacent to the upper row, and then the average of the image data of the two interpolation points is taken to obtain the field interpolation data corresponding to the interpolation point (to-be-interpolated point) of the target row corresponding to the upper row and the lower row in the to-be-supplemented field. The field copy method and the vertical direction interpolation method are simple to implement, but the de-interlacing effect is not good, and the vertical resolution of the interpolation result is not obviously improved. The gradient direction interpolation method is complex to implement, but the de-interlacing effect is good, and the vertical resolution of the video can be effectively improved. The mainstream method is the gradient direction interpolation method. Here, the various operation methods of field interpolation can refer to the introduction in the prior art, which will not be described here. It is worth mentioning that, as shown in the figure, Figure 7c When the gradient direction interpolation method is used, the programmable logic device 530 needs to further perform field gradient solving and field gradient difference solving in the gradient direction to obtain the field difference in the gradient direction before interpolation. The specific method can use the gradient interpolation method in the prior art, which will not be described here.

[0070] Then, the programmable logic device 530 performs inter-field interpolation operation according to the previous field image data and the current field image data to obtain inter-field interpolation data corresponding to the to-be-supplemented field. Specifically, according to image data of a target pixel point in a target row of the previous field image data and image data of a target pixel point in a target row of the intra-field interpolation data corresponding to the to-be-supplemented field, an interpolation operation is performed to obtain inter-field interpolation data of a corresponding pixel point in a corresponding row corresponding to the to-be-supplemented field. Corresponding rows of the previous field and the current field are, for example, the first row of the previous field corresponds to the first row of the current field, and the first pixel point (interpolation point) of the first row of the previous field corresponds to the first pixel point (interpolation point) of the first row of the current field; the second row of the previous field corresponds to the second row of the current field, and the first pixel point (interpolation point) of the second row of the previous field corresponds to the first pixel point (interpolation point) of the second row of the current field. Interpolation is performed on the first pixel point (interpolation point) of the first row of the previous field and the first pixel point (interpolation point) of the first row of the current field to obtain inter-field interpolation data of the first pixel point of the first row of the to-be-supplemented field; interpolation is performed on the second pixel point (interpolation point) of the first row of the previous field and the second pixel point (interpolation point) of the first row of the current field to obtain inter-field interpolation data of the second pixel point of the first row of the to-be-supplemented field, and so on.

[0071] After that, the programmable logic device 530 performs motion estimation according to the previous field image data and the current field image data to obtain a motion weight value corresponding to the to-be-supplemented field. The motion estimation here means judging whether the image data or interpolation result of the to-be-supplemented field is more inclined to intra-field interpolation or inter-field interpolation according to the previous field image data and the current field image data, so as to obtain a motion weight value, and finally weighting the inter-field interpolation data and the intra-field interpolation data according to the motion weight value to obtain the final interpolation result (i.e., image data) of the to-be-supplemented field. Specifically, the programmable logic device 530 obtains an intra-field difference weight value corresponding to the to-be-supplemented field according to the current field image data, and then obtains an inter-field difference weight value corresponding to the to-be-supplemented field according to the previous field image data and the intra-field interpolation data; and then obtains the motion weight value corresponding to the to-be-supplemented field according to the inter-field difference weight value and the intra-field difference weight value.

[0072] As described above, the programmable logic device 530 obtains the intra-field difference weight value corresponding to the to-be-supplemented field according to the current field image data specifically includes: obtaining intra-field difference data corresponding to the to-be-supplemented field according to the current field image data, and then obtaining the intra-field difference weight value corresponding to the to-be-supplemented field according to the intra-field difference data.

[0073] Specifically, the programmable logic device 530 obtains the intra-field difference data corresponding to the to-be-supplemented field according to the current field image data, including: performing absolute difference operation on image data of a first target pixel point in a first target row of the current field image data and image data of a second target pixel point in a second target row of the current field image data, to obtain intra-field difference data of a third target pixel point corresponding to a third target row of the to-be-supplemented field, wherein the second target row is adjacent to the first target row and the second target pixel point corresponds to the first target pixel point. The absolute difference operation here is specifically to perform subtraction operation on the image data of the first target pixel point and the second target pixel point to obtain a difference value, and to take the absolute value of the difference value, and the obtained data is the intra-field difference data of the third target pixel point corresponding to the to-be-supplemented field. For example, when the current field is an odd field, two adjacent odd rows correspond, for example, the first row (corresponding to the first target row) of the current field corresponds to the third row (corresponding to the second target row) of the current field, for example, the first pixel point (corresponding to the first target pixel point) of the first row of the current field corresponds to the first pixel point (corresponding to the second target pixel point) of the third row of the current field. The second pixel point of the first row of the current field corresponds to the second pixel point of the third row of the current field. When the current field is an even field, two adjacent even rows correspond, for example, the second row (corresponding to the first target row) of the current field corresponds to the fourth row (corresponding to the second target row) of the current field, and the first pixel point (corresponding to the first target pixel point) of the second row of the current field corresponds to the first pixel point (corresponding to the second target pixel point) of the fourth row of the current field. Of course, there can be other corresponding methods, and the present application is not limited thereto. Taking the current field as an odd field as an example, the corresponding to-be-supplemented field is an even field, that is, the current field includes odd rows and the to-be-supplemented field includes even rows, and the first pixel point of the first row of the current field and the first pixel point of the third row of the current field are subjected to absolute difference operation to obtain the intra-field difference data of the first pixel point (corresponding to the third target pixel point) of the second row (corresponding to the third target row) of the to-be-supplemented field. By analogy, the second pixel point (interpolation point) of the first row of the current field and the second pixel point (interpolation point) of the third row of the current field are subjected to absolute difference operation to obtain the intra-field difference data of the second pixel point of the second row of the to-be-supplemented field, until the intra-field difference data of all pixel points of the first row of the to-be-supplemented field is obtained. Then, the first pixel point of the third row of the current field and the first pixel point of the fifth row of the current field are subjected to absolute difference operation to obtain the intra-field difference data of the first pixel point of the third row of the to-be-supplemented field, until the intra-field difference data of all rows of pixel points of the to-be-supplemented field is obtained. When the current field is an even field, the method is similar, which will not be described here. That is, the first target row and the second target row correspond to two adjacent rows of the current field, and the first target pixel point and the second target pixel point correspond to two corresponding pixel points on the two adjacent rows of the current field.

[0074] Then, the programmable logic device 530 obtains the field-in difference weight value corresponding to the field to be supplemented according to the field-in difference data. Specifically, the field-in difference weight value is obtained from the field-in difference data and satisfies:

[0075]

[0076] wherein α1 is the field-in difference weight value, and the value range of α1 is [0, 1], D intra represents the field-in difference data, Thresh1 and Thresh2 are two thresholds, wherein the value range of Thresh1 is for example [16, 128], and the value range of Thresh2 is for example [0, 1]. Of course, the values of Thresh1 and Thresh2 can be set by the user according to the actual situation.

[0077] In addition, as Figure 7b indicated, the programmable logic device 530 obtains the inter-field difference weight value corresponding to the field to be supplemented according to the previous field image data and the field interpolation value data, for example, including: obtaining the inter-field difference data corresponding to the field to be supplemented according to the previous field image data and the field interpolation value data; and obtaining the inter-field difference weight value corresponding to the field to be supplemented according to the inter-field difference data.

[0078] According to the foregoing, the programmable logic device 530 obtains the inter-field difference data corresponding to the to-be-supplemented field according to the previous field image data and the field interpolation value data, for example, including: performing absolute difference operation on the image data of a fourth target pixel point in a fourth target row of the previous field image data and the image data of a fifth target pixel point in a fifth target row of the field interpolation value data, to obtain the inter-field difference data of a sixth target pixel point corresponding to the sixth target row of the to-be-supplemented field and corresponding to the fourth target pixel point and the fifth target pixel point, wherein the fourth target row corresponds to the fifth target row and the fourth target pixel point corresponds to the fifth target pixel point. The absolute difference operation here is specifically to perform subtraction operation on the image data of the fourth target pixel point and the fifth target pixel point to obtain a difference value, and to take the absolute value of the difference value to obtain the inter-field difference data of the sixth target pixel point corresponding to the to-be-supplemented field. For example, when the previous field is an odd field, then the current field is an even field, and the field interpolation value data is obtained by interpolating two adjacent even rows of the current field, so the field interpolation value data also includes interpolation data of odd rows, therefore, the first row (corresponding to the fourth target row) of the previous field corresponds to the first row (corresponding to the fifth target row) of the field interpolation value data of the current field, and the first pixel point (corresponding to the fourth target pixel point) of the first row of the previous field corresponds to the first pixel point (corresponding to the fifth target pixel point) of the first row of the field interpolation value data of the current field. When the previous field is an even field, then the current field is an odd field, and the field interpolation value data is obtained by interpolating two adjacent odd rows of the current field, so the field interpolation value data also includes interpolation data of even rows, therefore, the second row (corresponding to the fourth target row) of the previous field corresponds to the second row (corresponding to the fifth target row) of the field interpolation value data of the current field, and the first pixel point (corresponding to the fourth target pixel point) of the second row of the previous field corresponds to the first pixel point (corresponding to the fifth target pixel point) of the second row of the field interpolation value data of the current field. The second pixel point (interpolation point) of the second row of the previous field corresponds to the second pixel point (interpolation point) of the second row of the field interpolation value data of the current field. Of course, there can be other corresponding modes, and the present application is not limited thereto.The previous field is an odd field, and the current field is an even field. That is, the previous field includes odd rows, and the current field includes even rows. The field interpolation data of the current field includes odd rows. An absolute difference operation is performed between the first pixel point (corresponding to the fourth target pixel point) of the first row (corresponding to the fourth target row) of the previous field and the first pixel point (corresponding to the fifth target pixel point) of the first row (corresponding to the fifth target row) of the field interpolation data of the current field, to obtain the inter-field difference data of the first pixel point (corresponding to the sixth target pixel point) of the first row (corresponding to the sixth target row) of the field to be supplemented. An absolute difference operation is performed between the second pixel point (interpolation point) of the first row of the previous field and the second pixel point (interpolation point) of the first row of the field interpolation data of the current field, to obtain the inter-field difference data of the second pixel point of the first row of the field to be supplemented. Similarly, the inter-field difference data of all pixel points of the first row of the field to be supplemented is obtained. Then, an absolute difference operation is performed between the first pixel point of the third row of the previous field and the first pixel point of the third row of the field interpolation data of the current field, to obtain the inter-field difference data of the first pixel point of the third row of the field to be supplemented. Similarly, the inter-field difference data of all pixel points of all rows of the field to be supplemented is obtained. When the previous field is an even field, the method is similar, which will not be described here. That is, the fourth target row and the fifth target row correspond to one pixel row of the previous field and one pixel row of the field interpolation data of the current field, respectively. The first target pixel point and the second target pixel point correspond to one pixel point on the fourth target row of the previous field and one pixel point on the fifth target row of the field interpolation data of the current field, respectively.

[0079] As described above, the programmable logic device 530 then obtains the inter-field difference weight corresponding to the field to be supplemented according to the inter-field difference data. Specifically, the inter-field difference weight is obtained from the inter-field difference data and satisfies:

[0080]

[0081] wherein a2 is the inter-field difference weight, and a2 is in the range of [0, 1], D inter represents the inter-field difference data, Thresh3 and Thresh4 are two thresholds of a2, wherein the value range of Thresh3 is, for example, [4, 32], and the value range of Thresh4 is, for example, [0, 1]. Of course, the values of Thresh3 and Thresh4 can be set by the user according to the actual situation.

[0082] Next, as Figure 7bAs shown, the programmable logic device 530 obtains the motion weight value corresponding to the field to be supplemented according to the inter-field difference weight value and the intra-field difference weight value. Specifically, the motion weight value satisfies: a = a1 x a2, where a1 is the intra-field difference weight value, a2 is the inter-field difference weight value, and a1, a2 and a are all values between 0 and 1. a2 represents the motion degree estimated by the inter-field difference, a1 is the attenuation value of a2, and the intra-field difference data is used to measure the reliability of the intra-field interpolation. The greater the intra-field difference data is, the smaller a1 is, and thus the smaller a is, and the interpolation result (or image data) of the field to be supplemented is more inclined to the inter-field interpolation data.

[0083] Then, the programmable logic device 530 obtains the image data (i.e. the interpolation result) of the field to be supplemented according to the intra-field interpolation data, the inter-field interpolation data and the motion weight value a corresponding to the field to be supplemented. The interpolation result of the field to be supplemented satisfies:

[0084] Res = Intra x a + Inter x (1-a),

[0085] where Intra represents the intra-field interpolation data, and Inter represents the inter-field interpolation data.

[0086] Finally, the programmable logic device 530 merges the current field image data and the image data corresponding to the field to be supplemented to obtain the frame image data. Specifically, the odd-even field merging refers to arranging the interpolation result (the image data of the field to be supplemented) and the current field image data in a row interlaced manner to obtain the whole frame image data, thereby completing the conversion from the interlaced video source to the progressive video source. The specific process of the merging can refer to the odd-even field merging scheme in the prior art, which will not be described here.

[0087] In addition, in other specific embodiments of the present application, in order to ensure the continuity of the interpolation result (or image data) in a certain area, after obtaining the intra-field difference D intra and the inter-field difference D inter , it is necessary to average the intra-field difference D intra and the inter-field difference D inter in a preset range to obtain the intra-field difference average D intra_LPF and the inter-field difference average D inter_LPF , and then use D intra_LPF and D inter_LPF to replace D intra and D inter to solve the weight values a1 and a2. Specifically, taking the intra-field difference data D intra as an example, the intra-field difference data D intra of the pixel points in the first preset range is averaged to obtain the intra-field difference average D intra_LPF, i.e. the field difference data of all the pixel points in a range of one pixel width from the target pixel point in the row direction and the column direction is averaged, and then the average value is used to replace the field difference of the target pixel point to obtain the field difference weight. Of course, the first preset range can also be a range of two pixel widths, three pixel widths or other widths from the target pixel point in the row direction and the column direction, and the embodiments of the present application are not limited thereto. Similarly, taking the field difference data D inter of the pixel points in the second preset range as an example, the field difference data D inter of the pixel points in the second preset range is averaged to obtain the field difference average value D intra_LPF , which is used to replace D inter to obtain the field difference weight.

[0088] In summary, the embodiments of the present application obtain the to-be-supplemented field image data by field interpolation, inter-field interpolation and motion estimation according to the previous field image data and the current field image data, and finally obtain the complete frame image data of the interlaced video source, realize the conversion of the interlaced video source to the progressive video source, reduce the problem of at least one frame delay in the prior art which needs to use the next field image data after the current field to realize the de-interlacing of the video source, eliminate the dependence on the next field data, reduce the delay time, and improve the fluency and display effect of video playing. In addition, the field difference average value D intra_LPF and the inter-field difference average value D inter_LPF are used to replace the field difference D intra and the inter-field difference D inter to obtain the field difference weight α1 and the inter-field difference weight α2, which ensures the continuity of the interpolation results (or image data) in the preset range and improves the de-interlacing effect.

[0089]

Second Embodiment

[0090] As shown in Figure 8 , the second embodiment of the present application provides an interlaced video data processing device 600. The interlaced video data processing device 600 comprises, for example: a field data acquisition module 610, a field interpolation module 620, an inter-field interpolation module 630, a motion weight determination module 640, a to-be-supplemented field data determination module 650 and an image data merging module 660.

[0091] Specifically, the field data acquisition module 610 is used to acquire consecutive previous field image data and current field image data from an interlaced video source, wherein the current field image data is located after the previous field image data. The intra-field interpolation module 620 is used to perform intra-field interpolation operations based on the current field image data to obtain intra-field interpolation data corresponding to the field to be supplemented; the inter-field interpolation module 630 is used to perform inter-field interpolation operations based on the previous field image data and the current field image data to obtain inter-field interpolation data corresponding to the field to be supplemented; the motion weight determination module 640 is used to perform motion estimation based on the previous field image data and the current field image data to obtain motion weights; the field data to be supplemented determination module 650 is used to obtain the field image data to be supplemented based on the intra-field interpolation data, the inter-field interpolation data, and the motion weights corresponding to the field to be supplemented; and the image data merging module 660 is used to merge the current field image data and the field image data to be supplemented to obtain frame image data.

[0092] Each module of the interlaced video data processing device 600 in this embodiment can be integrated, for example, into the programmable logic device in the aforementioned embodiment, and the specific working process and technical effects between each module are described in the aforementioned first embodiment.

[0093] [Third Embodiment]

[0094] like Figure 9 As shown, a third embodiment of the present invention provides an interlaced video data processing system 700. The interlaced video data processing system 700 includes, for example, a processor 730 and a memory 710 connected to the processor 730. The memory 710 may be, for example, a non-volatile memory, on which a computer program 711 is stored. The processor 730 may be, for example, a central processing unit (CPU). When the processor 730 runs the computer program 711, it executes the interlaced video data processing method provided in the first embodiment described above.

[0095] The specific working process and technical effects of the interlaced video data processing system 700 in this embodiment are described in the aforementioned first embodiment and will not be repeated here.

[0096] [Fourth Embodiment]

[0097] like Figure 10As shown, the fourth embodiment of the present application provides a computer readable storage medium 800 storing computer executable instructions 810. The computer executable instructions 810 are used to execute the media display method as described in the first embodiment. The computer readable storage medium 800 is, for example, a non-volatile memory, such as a magnetic medium (e.g., a hard disk, a floppy disk, and a magnetic tape), an optical medium (e.g., a CD ROM disk and a DVD), a magneto-optical medium (e.g., an optical disk), and a hardware device specially constructed to store and execute computer executable instructions (e.g., a read-only memory (ROM), a random access memory (RAM), a flash memory, etc.). The computer readable storage medium 800 can be executed by one or more processors or processing devices to implement the interlaced video data processing method as described in the first embodiment.

[0098] In addition, it should be understood that the above-described various embodiments are merely exemplary descriptions of the present application, and the technical solutions of the various embodiments can be combined and used as desired, without conflict in technical features, contradiction in structure, or deviation from the purpose of the present application.

[0099] In the several embodiments of the present application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the device embodiments described above are merely schematic, and the division of the units is merely a logical function division. In actual implementation, another division mode can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices, or units, and can be electrical, mechanical, or other forms.

[0100] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0101] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or hardware plus software functional units.

[0102] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of processing interlaced video data, the method comprising: The method comprises: obtaining image data of a previous field and image data of a current field in a continuous interlaced video source, wherein the current field is located after the previous field; performing field interpolation on the image data of the current field to obtain field interpolation data corresponding to a to-be-supplemented field; performing inter-field interpolation on the image data of the previous field and the image data of the current field to obtain inter-field interpolation data corresponding to the to-be-supplemented field; performing motion estimation on the image data of the previous field and the image data of the current field to obtain a motion weight corresponding to the to-be-supplemented field; obtaining to-be-supplemented field image data according to the field interpolation data, the inter-field interpolation data, and the motion weight corresponding to the to-be-supplemented field; and merging the image data of the current field and the to-be-supplemented field image data to obtain frame image data. The method further comprises: obtaining field difference data corresponding to the to-be-supplemented field according to the image data of the current field; obtaining a field difference weight corresponding to the to-be-supplemented field according to the field difference data; obtaining an inter-field difference weight corresponding to the to-be-supplemented field according to the image data of the previous field and the field interpolation data; and obtaining the motion weight according to the inter-field difference weight and the field difference weight.

2. The interlaced video data processing method of claim 1, wherein, The method further comprises: performing absolute difference operation on image data of a first target pixel point in a first target row of the image data of the current field and image data of a second target pixel point in a second target row of the image data of the current field to obtain field difference data of a third target pixel point corresponding to the first target pixel point and the second target pixel point in a third target row of the to-be-supplemented field, wherein the second target row is adjacent to the first target row and the second target pixel point corresponds to the first target pixel point.

3. The interlaced video data processing method of claim 1, wherein, The method further comprises: averaging the field difference data in a first preset range to obtain a field difference average of the field difference data; and obtaining the field difference weight according to the field difference average.

4. The interlaced video data processing method of claim 1, wherein, The method further comprises: obtaining inter-field difference data corresponding to the to-be-supplemented field according to the image data of the previous field and the field interpolation data; and obtaining the inter-field difference weight corresponding to the to-be-supplemented field according to the inter-field difference data.

5. The interlaced video data processing method of claim 4, wherein, The method further comprises: obtaining inter-field difference data corresponding to the to-be-supplemented field according to the image data of the previous field and the field interpolation data. performing absolute difference operation on image data of a fourth target pixel point in a fourth target row of the previous field image data and image data of a fifth target pixel point in a fifth target row in the field interpolation value data, to obtain field difference data of a sixth target pixel point corresponding to the fourth target pixel point and the fifth target pixel point in a sixth target row of the field to be supplemented, wherein the fourth target row corresponds to the fifth target row and the fourth target pixel point corresponds to the fifth target pixel point.

6. The interlaced video data processing method of claim 5, wherein, The field difference weight value corresponding to the field to be supplemented is obtained according to the field difference data, comprising: taking an average value of the field difference data in a second preset range to obtain a field difference average value of the field difference data; and obtaining the field difference weight value according to the field difference average value.

7. An apparatus for processing interlaced video data, the apparatus comprising: The method for processing interlaced video data according to any one of claims 1 to 6, comprising: a field data acquisition module, configured to acquire continuous previous field image data and current field image data in an interlaced video source, wherein the current field image data is located after the previous field image data; a field interpolation value module, configured to perform field interpolation value operation according to the current field image data to obtain field interpolation value data corresponding to a field to be supplemented; a field interpolation value module, configured to perform field interpolation value operation according to the previous field image data and the current field image data to obtain field interpolation value data corresponding to the field to be supplemented; a motion weight value determination module, configured to perform motion estimation according to the previous field image data and the current field image data to obtain a motion weight value; a field to be supplemented data determination module, configured to obtain field to be supplemented image data according to the field interpolation value data, the field interpolation value data and the motion weight value corresponding to the field to be supplemented; and an image data merging module, configured to merge the current field image data and the field to be supplemented image data to obtain frame image data.

8. An interlaced video data processing system, characterized by comprising: a processor and a memory connected to the processor; wherein the memory stores a computer program, and the processor executes the computer program to perform the method for processing interlaced video data according to any one of claims 1 to 6.

9. A video processing device, comprising: comprising; a video input interface; and a programmable logic device electrically connected to the video input interface; wherein the programmable logic device is configured to acquire previous field image data and current field image data through the video input interface, perform field interpolation value operation according to the current field image data to obtain field interpolation value data corresponding to a field to be supplemented, perform field interpolation value operation according to the previous field image data and the current field image data to obtain field interpolation value data corresponding to the field to be supplemented, perform motion estimation according to the previous field image data and the current field image data to obtain a motion weight value corresponding to the field to be supplemented, obtain field to be supplemented image data according to the field interpolation value data, the field interpolation value data and the motion weight value corresponding to the field to be supplemented, and merge the current field image data and the field to be supplemented image data to obtain frame image data, wherein the current field is located after the previous field. The programmable logic device is further configured to obtain intra-field difference data corresponding to the to-be-supplemented field according to the current field image data, obtain intra-field difference weight corresponding to the to-be-supplemented field according to the intra-field difference data, obtain inter-field difference weight corresponding to the to-be-supplemented field according to the previous field image data and the intra-field interpolation data, and obtain the motion weight according to the inter-field difference weight and the intra-field difference weight.

Images