On-screen image processing method

Abstract

A method of processing an on-screen display image generates an interpolated image frame having interpolated blocks and performs motion compensation to generate motion compensated data according to first and second motion vectors respectively pointing to a previous image frame and a current image frame. Zero motion data is generated according to a zero motion vector pointing to the previous image frame and the current image frame. The motion compensated data of a first weight and the zero motion data of a second weight are blended to generate each interpolated block and as a processed block. When a number of pixels having on-screen display data in the processed block is less than a first positive threshold and greater than a second positive threshold, the processed block is determined to be an extended on-screen display block and the second weight of the extended on-screen display block is less than the second weight of the on-screen display block.

Description

Technical Field

[0001] This invention relates to an image processing method, and more particularly to an on-screen display (OSD) image processing method. Background Technology

[0002] In inter-frame predictive coding, there is a certain correlation between scenes in adjacent frames of a moving image. Therefore, the moving image can be divided into several blocks. These blocks can reduce hardware costs. The position of each block in adjacent frames is found to obtain the relative offset between the spatial positions of the two blocks. The relative offset is usually called the motion vector. The process of obtaining the motion vector is called motion estimation.

[0003] Figure 1 A schematic diagram of the imaging area displayed on a screen in the prior art. Figure 2 A schematic diagram showing the imaging area on another screen using existing technology. See also... Figure 1 and Figure 2 An image frame has an on-screen display (OSD) imaging region. This image frame is divided into multiple blocks, some of which are located within the on-screen display imaging region. These blocks contain either on-screen display (OSD) data or non-on-screen display (non-OSD) data. Blocks located on the boundaries of the on-screen display imaging region contain both on-screen display data and non-on-screen display data. If the number of pixels in a block containing on-screen display data is greater than or equal to a threshold, this block is determined to be an on-screen display (OSD) block, such as... Figure 1 As shown. If the number of pixels in a block containing data to be displayed on the screen is less than this threshold, then this block will be classified as a non-OSD (non-display-on-screen) block, such as... Figure 2 As shown. Blocks displayed on the screen are represented by blank blocks, while blocks not displayed on the screen are represented by diagonal lines. For example... Figure 1 As shown, if a block representing the boundary of the imaging area on the screen is identified as a display block on the screen, it will cause a severe halo effect. For example... Figure 2 As shown, if a block at the boundary of the imaging area displayed on the screen is judged as a non-displayed block, it will cause the imaging area displayed on the screen to be broken.

[0004] Figure 3 This is a schematic diagram of a previous image frame, an interpolated image frame, and a current image frame in the prior art. Figure 4 This diagram illustrates a previous image frame, another interpolated image frame, and the current image frame in the prior art, where on-screen display blocks are represented by blank blocks, and non-OSD (non-screen display) blocks are represented by diagonal lines. See also... Figure 3 and Figure 4An interpolated image frame 1 is interpolated on the time axis between a previous image frame 1' and a current image frame 1" . The interpolated image frame 1 displays the OSD (On-Screen Display) imaging area on the screen. The interpolated image frame 1 has multiple blocks, some of which are located within the OSD imaging area on the screen. The blocks of the interpolated image frame 1 are generated by combining the data from the previous image frame 1' and the current image frame 1" pointed to by the motion vector and the zero motion vector. The blocks of the interpolated image frame 1 have a first weight using the data from the previous image frame 1' pointed to by the motion vector, a second weight using the data from the current image frame 1" pointed to by the motion vector, and a third weight using the data from the previous image frame 1' and the current image frame 1" pointed to by the zero motion vector. Figure 3 As shown, the interpolated image frame 1 block located on the boundary of the display imaging area on the screen is pointed to by two motion vectors as blocks of the previous image frame 1' and the current image frame 1" , where the pointed-to blocks of the previous image frame 1' and the current image frame 1" are determined to be on-screen display (OSD) blocks. In this example, if the interpolated image frame 1 block on the boundary of the display imaging area on the screen is determined to be an on-screen display block, a third weight will be added, but this will cause a severe halo effect. Figure 4 As shown, if the block currently pointed to by image frame 1” contains on-screen display (OSD) data, but is judged as a non-on-screen display block, and the block of interpolated image frame 1 is judged as a non-on-screen display block, then the weight corresponding to the block currently pointed to by image frame 1” will increase, but this will cause the block of interpolated image frame 1 to fly out. Summary of the Invention

[0005] The present invention provides an image processing method for displaying images on a screen, which significantly reduces halo phenomena, reduces the degree of fragmentation of the boundary of the display imaging area on the screen, and avoids the phenomenon of flying off.

[0006] In one embodiment of the present invention, a screen display image processing method is provided, which generates an interpolated image frame from a previous image frame and a current image frame. The interpolated image frame includes multiple interpolated blocks, which have three types: on-screen display (OSD) blocks, non-on-screen display blocks, and extended on-screen display blocks. The screen display image processing method includes the following steps: performing motion compensation based on a first motion vector and a second motion vector to generate motion compensation data, wherein the first motion vector points to the previous image frame and the second motion vector points to the current image frame; generating zero motion data based on a zero motion vector, wherein the zero motion vector points to both the previous image frame and the current image frame, and the value of the zero motion vector is zero; and mixing the motion compensation data with a first weight and the zero motion data with a second weight to generate each interpolated block, and using this as the processed block; wherein when the number of pixels with on-screen display (OSD) data in the processed block is less than a first positive threshold and greater than a second positive threshold, the processed block is determined to be an extended on-screen display block, and the second weight of the extended on-screen display block is less than the second weight of the on-screen display block.

[0007] In one embodiment of the present invention, the sum of the first weight and the second weight is a fixed positive value.

[0008] In one embodiment of the present invention, when the number of pixels having data displayed on the screen is greater than or equal to a first positive threshold, the processed block is determined to be a block displayed on the screen.

[0009] In one embodiment of the present invention, when the number of pixels having data displayed on the screen is less than or equal to a second positive threshold, the processed block is determined to be a non-displayed block on the screen, and the second weight of the processed block is zero.

[0010] In one embodiment of the present invention, motion compensation data is obtained by mixing first data of the previous image frame pointed to by the first motion vector of the processed block, second data of the current image frame pointed to by the second motion vector of the processed block, and zero motion data. The first data, the second data, and the corresponding zero motion data have third weights, fourth weights, and fifth weights, respectively.

[0011] In one embodiment of the present invention, the sum of the third weight, the fourth weight and the fifth weight is a fixed positive value.

[0012] In one embodiment of the present invention, a previous image frame has a first block, and the processed block is directed to the first block as a first reference block using a first motion vector. When the first reference block is determined to be an extended screen display block and the processed block is determined to be a non-screen display block, the third weight of the processed block is reduced.

[0013] In one embodiment of the present invention, the current image frame has a second block, the processed block is directed to the second block as the second reference block using a second motion vector, when the second reference block is determined to be an extended screen display block and the processed block is determined to be a non-screen display block, the fourth weight of the processed block is reduced.

[0014] In one embodiment of the present invention, a previous image frame has a first block, and the processed block is directed to the first block, which serves as a first reference block, using a first motion vector. When the first reference block and the processed block are determined to be blocks displayed on an extended screen, a third weight is added to the processed block.

[0015] In one embodiment of the present invention, the current image frame has a second block, the processed block is pointed to the second block as the second reference block using a second motion vector, and when the second reference block and the processed block are determined to be blocks to be displayed on the extended screen, a fourth weight is added to the processed block.

[0016] In one embodiment of the present invention, when the processed block is determined to be a block displayed on the extended screen, the second weight of the block displayed on the extended screen is greater than zero.

[0017] Based on the above, the new definition of the on-screen image processing method is an extended on-screen display block that is different from the on-screen display block and the non-on-screen display block, so as to greatly reduce the halo phenomenon, reduce the degree of fragmentation of the boundary of the on-screen display imaging area, and avoid the fly-out phenomenon. Attached Figure Description

[0018] Figure 1 A schematic diagram of the imaging area displayed on a screen in the prior art.

[0019] Figure 2 A schematic diagram of the imaging area displayed on another screen in the prior art.

[0020] Figure 3 This is a schematic diagram of a previous image frame, an interpolated image frame, and a current image frame in the prior art.

[0021] Figure 4 This is a schematic diagram of a previous image frame, another interpolated image frame, and the current image frame in the prior art.

[0022] Figure 5 This is a schematic diagram of a motion estimation and motion compensation (MEMC) device according to an embodiment of the present invention.

[0023] Figure 6 This is a flowchart of a screen-displaying image processing method according to an embodiment of the present invention.

[0024] Figure 7This is a schematic diagram of a previous image frame, an interpolated image frame, and a current image frame according to an embodiment of the present invention.

[0025] [Symbol Explanation]

[0026] 1…Interpolated image frames

[0027] 1'...Previous image frame

[0028] 1”...Current image frame

[0029] 2…Motion estimation and motion compensation devices

[0030] 20…Motion Estimation Processor

[0031] 21…Motion-compensated image frame rate converter

[0032] F…interpolated image frame

[0033] F1…Previous image frame

[0034] F2… Current image frame

[0035] V1…First motion vector

[0036] V2…Second motion vector

[0037] Steps S10, S12, S14… Detailed Implementation

[0038] Embodiments of the present invention will be further explained below with reference to the accompanying drawings. Wherever possible, the same reference numerals represent the same or similar components in the drawings and description. In the drawings, shapes and thicknesses may be exaggerated for simplicity and convenience. It is understood that elements not specifically shown in the drawings or described in the description are forms known to those skilled in the art. Those skilled in the art can make various changes and modifications based on the content of this invention.

[0039] Unless otherwise specified, certain conditional clauses or words, such as "can," "could," "might," or "may," are generally intended to express features, elements, or steps that are present in the embodiments of this application, but may also be interpreted as features, elements, or steps that may not be necessary. In other embodiments, these features, elements, or steps may be unnecessary.

[0040] In the following description of "one embodiment" or "an embodiment," the term refers to a specific element, structure, or feature associated with at least one embodiment. Therefore, the multiple descriptions of "one embodiment" or "an embodiment" appearing in various places below do not refer to the same embodiment. Furthermore, specific components, structures, and features in one or more embodiments may be combined in a suitable manner.

[0041] Certain terms are used in the specification and claims to refer to specific elements. However, those skilled in the art will understand that the same element may be referred to by different names. The specification and claims do not distinguish elements by differences in name, but by differences in function. The word "comprising" in the specification and claims is an open-ended term and should be interpreted as "including but not limited to". In addition, "coupled" here includes any direct and indirect connection means. Therefore, if the text describes a first element coupled to a second element, it means that the first element can be directly connected to the second element through electrical connection or signal connection methods such as wireless transmission or optical transmission, or indirectly electrically or signal connected to the second element through other elements or connection means.

[0042] The disclosure is specifically described by the following examples, which are merely illustrative. Various modifications and refinements can be made by those skilled in the art without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure is determined by the appended claims. Throughout the specification and claims, unless explicitly stated otherwise, the words “a” and “described” mean that such a statement includes “a or at least one” of the stated elements or components. Furthermore, as used in this disclosure, the singular article also includes a statement of a plurality of elements or components unless it is clearly apparent from the specific context that a plurality is excluded. Moreover, when applied in this description and in all the claims below, unless explicitly stated otherwise, “in which” means both “in which” and “therein”. The terms used throughout the specification and claims, unless otherwise specified, generally have their ordinary meaning in the art, in the content of this disclosure, and in the specific context. Certain terms used to describe this disclosure will be discussed below or elsewhere in this specification to provide additional guidance to a practitioner in describing this disclosure. Examples throughout this specification, including examples of any terms discussed herein, are for illustrative purposes only and do not, of course, limit the scope or meaning of this disclosure or any of the illustrative terms. Similarly, this disclosure is not limited to the various embodiments set forth in this specification.

[0043] The following description will provide a screen display image processing method, which has a new definition of extended screen display blocks that are different from screen display blocks and non-screen display blocks, in order to significantly reduce halos and reduce the degree of fragmentation of the boundaries of the screen display imaging area, and also avoid fly-out phenomena.

[0044] Figure 5This is a schematic diagram of a motion estimation and motion compensation (MEMC) device 2 according to an embodiment of the present invention. Figure 6 This is a flowchart of a screen-displaying image processing method according to an embodiment of the present invention. Figure 7 This is a schematic diagram of a previous image frame F1, an interpolated image frame F, and a current image frame F2, according to an embodiment of the present invention. Please refer to [link / reference]. Figure 5 The motion estimation and motion compensation device 2 includes a motion estimation processor 20 and a motion compensation image frame rate converter 21, with the motion estimation processor 20 coupled to the motion compensation image frame rate converter 21. Figure 5 The architecture can be applied to the following embodiments, but the present invention is not limited to these embodiments. Figure 5 2. Motion estimation and motion compensation device.

[0045] Please see Figure 5 , Figure 6 and Figure 7 The on-screen image processing method can be, but is not limited to, executed by the motion estimation and motion compensation device 2 or a processor. In the following embodiment, the on-screen image processing method is executed by the motion estimation and motion compensation device 2. The on-screen image processing method generates an interpolated image frame F from a previous image frame F1 and a current image frame F2. The interpolated image frame F includes multiple interpolation blocks, all of which have on-screen display (OSD) blocks, non-on-screen display (non-OSD) blocks, and extended on-screen display (extended OSD) blocks. Because the on-screen display blocks are static, the types of OSD blocks (on-screen display blocks, non-on-screen display blocks, extended on-screen display blocks) are the same at corresponding positions in the previous image frame F1, the current image frame F2, and the interpolated image frame F. In other words, before generating the interpolated image frame F, in most static cases, the on-screen display (OSD) data of the previous image frame F1 and the current image frame F2 are the same.

[0046] In step S10, the motion estimation and motion compensation device 2 sequentially receives the previous image frame F1 and the current image frame F2 on the time axis, and performs motion compensation based on the first motion vector V1 and the second motion vector V2 to generate motion compensation data MC, wherein the first motion vector V1 points to the previous image frame F1 and the second motion vector V2 points to the current image frame F2. In step S12, the motion estimation and motion compensation device 2 generates zero motion data Z based on the zero motion vector, wherein the zero motion vector points to the previous image frame F1 and the current image frame F2, and the value of the zero motion vector is zero. That is, the positions of the previous image frame F1, the current image frame F2, and the interpolated image frame F corresponding to the zero motion vector are the same. In step S14, the motion estimation and motion compensation device 2 mixes the motion compensation data MC with the first weight W1 and the zero motion data Z with the second weight W2 to generate each interpolation block, and uses this as the processed block, thereby interpolating the interpolated image frame F between the previous image frame F1 and the current image frame F2. Assuming substantially the same result is obtained, the steps of the image processing method displayed on the screen do not need to be performed in the exact order shown, nor do they need to be sequential; that is, other steps can be inserted. The processed block has on-screen display (OSD) data or non-on-screen display (non-OSD) data. On-screen display data is static data, such as logos, titles, captions, or non-video data, but the invention is not limited thereto. Non-on-screen display data is non-static data, such as video data. The interpolated image frame F has an on-screen display (OSD) imaging area. Some processed blocks are located on the on-screen display imaging area.

[0047] In one embodiment, motion compensation data MC and zero motion data can be mixed according to mathematical relation (1) or mathematical relations (1) and (2). In mathematical relation (1), the result of the operation MC×W1+Z×W2 is defined as the interpolated image data of the processed block.

[0048] MC×W1+Z×W2 (1)

[0049] MC=D1×W3+D2×W4+Z×W5 (2)

[0050] In mathematical relation (1), the sum of the first weight W1 and the second weight W2 can be, but is not limited to, 100% as a fixed positive value. Figure 7This is a schematic diagram of a previous image frame F1, an interpolated image frame F, and a current image frame F2 according to an embodiment of the present invention. When the number of pixels with on-screen display (OSD) data in the processed block is greater than or equal to a first positive threshold, the motion estimation and motion compensation device 2 determines that the processed block is an on-screen display block. When the number of pixels with on-screen display data in the processed block is less than the first positive threshold but greater than a second positive threshold, the motion estimation and motion compensation device 2 determines that the processed block is an extended on-screen display block, and adjusts the second weight W2 of the extended on-screen display block to be less than the second weight W2 of the on-screen display block, wherein the second positive threshold is less than the first positive threshold. When the processed block is determined to be an extended on-screen display block or an on-screen display block, the second weight W2 of the processed block can be greater than zero. Because the block located at the boundary of the on-screen display imaging area is determined to be an extended on-screen display block, the halo effect at the boundary of the on-screen display imaging area can be reduced, and the on-screen display imaging area can be protected. Furthermore, when the number of pixels with screen-display data in the processed block is less than or equal to the second positive threshold, the motion estimation and motion compensation device 2 determines that the processed block is a non-screen-display block and adjusts the second weight W2 of the processed block to zero.

[0051] In mathematical expressions (1) and (2), the sum of the third weight W3, the fourth weight W4, and the fifth weight W5 can be, but is not limited to, 100% as a fixed positive value. In another embodiment, motion compensation data MC can be obtained by mixing the first data D1 of the previous image frame F1 pointed to by the first motion vector V1 of the processed block, the second data D2 of the current image frame F2 pointed to by the second motion vector V2 of the processed block, and zero motion data Z, wherein the first data D1, the second data D2, and their corresponding zero motion data Z have the third weight W3, the fourth weight W4, and the fifth weight W5, respectively.

[0052] Specifically, the previous image frame F1 had a first block, and the processed block was pointed to by a first motion vector V1, which served as a first reference block. When the number of pixels with on-screen display data in the first reference block was greater than or equal to a first positive threshold, the motion estimation and motion compensation device 2 determined that the first reference block was an on-screen display block. When the number of pixels with on-screen display data in the first reference block was less than the first positive threshold but greater than a second positive threshold, the motion estimation and motion compensation device 2 determined that the first reference block was an extended on-screen display block. When the number of pixels with on-screen display data in the first reference block was less than or equal to the second positive threshold, the motion estimation and motion compensation device 2 determined that the first reference block was a non-on-screen display block. Currently, the image frame F2 has a second block, and the processed block was pointed to by a second motion vector V2, which served as a second reference block. When the number of pixels with on-screen display data in the second reference block was greater than or equal to the first positive threshold, the motion estimation and motion compensation device 2 determined that the second reference block was an on-screen display block. When the number of pixels in the second reference block that contain data to be displayed on the screen is less than a first positive threshold but greater than a second positive threshold, the motion estimation and motion compensation device 2 determines that the second reference block is an extended on-screen display block. When the number of pixels in the second reference block that contain data to be displayed on the screen is less than or equal to the second positive threshold, the motion estimation and motion compensation device 2 determines that the second reference block is a non-on-screen display block.

[0053] When the motion estimation and motion compensation device 2 determines that the first reference block and the processed block are extended screen display blocks, the motion estimation and motion compensation device 2 increases the third weight W3 of the processed block. Similarly, when the motion estimation and motion compensation device 2 determines that the second reference block and the processed block are extended screen display blocks, the motion estimation and motion compensation device 2 increases the fourth weight W4 of the processed block. Therefore, the halo phenomenon at the boundary of the screen display imaging area determined to be an extended screen display block is weakened.

[0054] When the motion estimation and motion compensation device 2 determines that the first reference block is an extended-screen display block and that the processed block is a non-screen display block, the motion estimation and motion compensation device 2 reduces the third weight W3 of the processed block. Similarly, when the motion estimation and motion compensation device 2 determines that the second reference block is an extended-screen display block and that the processed block is a non-screen display block, the motion estimation and motion compensation device 2 reduces the fourth weight W4 of the processed block. Therefore, the interpolated image frame F can avoid fly-out.

[0055] According to the above embodiments, the new on-screen image processing method defines an extended on-screen display block that is different from the on-screen display block and the non-on-screen display block, so as to greatly reduce the halo phenomenon, reduce the degree of fragmentation of the boundary of the on-screen display imaging area, and also avoid the flying-out phenomenon.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. Therefore, all equivalent variations and modifications made in accordance with the shape, structure, features and spirit described in the claims of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A screen display image processing method, which generates an interpolated image frame from a previous image frame and a current image frame, the interpolated image frame including a plurality of interpolation blocks, the plurality of interpolation blocks having on-screen display (OSD) blocks, non-on-screen display (non-OSD) blocks and extended on-screen display (extended OSD) blocks, the screen display image processing method comprising the following steps: Motion compensation is performed based on a first motion vector and a second motion vector to generate motion compensation data, wherein the first motion vector points to the previous image frame and the second motion vector points to the current image frame; Zero motion data is generated based on the zero motion vector, wherein the zero motion vector points to the previous image frame and the current image frame, and the value of the zero motion vector is zero; as well as The motion compensation data with a first weight is mixed with the zero motion data with a second weight to generate each interpolation block, which is then used as the processed block. When the number of pixels with on-screen display (OSD) data in the processed block is less than a first positive threshold and greater than a second positive threshold, the processed block is determined to be the extended on-screen display block, and the second weight of the extended on-screen display block is less than the second weight of the on-screen display block.

2. The image processing method for displaying an image on the screen as described in claim 1, wherein the sum of the first weight and the second weight is a fixed positive value.

3. The on-screen image processing method as claimed in claim 1, wherein when the number of pixels having the on-screen display data is greater than or equal to the first positive threshold, the processed block is determined to be the on-screen display block.

4. The on-screen image processing method as claimed in claim 1, wherein when the number of pixels having the on-screen display data is less than or equal to the second positive threshold, the processed block is determined to be the non-on-screen display block, and the second weight of the processed block is zero.

5. The on-screen image processing method as claimed in claim 1, wherein the motion compensation data is obtained by mixing the first data of the previous image frame pointed to by the first motion vector of the processed block, the second data of the current image frame pointed to by the second motion vector of the processed block, and the zero motion data, wherein the first data, the second data, and the corresponding zero motion data have a third weight, a fourth weight, and a fifth weight, respectively.

6. The on-screen image processing method as described in claim 5, wherein the sum of the third weight, the fourth weight, and the fifth weight is a fixed positive value.

7. The on-screen image processing method of claim 5, wherein the previous image frame has a first block, the processed block is directed to the first block as a first reference block using the first motion vector, and when the first reference block is determined to be the extended on-screen display block and the processed block is determined to be the non-on-screen display block, the third weight of the processed block is reduced.

8. The on-screen image processing method of claim 5, wherein the current image frame has a second block, the processed block is directed to the second block as a second reference block using the second motion vector, and when the second reference block is determined to be the extended on-screen display block and the processed block is determined to be the non-on-screen display block, the fourth weight of the processed block is reduced.

9. The on-screen image processing method of claim 5, wherein the previous image frame has a first block, the processed block is directed to the first block as a first reference block using the first motion vector, and when the first reference block and the processed block are determined to be the extended on-screen display blocks, the third weight of the processed block is increased.

10. The on-screen image processing method of claim 5, wherein the current image frame has a second block, the processed block is directed to the second block as a second reference block using the second motion vector, and when the second reference block and the processed block are determined to be the extended on-screen display blocks, the fourth weight of the processed block is increased.

11. The screen display image processing method as claimed in claim 1, wherein when the processed block is determined to be the extended screen display block, the second weight of the extended screen display block is greater than zero.

Images