Image processing method, display system and device

Abstract

The application discloses an image processing method, a display system and a device, and relates to the field of image processing, and comprises the following steps: acquiring image data, wherein the image data comprises images of continuous odd frames and even frames, and the original resolution of the images is a preset size; performing downsampling processing on the odd frames to obtain an odd sampling point set, and performing downsampling processing on the even frames to obtain an even sampling point set; performing upsampling processing on the odd sampling point set and the even sampling point set respectively to reconstruct images with the original resolution, wherein the upsampling processing comprises calculating the pixel value of an unsampled missing pixel through directional interpolation; and outputting the reconstructed images after the upsampling processing. The application adopts an odd-even frame interleaved sampling mode, reduces the number of pixels processed per frame, reduces the data transmission load and processing cost, and meanwhile, the sampling points of two continuous frames are complementary in space, so that a more complete information basis is reserved for subsequent high-quality image reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and particularly relates to an image processing method, display system and device. Background Technology

[0002] In image and video processing, it is often necessary to reduce data transmission volume or processing load while maintaining image quality. Common downsampling methods include uniform downsampling and region averaging, but these methods often lead to loss of image details, blurred edges, or jagged edges during reconstruction. Especially for continuous frame images (such as video sequences), traditional intra-frame interpolation methods often do not fully utilize inter-frame and spatial information, resulting in artifacts or texture degradation in moving areas.

[0003] Furthermore, while existing interpolation algorithms such as bilinear interpolation and bicubic interpolation are computationally simple, they lack directional adaptability, often resulting in unclear interpolation results for images with rich edges and textures. Therefore, an image processing method is needed that can combine inter-frame sampling differences with spatial orientation features to achieve high-quality image reconstruction while reducing the sampling rate. Summary of the Invention

[0004] The purpose of this invention is to solve the problems in the prior art, and to propose an image processing method, display system and device.

[0005] In a first aspect, the present invention proposes an image processing method, comprising the following steps: Acquire image data of consecutive frames, including images of odd-numbered frames and images of even-numbered frames, wherein the original resolution of the images is a preset size; The odd-numbered frames are downsampled to obtain a set of odd-numbered sampling points, and the even-numbered frames are downsampled to obtain a set of even-numbered sampling points, wherein: The set of odd sampling points includes sampling all pixels in the image of odd-numbered frames that are located in odd-numbered columns and odd-numbered rows; The even-numbered sampling point set includes sampling all pixels in the image of even-numbered frames that are located in even-numbered columns and even-numbered rows; Upsampling is performed on the odd-numbered sampling point set and the even-numbered sampling point set respectively to reconstruct an image at the original resolution. The upsampling process includes calculating the pixel values ​​of missing pixels that were not sampled during downsampling by directional interpolation. Output the reconstructed image based on the upsampling process; The process of calculating missing pixels using directional interpolation includes: (1) Define the pixel value of the outermost pixel located outside the image edge; (2) Treat the sampled pixels and outer pixels as known pixels; (3) Determine the neighboring pixels of the target missing pixel and define multiple interpolation directions. Each interpolation direction corresponds to multiple sets of known pixels located in that direction, and each interpolation direction corresponds to a pixel calculation direction, which passes through the target missing pixel. (4) Calculate the pixel gradient value for each interpolation direction: the pixel gradient value is the sum of the absolute values ​​of the differences between multiple known pixel values ​​in the interpolation direction, and then divided by the number of pixel groups in the interpolation direction; (5) Calculate the direct gradient value of the pixel calculation direction corresponding to each interpolation direction: the direct gradient value is the absolute value of the difference between two known pixel values ​​located on both sides of the target missing pixel in the pixel calculation direction; (6) Calculate candidate pixel values ​​for each pixel in a direction, wherein the candidate pixel values ​​are the average of the known pixel values ​​located on both sides of the target missing pixel in that direction; (7) By comparing the pixel gradient value and the direct gradient value, the optimal interpolation direction and the corresponding pixel calculation direction are selected, and the candidate pixel value in the direction is used as the final pixel value of the target missing pixel. In the above processing method, the directional interpolation calculation step in odd-numbered frames includes: Calculate the missing pixels in odd-numbered rows row by row / column by column; Calculate the missing pixels in even-numbered rows row by row / column; Furthermore, in even-numbered frames, the directional interpolation calculation step includes: Calculate the missing pixels in even-numbered rows row by row / column; Calculate the missing pixels in odd-numbered rows, row by row and column by column.

[0006] In the above processing method, when calculating the target missing pixel in an odd-numbered row of an odd-numbered frame or an even-numbered row of an even-numbered frame, the six nearest known pixels in six directions (northwest, northeast, west, east, southwest, and southeast) are selected as adjacent pixels, with the target missing pixel as the center. The interpolation directions of the target missing pixel include: The first direction corresponds to two sets of pixels in the "northeast-west" direction and the "east-southwest" direction. The calculation direction of the pixels in the first direction is the "northeast-southwest" direction. The second direction corresponds to three groups of pixels in the "northeast-northwest", "east-west" and "southeast-southwest" directions. The calculation direction of the pixels corresponding to the second direction is the "east-west" direction. The third direction corresponds to two sets of pixels in the "east-northwest" and "southeast-west" directions, and the pixel calculation direction corresponding to the third direction is the "southeast-northwest" direction.

[0007] In the above processing method, when calculating the target missing pixel in the even rows of odd frames or the odd rows of even frames, taking the target missing pixel as the center, the nearest six known pixels are selected as adjacent pixels in six directions of northwest, north, northeast, southwest, south and southeast, and the interpolation directions of the target missing pixel include: a fourth direction, corresponding to two sets of pixels in the "southeast-north" direction and the "south-northwest" direction, wherein the pixel calculation direction corresponding to the fourth direction is the southeast-northwest direction; a fifth direction, corresponding to three sets of pixels in the "southwest-northwest", "south-north" and "southeast-northeast" directions, wherein the pixel calculation direction corresponding to the fifth direction is the south-north direction; a sixth direction, corresponding to two sets of pixels in the "northeast-south" direction and the "north-southwest" direction, wherein the pixel calculation direction corresponding to the sixth direction is the "northeast-southwest" direction; In the above processing method, the defining of the pixel values of the outer pixels located outside the edge of the image includes: The pixel value of the outer pixel is the same as the value of the sampling pixel adjacent thereto in the row direction or the column direction.

[0008] In the above processing method, in the calculation of the target missing pixels in the odd rows of odd frames and the even rows of even frames, the value of the target missing pixel specifically includes: If A1<A2 and A1<A3, and; If B1<B2, the candidate pixel value corresponding to the first direction is selected; If B1>B2, the candidate pixel value corresponding to the second direction is selected; Otherwise, if A3<A2 and A3<A1, and; If B3<B2, the candidate pixel value corresponding to the third direction is selected; If B3>B2, the candidate pixel value corresponding to the second direction is selected; In other cases, the candidate pixel value corresponding to the second direction is selected; Wherein A1, A2 and A3 are the pixel gradient values of the first direction, the second direction and the third direction respectively; B1, B2 and B3 are the direct gradient values in the pixel calculation directions corresponding to the first direction, the second direction and the third direction respectively.

[0009] In the above processing method, in the calculation of the target missing pixels in the even rows of odd frames and the odd rows of even frames, If A4<A5, A4<A6, and; If B4<B5, the candidate pixel value corresponding to the fourth direction is selected; If B4>B5, the candidate pixel value corresponding to the fifth direction is selected; Otherwise, if A6<A5 and A6<A4, and; If B6<B5, select the candidate pixel value corresponding to the sixth direction; If B6>B5, select the candidate pixel value corresponding to the fifth direction; In other cases, select the candidate pixel value corresponding to the fifth direction; wherein A4, A5, and A6 are the pixel gradient values of the fourth direction, the fifth direction, and the sixth direction respectively; B4, B5 and B6 are the direct gradient values in the calculation direction of the pixels corresponding to the fourth direction, the fifth direction, and the sixth direction respectively.

[0010] In the above processing method, during the display process, the even frames are spatially offset and displayed relative to the odd frames, or the odd frames are spatially offset and displayed relative to the even frames, which specifically includes: The offset display offsets adjacent odd sampling points and even sampling points in the odd frames and even frames in directions away from each other by at least one pixel, so as to optimize the display transition of adjacent sampling points; In the above processing method, when outputting the reconstructed image obtained after upsampling processing, the rows and columns that are offset beyond the display area in the odd frames or even frames are cropped according to the offset display direction; In a second aspect, the present invention provides a display system, which adopts the above image processing method, and comprises: a data receiving module, configured to receive input continuous frame image data; a downsampling module, configured to perform downsampling on the input continuous frame image data; an upsampling module, configured to perform upsampling processing on the odd sampling point set and the even sampling point set to reconstruct an image with the original resolution; an image output module, configured to output the image processed by the upsampling module; In the above display system, the downsampling module respectively generates the odd sampling point set and the even sampling point set in the odd frames and the even frames, including: The odd sampling point set includes sampling all pixels located in odd columns and odd rows in the image of the odd frames; The even sampling point set includes sampling all pixels located in even columns and even rows in the image of the even frames.

[0011] In the above display system, the upsampling module is configured to reconstruct the downsampled image into an image with the original resolution, including: The pixel values of missing pixels that are not sampled in downsampling are calculated through directional interpolation, wherein the process of calculating the missing pixels through directional interpolation includes: (1) defining the pixel values of outer pixels located outside the edge of the image; (2) Treat the sampled pixels and outer pixels as known pixels; (3) Determine the neighboring pixels of the target missing pixel and define multiple interpolation directions. Each direction corresponds to multiple sets of pixels. Each interpolation direction corresponds to a pixel calculation direction. The pixel calculation direction passes through the target missing pixel. (4) Calculate the pixel gradient value for each interpolation direction, wherein the pixel gradient value is obtained by summing the absolute values ​​of the differences between two known pixel values ​​in multiple groups located in that direction and dividing by the number of pixel groups in that interpolation direction; (5) Calculate the direct gradient value of the pixel calculation direction corresponding to each interpolation direction, wherein the direct gradient value is the absolute value of the difference between two known pixel values ​​located on both sides of the target missing pixel in the pixel calculation direction; (6) Calculate candidate pixel values ​​for each pixel in a direction, wherein the candidate pixel values ​​are the average of the known pixel values ​​located on both sides of the target missing pixel in that direction; (7) By comparing the pixel gradient value and the direct gradient value, the optimal interpolation direction and the corresponding pixel calculation direction are selected, and the candidate pixel value in this direction is used as the final pixel value of the target missing pixel.

[0012] In the above display system, the image output module outputs the upsampled odd-numbered frame images and even-numbered frame images, including: The even-numbered frames are spatially offset relative to the odd-numbered frames or the odd-numbered frames are spatially offset relative to the even-numbered frames. Specifically, the offset display shifts adjacent odd and even sampling points in the odd and even frames by at least one pixel in a direction that moves them away from each other, in order to optimize the display transition of neighboring sampling points. Thirdly, the present invention provides a display device comprising the above-described display system.

[0013] The beneficial effects of this invention are as follows: 1. This invention employs an alternating sampling method for odd and even frames, which reduces the number of pixels processed per frame, significantly lowering data transmission load and processing overhead. At the same time, the sampling points of two consecutive frames are spatially complementary, preserving a more complete information foundation for subsequent high-quality image reconstruction.

[0014] 2. By making a slight relative offset between odd and even frame images during display, the sampling points of adjacent frames are appropriately spaced apart on the screen. This effectively reduces visual flickering and grid-like effects that may occur due to alternating sampling points, thereby improving the stability and smoothness of the image display.

[0015] 3. This invention employs an adaptive directional interpolation algorithm to reconstruct missing pixels. This algorithm automatically selects the most reasonable texture or edge extension direction for calculation by comparing pixel gradients in multiple directions. This better preserves edge sharpness, reduces blur and jagged edges, and obtains clearer details when reconstructing images. Attached Figure Description

[0016] Figure 1 This is a flowchart of an image processing method disclosed in this invention; Figure 2 This is a schematic diagram illustrating the principle of downsampling processing in an image processing method disclosed in this invention. Figure 3 This invention discloses an image processing method in which odd-numbered frames are offset relative to even-numbered frames, resulting in a distance-away image. Figure 4 This is a schematic diagram illustrating the calculation principle of missing pixels in odd-numbered frames and odd-numbered rows in an image processing method disclosed in this invention. Figure 5 This is a schematic diagram illustrating the calculation principle of missing pixels in even-numbered rows of odd-numbered frames in an image processing method disclosed in this invention. Figure 6 This is a schematic diagram illustrating the calculation principle of missing pixels in even-numbered rows of even-numbered frames in an image processing method disclosed in this invention. Figure 7 This is a schematic diagram illustrating the calculation principle of missing pixels in odd rows of even frames in an image processing method disclosed in this invention. Detailed Implementation To facilitate understanding of this application and to make the aforementioned objectives, features, and advantages of this application more apparent, a detailed description of specific embodiments of this application is provided below in conjunction with the accompanying drawings. Numerous specific details are set forth in the following description to provide a thorough understanding of this application, and preferred embodiments are shown in the accompanying drawings. However, this application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application. This application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified. In the description of this application, "several" means at least one, such as one, two, etc., unless otherwise explicitly specified. It should be noted that when an element is referred to as being "fixed to" another element, it can be directly attached to the other element or there may be an intervening element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementations. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is only for describing particular implementations and is not intended to limit the scope of this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0017] Reference Figures 1-7 This invention provides an image processing method, comprising the following steps: S1: Acquire image data of consecutive frames, which includes odd-numbered frame images and even-numbered frame images, with their original resolution being a preset size; such consecutive frames usually originate from video streams or real-time shooting systems, and each frame is closely adjacent in the time dimension and has a high content correlation, which provides favorable conditions for subsequent inter-frame sampling and reconstruction processing.

[0018] S2: Downsampling is performed on the odd-numbered frames to obtain a set of odd-numbered sampling points; downsampling is performed on the even-numbered frames to obtain a set of even-numbered sampling points. Downsampling is used to selectively extract some pixels from the original image, reducing the amount of data while preserving the main structural information of the image. Specifically, the set of odd-numbered sampling points includes all pixel samples located in odd-numbered columns and odd-numbered rows in the odd-numbered frame image; similarly, the set of even-numbered sampling points includes all pixel samples located in even-numbered columns and even-numbered rows in the even-numbered frame image. This interleaved sampling strategy can form complementary pixel coverage patterns between consecutive frames, thus laying the foundation for accurately restoring image details in the subsequent reconstruction stage.

[0019] S3: Perform upsampling processing on the odd-numbered and even-numbered sampled point sets respectively to reconstruct the original resolution image. The upsampling process calculates the values ​​of unsampled missing pixels in the downsampling through directional interpolation. This process aims to refill sparse pixels into a complete image. The directional interpolation method can adaptively select the optimal interpolation path based on the local structural features of the image, thereby more effectively preserving the edge contours and texture details of the image.

[0020] S4: Outputs a reconstructed image based on upsampling, with the same resolution as the original image and optimized visual effects, suitable for display, storage, or further image analysis tasks.

[0021] The process of calculating missing pixels using directional interpolation includes: (1) Define the pixel value of the outer edge of the image: usually, the boundary copy or extrapolation method is used to virtually define the edge region to ensure that the edge region can be effectively interpolated.

[0022] (2) The sampled pixels and the outer pixels are used as known pixels to form the reference points for the interpolation process.

[0023] (3) Determine the neighboring pixels of the target missing pixel and define multiple interpolation directions: each direction corresponds to multiple sets of known pixels located in that direction, and set the pixel calculation direction that passes through the target missing pixel. Common directions include horizontal, vertical and diagonal lines, which are used to capture structural features with different orientations.

[0024] (4) Calculate the pixel gradient value in each interpolation direction: Take the sum of the absolute values ​​of the differences between multiple sets of known pixel values ​​in that direction, and then divide by the number of pixel sets. The gradient value intuitively reflects the degree of change of pixel values ​​in that direction. The smaller the gradient value, the higher the pixel consistency in that direction, and therefore the more suitable it is as a reference for interpolation calculation.

[0025] (5) Calculate the direct gradient value of each pixel in the calculation direction: take the absolute value of the difference between the two known pixel values ​​on both sides of the missing pixel in the direction, and use it to evaluate the local continuity.

[0026] (6) Calculate candidate pixel values ​​for each pixel in the direction of calculation: take the average value of the known pixel values ​​on both sides of the missing pixel in the direction as the preliminary estimate.

[0027] (7) Select the optimal interpolation direction and the corresponding pixel calculation direction by comparing the pixel gradient value with the direct gradient value, and use the candidate pixel value as the final value of the target missing pixel. Usually, the direction with the smallest gradient is selected so that the interpolation result conforms to the local structural features and avoids edge blurring or distortion.

[0028] In a feasible embodiment, the directional interpolation step for odd-numbered frames includes: Calculate the missing pixels in odd-numbered rows row by row / column by column; Calculate the missing pixels in even-numbered rows row by row / column.

[0029] The directional interpolation steps for even-numbered frames include: Calculate missing pixels in even-numbered rows row by row / column; Calculate the missing pixels in odd-numbered rows row by row / column by column.

[0030] The advantage of this strategy is that, since the sampled pixels in odd-numbered frames are located in odd-numbered rows and columns, the odd-numbered rows or columns have the most known pixels, while the pixels in even-numbered rows or columns are all unknown. Prioritizing the calculation of missing pixels in odd-numbered rows or columns, and then using these calculated pixels as known values ​​to deduce the unknown pixels in even-numbered rows or columns, significantly improves the accuracy of pixel value calculation. Similarly, since the sampled pixels in even-numbered frames are located in even-numbered columns / rows, the missing pixels in even-numbered rows / columns must be calculated first, before processing the unknown pixels in odd-numbered rows / columns.

[0031] In a feasible embodiment, when calculating the target missing pixels in odd rows of odd-numbered frames or even rows of even-numbered frames, the nearest known pixels in six directions—northwest, northeast, west, east, southwest, and southeast—are selected as adjacent pixels, with the missing pixels as the center. The interpolation directions include: First direction: Corresponds to two groups of pixels in the "Northeast-West" direction and the "East-Southwest" direction, with the pixel calculation direction being "Northeast-Southwest"; The second direction corresponds to three groups of pixels in the "northeast-northwest", "east-west" and "southeast-southwest" directions, with the pixel calculation direction being "east-west". The third direction corresponds to two groups of pixels in the "east-northwest" and "southeast-west" directions, with the pixel calculation direction being "southeast-northwest".

[0032] This scheme selects two diagonal directions and one horizontal direction for interpolation, and optimizes the pixel gradient results by comparing the three directions. The advantage of this design is that by utilizing multiple known pixels around the target missing pixel and calculating the average pixel gradient value based on multiple sets of pixel pairs, it can more accurately reflect the pixel change trend in the interpolation direction.

[0033] In a feasible embodiment, when calculating the target missing pixel at even rows of odd frames or odd rows of even frames, taking the target missing pixel as the center, the nearest known pixels in six directions of northwest, north, northeast, southwest, south, and southeast are selected, and the interpolation directions include: The fourth direction: corresponding to two groups of pixels in the "southeast-north" direction and the "south-northwest" direction, and the pixel calculation direction is "southeast-northwest"; The fifth direction: corresponding to three groups of pixels in the "southwest-northwest" direction, the "south-north" direction and the "southeast-northeast" direction, and the pixel calculation direction is "south-north"; The sixth direction: corresponding to two groups of pixels in the "northeast-south" direction and the "north-southwest" direction, and the pixel calculation direction is "northeast-southwest".

[0034] In a feasible embodiment, when defining the pixel values outside the image edge: the outer pixel value is the same as the adjacent sampled pixel value in its row / column direction. This is because the calculation of the target missing pixel needs to be performed row by row or column by column, and highly depends on the known pixel values — for example, in the calculation scenario of the second column of the first row of an odd frame, only two known pixels exist initially, and at this time, the outer pixel values must be included in the category of known values to participate in the operation.

[0035] In a feasible embodiment, the value rule for target missing pixels in odd rows of odd frames and even rows of even frames is: If A1<A2 and A1<A3: When B1<B2, the candidate value in the first direction is selected; When B1>B2, the candidate value in the second direction is selected; If A3<A2 and A3<A1: When B3<B2, the candidate value in the third direction is selected; When B3>B2, the candidate value in the second direction is selected; In other cases, the candidate value in the second direction is selected.

[0036] Wherein A1, A2, and A3 are the pixel gradient values in the first, second, and third directions respectively; B1, B2, and B3 are the direct gradient values of their corresponding pixel calculation directions.

[0037] This embodiment stipulates that: the calculation of missing pixels preferentially selects the direction with the smallest pixel gradient value (that is, the direction with the gentlest pixel change). If the direct gradient value of the pixel calculation direction corresponding to this direction is also the smallest, its candidate value is adopted; otherwise, the horizontal direction, that is, the second direction, is always selected as the default interpolation direction. The reason for this arrangement is that although the weights of all known pixels are theoretically set to be the same, in actual operation, the known pixels in the horizontal direction are closest to the target missing pixel, and their influence weight on the target value should be the largest. Therefore, only when both the gradient value and the direct gradient value of the oblique direction are the smallest, the farther pixels are used to calculate the target value.

[0038] In a feasible embodiment, in the calculation of target missing pixels at even lines of odd frames and odd lines of even frames, if A4<A5, A4<A6, and; if B4<B5, the candidate pixel value corresponding to the fourth direction is selected; if B4>B5, the candidate pixel value corresponding to the fifth direction is selected; otherwise, if A6<A5 and A6<A4; if B6<B5, the candidate pixel value corresponding to the sixth direction is selected; if B6>B5, the candidate pixel value corresponding to the fifth direction is selected; in other cases, the candidate pixel value corresponding to the fifth direction is selected; wherein A4, A5 and A6 are respectively the pixel gradient values of the fourth direction, the fifth direction and the sixth direction; B4, B5 and B6 are respectively the direct gradient values in the pixel calculation directions corresponding to the fourth direction, the fifth direction and the sixth direction.

[0039] Similarly, in this embodiment, the interpolation direction with the smallest pixel gradient value is preferentially selected for missing pixel calculation, and this interpolation direction represents the gentlest pixel change. During direct calculation, the pixel gradient values of the fourth direction and the sixth direction located in the oblique direction can be calculated first. If there exists an interpolation direction whose corresponding direct gradient value in the pixel calculation direction is also the smallest among the two directions, the candidate value thereof is adopted; otherwise, the vertical direction, that is, the fifth direction, is always selected as the default interpolation direction.

[0040] In a feasible embodiment, during the display process, the even frames are displayed with a spatial offset relative to the odd frames, or the odd frames are displayed with a spatial offset relative to the even frames, which specifically includes: The offset display shifts adjacent odd and even sampling points in odd and even frames by at least one pixel in a direction that moves them away from each other. This optimizes the display transition of nearby sampling points. The reason for this setting is to ensure that sampling points that were originally staggered in adjacent frames are appropriately moved apart during display, thereby optimizing the display transition between sampling points and reducing aliasing. Specifically, taking sampling points (1,1) and (2,2) as examples, in odd frames, the sampling point is (1,1), so the pixel values ​​of the surrounding pixels (1,2) and (2,1) are all referenced to the value of pixel (1,1). In even frames... The sampling point is (2,2), and the surrounding pixels (1,2) and (2,1) will also refer to the value of pixel (2,2). If the pixel values ​​of pixels (1,1) and (2,2) are significantly different, then from a display perspective, there will be a noticeable change between odd-numbered frames and even-numbered frames at pixels (1,2) and (2,1), which may result in flickering in the display. Therefore, by moving the two sampling points away from each other by at least one pixel in this scheme, a transition space can be created between the pixels around the two sampling points, thereby improving the display effect.

[0041] In one feasible embodiment, when outputting the reconstructed image based on upsampling processing, the rows and columns of odd or even frames that are offset beyond the display area are cropped according to the direction of the offset display. When outputting the reconstructed image by offset, due to the offset between odd and even frames, when one frame is used as the display area, another frame will have a column or row that exceeds the display area. In this case, the excess part needs to be cropped, such as... Figure 3 As shown; The present invention also discloses a display system employing the above-described image processing method, comprising: The data receiving module is used to receive input continuous frame image data; The downsampling module is used to downsample the input continuous frame image data; The upsampling module is used to perform upsampling processing on the odd-numbered sampling point set and the even-numbered sampling point set to reconstruct an image at the original resolution; The image output module is used to output the image processed by the upsampling module.

[0042] The downsampling module generates sets of odd sampling points and sets of even sampling points in odd and even frames, respectively, including: The set of odd sampling points includes sampling all pixels in the image of odd-numbered frames that are located in odd-numbered columns and odd-numbered rows; The even-numbered sampling point set includes sampling all pixels in the image of even-numbered frames that are located in even-numbered columns and even-numbered rows. The above sampling method is used to select different sampling points in odd-numbered frames and even-numbered frames respectively.

[0043] The upsampling module is used to reconstruct the downsampled image back to its original resolution, including: The pixel values ​​of missing pixels that were not sampled during downsampling are calculated using directional interpolation, wherein the process of calculating the missing pixels using directional interpolation includes: (1) Define the pixel value of the outermost pixel located outside the image edge; (2) Treat the sampled pixels and outer pixels as known pixels; (3) Determine the neighboring pixels of the target missing pixel and define multiple interpolation directions. Each direction corresponds to multiple sets of pixels. Each interpolation direction corresponds to a pixel calculation direction. The pixel calculation direction passes through the target missing pixel. (4) Calculate the pixel gradient value for each interpolation direction, wherein the pixel gradient value is obtained by summing the absolute values ​​of the differences between two known pixel values ​​in multiple groups located in that direction and dividing by the number of pixel groups in that interpolation direction; (5) Calculate the direct gradient value of the pixel calculation direction corresponding to each interpolation direction, wherein the direct gradient value is the absolute value of the difference between two known pixel values ​​located on both sides of the target missing pixel in the pixel calculation direction; (6) Calculate candidate pixel values ​​for each pixel in a direction, wherein the candidate pixel values ​​are the average of the known pixel values ​​located on both sides of the target missing pixel in that direction; (7) By comparing the pixel gradient value and the direct gradient value, the optimal interpolation direction and the corresponding pixel calculation direction are selected, and the candidate pixel value in this direction is used as the final pixel value of the target missing pixel.

[0044] The image output module outputs the upsampled odd-numbered frame images and even-numbered frame images, including: The even-numbered frames are spatially offset relative to the odd-numbered frames or the odd-numbered frames are spatially offset relative to the even-numbered frames. Specifically, the offset display shifts adjacent odd and even sampling points in the odd and even frames by at least one pixel in a direction that moves them away from each other, in order to optimize the display transition of neighboring sampling points.

[0045] In specific implementation, with Figure 4 For example, assuming the original resolution is 8×8, then firstly: define Z xy The value of the pixel in the Xth row and Yth column is where X or Y equals 0 or 9, which means that the pixel is outside the display area, i.e., the corresponding pixel is the outer pixel. Therefore, in the calculation of missing pixels in odd-numbered frames and odd-numbered rows, we can refer to... Figure 4 a and Figure 4 As shown in b; The interpolation directions are C1, C2, and C3. C1 includes two groups, namely the "southwest-east" and "west-northeast" directions of the target missing pixels. C2 includes three groups, namely the "southwest-southeast", "west-east", and "northwest-northeast" directions of the target missing pixels. C3 includes two groups, namely the "east-northwest" and "southeast-west" directions. The pixel calculation directions are D1, D2 and D3, where D1 is the "northeast-southwest" direction corresponding to C1; D2 corresponds to the "east-west" direction of C2; D3 corresponds to the "southeast-northwest" direction of C3.

[0046] Among them, pixel Z 12 Taking the calculation as an example, a search is performed around it, and the nearest known pixel searched in its six directions includes the Z pixel located in the northwest. 00 Z, located on the northeast side 03 Z, located on the west side 11 Z, located on the east side 13 Z, located on the southwest side 31 Z, located on the southeast side 33 Among them, Z 00 The value is equal to Z 11 Z 03 The value is equal to Z 13 At this point, the corresponding A1, A2, A3, B1, B2 and B3 values ​​are calculated for the six known pixels mentioned above. A1=(Z 13 -Z 31 +Z 03 -Z 11 ) / 2 A2=(Z 03 -Z 00 +Z 13 -Z 11 +Z 33 -Z 31 ) / 3 A3=(Z 11 -Z 33 +Z 03 -Z 13 ) / 2 B1=Z 31 -Z 03 B2=Z 13 -Z 11 B3=Z 33 -Z 00 Then, the interpolation direction is selected by comparing the values ​​of A1, A2, A3 and B1, B2, B3, and choosing the interpolation direction based on the comparison results. The pixel calculation direction corresponding to this interpolation direction is then used as the final pixel value calculation direction. For example, if both A1 and B1 are found to be the smallest, then the "northeast-southwest" direction is the final pixel value calculation direction. Finally, Z... 12 Pixel values ​​are: (Z) 31 +Z 13 ) / 2 Complete Z 12 After calculating the pixels, continue to complete the first row Z. 14 Z 16 Z 18 The remaining missing pixels are calculated, and then the pixels calculated in the first row are used as known pixels to calculate the missing pixels in the third row. After completing the calculation of the odd-numbered rows, the pixels in the even-numbered rows are calculated. The calculation for even-numbered rows can be found by referring to... Figure 5 a and Figure 5 b proceeds, with Z 21 Taking the calculation as an example, the interpolation directions are C4, C5 and C6. C4 includes two groups, which include the target missing pixels in the "southeast-north" direction and the "south-northwest" direction respectively. C5 includes three groups, which include the target missing pixels in the "southwest-northwest", "south-north" and "southeast-northeast" directions respectively. C6 includes two groups, in the "northeast-south" direction and the "north-southwest" direction. The pixel calculation directions are D4, D5, and D6, with D4 corresponding to the "southeast-northwest" direction of C4; D5 corresponds to the "South-North" direction of C5; D6 corresponds to the "northeast-southwest" direction of C6.

[0047] Among them, pixel Z 21 Taking the calculation as an example, a search is performed around it, and the nearest known pixel searched in its six directions includes the Z pixel located in the northwest. 10 Z, located on the north side 11 Z, located on the northeast side 12 Z, located on the southwest side 30 Z, located on the south side 31 Z, located on the southeast side 32 Among them, Z 10 The value is equal to Z 11 Z 30 The value is equal to Z 31 At this point, the corresponding A4, A5, A6, B4, B5, and B6 values ​​are calculated for the six known pixels mentioned above. A4=(Z31 -Z 10 +Z 32 -Z 11 ) / 2 A2=(Z 30 -Z 10 +Z 31 -Z 11 +Z 32 -Z 12 ) / 3 A3=(Z 30 -Z 11 +Z 31 -Z 12 ) / 2 B1=Z 32 -Z 10 B2=Z 31 -Z 11 B3=Z 30 -Z 12 Then, the interpolation direction is selected by comparing the values ​​of A4, A5, A6, and B4, B5, B6, and choosing the interpolation direction based on the comparison results. The pixel calculation direction corresponding to this interpolation direction is then used as the final pixel value calculation direction. For example, if both A4 and B4 values ​​are found to be the smallest, then the "southeast-northwest" direction is the final pixel value calculation direction. Finally, Z... 21 Pixel values ​​are: (Z) 32 +Z 01 ) / 2 Following this process, after calculating the missing pixels in the second row, the aforementioned pixel values ​​are used as known pixels in the calculation of the missing pixels in the fourth row, ultimately completing the calculation of the final pixel values ​​for all missing pixels in the 8×8 display image.

[0048] Reference Figure 6 a, Figure 6 b and Figure 7 a and Figure 7 b calculates the missing pixels in even-numbered frames, and finally restores the images in both odd-numbered and even-numbered frames, outputting the reconstructed image.

[0049] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative and not exhaustive. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. An image processing method, characterized in that, Includes the following steps: Acquire image data of consecutive frames, including images of odd-numbered frames and images of even-numbered frames, wherein the original resolution of the images is a preset size; The odd-numbered frames are downsampled to obtain a set of odd-numbered sampling points, and the even-numbered frames are downsampled to obtain a set of even-numbered sampling points, wherein: The set of odd sampling points includes sampling all pixels in the image of odd-numbered frames that are located in odd-numbered columns and odd-numbered rows; The even-numbered sampling point set includes sampling all pixels in the image of even-numbered frames that are located in even-numbered columns and even-numbered rows; Upsampling is performed on the odd-numbered sampling point set and the even-numbered sampling point set respectively to reconstruct an image at the original resolution. The upsampling process includes calculating the pixel values ​​of missing pixels that were not sampled during downsampling by directional interpolation. Output the reconstructed image based on the upsampling process; The process of calculating missing pixels using directional interpolation includes: (1) Define the pixel value of the outermost pixel located outside the image edge; (2) Treat the sampled pixels and outer pixels as known pixels; (3) Determine the neighboring pixels of the target missing pixel and define multiple interpolation directions. Each interpolation direction corresponds to multiple sets of known pixels located in that direction, and each interpolation direction corresponds to a pixel calculation direction, which passes through the target missing pixel. (4) Calculate the pixel gradient value for each interpolation direction: The pixel gradient value is the sum of the absolute values ​​of the differences between multiple known pixel values ​​in the interpolation direction, and then divided by the number of pixel groups in the interpolation direction. (5) Calculate the direct gradient value of the pixel calculation direction corresponding to each interpolation direction: the direct gradient value is the absolute value of the difference between the known pixel values ​​on both sides of the target missing pixel in the pixel calculation direction; (6) Calculate candidate pixel values ​​for each pixel in a direction, wherein the candidate pixel values ​​are the average of the known pixel values ​​located on both sides of the target missing pixel in that direction; (7) By comparing the pixel gradient value and the direct gradient value, the optimal interpolation direction and the corresponding pixel calculation direction are selected, and the candidate pixel value in the direction is used as the final pixel value of the target missing pixel.

2. The image processing method according to claim 1, characterized in that, In odd-numbered frames, the directional interpolation calculation steps include: Calculate the missing pixels in odd-numbered rows row by row / column by column; Calculate the missing pixels in even-numbered rows row by row / column; Furthermore, in even-numbered frames, the directional interpolation calculation step includes: Calculate the missing pixels in even-numbered rows row by row / column; Calculate the missing pixels in odd-numbered rows, row by row and column by column.

3. The image processing method according to claim 1, characterized in that, When calculating the target missing pixel in an odd-numbered row of an odd-numbered frame or an even-numbered row of an even-numbered frame, the six known pixels closest to the target missing pixel are selected as adjacent pixels in six directions: northwest, northeast, west, east, southwest, and southeast, with the target missing pixel as the center. The interpolation directions of the target missing pixel include: The first direction corresponds to two sets of pixels in the "northeast-west" direction and the "east-southwest" direction. The calculation direction of the pixels corresponding to the first direction is the "northeast-southwest" direction. the second direction, corresponding to three groups of pixels in the "northeast-northwest" direction, "east-west" direction, and "southeast-southwest" direction, and the pixel calculation direction corresponding to the second direction is the "east-west" direction; the third direction, corresponding to two groups of pixels in the "east-northwest" direction and the "southeast-west" direction, and the pixel calculation direction corresponding to the third direction is the "southeast-northwest" direction.

4. The image processing method according to claim 2, characterized in that, When calculating the target missing pixel in even rows of odd frames or in odd rows of even frames, taking the target missing pixel as the center, the six nearest known pixels are selected in the six directions of northwest, north, northeast, southwest, south, and southeast as adjacent pixels, and the interpolation directions of the target missing pixel include: the fourth direction, corresponding to two groups of pixels in the "southeast-north" direction and the "south-northwest" direction, and the pixel calculation direction corresponding to the fourth direction is the "southeast-northwest" direction; the fifth direction, corresponding to three groups of pixels in the "southwest-northwest" direction, "south-north" direction, and "southeast-northeast" direction, and the pixel calculation direction corresponding to the fifth direction is the "south-north" direction; the sixth direction, corresponding to two groups of pixels in the "northeast-south" direction and the "north-southwest" direction, and the pixel calculation direction corresponding to the sixth direction is the "northeast-southwest" direction.

5. The image processing method according to claim 1, characterized in that, The defining of the pixel values of the outer pixels located outside the edge of the image includes: The pixel value of the outer pixel is the same as the value of the sampling pixel adjacent thereto in the row direction or the column direction.

6. The image processing method according to claim 3, characterized in that, In the calculation of the target missing pixel in odd rows of odd frames and even rows of even frames, the value of the target missing pixel specifically includes: If A1<A2 and A1<A3, and; If B1<B2, the candidate pixel value corresponding to the first direction is selected; If B1>B2, the candidate pixel value corresponding to the second direction is selected; Otherwise, if A3<A2 and A3<A1, If B3<B2, the candidate pixel value corresponding to the third direction is selected; If B3>B2, the candidate pixel value corresponding to the second direction is selected; In other cases, the candidate pixel value corresponding to the second direction is selected; wherein, A1, A2, and A3 are respectively the pixel gradient values of the first direction, the second direction, and the third direction; B1, B2, and B3 are respectively the direct gradient values in the pixel calculation directions corresponding to the first direction, the second direction, and the third direction.

7. The image processing method according to claim 4, characterized in that, In the calculation of the target missing pixel in even rows of odd frames and odd rows of even frames, If A4<A5, A4<A6, and; If B4<B5, the candidate pixel value corresponding to the fourth direction is selected; If B4>B5, the candidate pixel value corresponding to the fifth direction is selected; Otherwise, if A6<A5 and A6<A4; If B6<B5, the candidate pixel value corresponding to the sixth direction is selected; If B6>B5, the candidate pixel value corresponding to the fifth direction is selected; In other cases, the candidate pixel value corresponding to the fifth direction is selected; wherein, A4, A5, and A6 are respectively the pixel gradient values of the fourth direction, the fifth direction, and the sixth direction; B4, B5, and B6 are respectively the direct gradient values in the pixel calculation directions corresponding to the fourth direction, the fifth direction, and the sixth direction.

8. The image processing method according to claim 1, characterized in that, In the display process, the even frames are spatially shifted for display relative to the odd frames, or the odd frames are spatially shifted for display relative to the even frames, which specifically includes: The offset display shifts adjacent odd and even sampling points in odd and even frames by at least one pixel in a direction that moves them away from each other, in order to optimize the display transition of neighboring sampling points.

9. The image processing method according to claim 1, characterized in that, When outputting the reconstructed image based on upsampling, the rows and columns of odd or even frames that are offset beyond the display area are cropped according to the direction of the offset display.

10. A display system, characterized in that, The image processing method described in any one of claims 1 to 9 includes: The data receiving module is used to receive input continuous frame image data; The downsampling module is used to downsample the input continuous frame image data; The upsampling module is used to perform upsampling processing on the odd-numbered sampling point set and the even-numbered sampling point set to reconstruct an image at the original resolution; The image output module is used to output the image processed by the upsampling module.

11. The display system according to claim 10, characterized in that, The downsampling module generates sets of odd sampling points and sets of even sampling points in odd and even frames, respectively, including: The set of odd sampling points includes sampling all pixels in the image of odd-numbered frames that are located in odd-numbered columns and odd-numbered rows; The even-numbered sampling point set includes sampling all pixels in the image of even-numbered frames that are located in even-numbered columns and even-numbered rows.

12. The display system according to claim 10, characterized in that, The upsampling module is used to reconstruct the downsampled image back to its original resolution, including: The pixel values ​​of missing pixels that were not sampled during downsampling are calculated using directional interpolation, wherein the process of calculating the missing pixels using directional interpolation includes: (1) Define the pixel value of the outermost pixel located outside the image edge; (2) Treat the sampled pixels and outer pixels as known pixels; (3) Determine the neighboring pixels of the target missing pixel and define multiple interpolation directions. Each direction corresponds to multiple sets of pixels. Each interpolation direction corresponds to a pixel calculation direction. The pixel calculation direction passes through the target missing pixel. (4) Calculate the pixel gradient value for each interpolation direction, wherein the pixel gradient value is obtained by summing the absolute values ​​of the differences between two known pixel values ​​in multiple groups located in that direction and dividing by the number of pixel groups in that interpolation direction; (5) Calculate the direct gradient value of the pixel calculation direction corresponding to each interpolation direction, wherein the direct gradient value is the absolute value of the difference between two known pixel values ​​located on both sides of the target missing pixel in the pixel calculation direction; (6) Calculate candidate pixel values ​​for each pixel in a direction, wherein the candidate pixel values ​​are the average of the known pixel values ​​located on both sides of the target missing pixel in that direction; (7) By comparing the pixel gradient value and the direct gradient value, the optimal interpolation direction and the corresponding pixel calculation direction are selected, and the candidate pixel value in this direction is used as the final pixel value of the target missing pixel.

13. The display system according to claim 10, characterized in that, The image output module outputs the upsampled odd-numbered frame images and even-numbered frame images, including: The even-numbered frames are spatially offset relative to the odd-numbered frames or the odd-numbered frames are spatially offset relative to the even-numbered frames. Specifically, the offset display shifts adjacent odd and even sampling points in the odd and even frames by at least one pixel in a direction that moves them away from each other, in order to optimize the display transition of neighboring sampling points.

14. A display device, characterized in that, It includes the display system of claim 10.

Images