Image processing apparatus, method, and program

Abstract

The present application provides an image processing device, method and program, the image processing device includes data processing unit, the data processing unit is used for obtaining original image, and the original image is down-sampled by using first sub-magnification, and the transition image is obtained, and the transition image is down-sampled by using second sub-magnification, and the target image is obtained;Wherein, the first sub-magnification and the second sub-magnification are obtained by the down-sampling magnification of original image, and the product of the first sub-magnification and the second sub-magnification is equal to the down-sampling magnification. Both can better preserve the texture information of the original image in the original image down-sampling process, and can also make the change of the detail texture in the image continuous in the process of continuous change of sampling magnification, and then improve the display effect of the filtered image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image display, and in particular to an image processing device, method and program. BACKGROUND

[0002] With the continuous development of image display technology, the display form of image content is more and more diversified. For example: picture-in-picture. Picture-in-picture is to superimpose a large and a small image screen, so as to simultaneously present two picture signals. The large image adopts the original resolution, and the small image needs to be down-sampled at the original resolution to obtain a low-resolution image, and then superimposed on the large image for display.

[0003] At present, the commonly used image down-sampling methods mainly include two kinds: the first kind is a nearest neighbor interpolation (NNI) down-sampling method. The second kind is a linear down-sampling method, such as a bilinear interpolation, a bi-cubic interpolation, etc.

[0004] However, the above two down-sampling methods have a serious loss of texture of the image. In some scenarios, the clear texture information in the image becomes blurred, or even lost, thereby reducing the display effect of the down-sampled image. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide an image processing device, method and program, so as to retain more texture information of the image in the down-sampling process.

[0006] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0007] The first aspect of the present application provides an image processing device, the device comprising: a data processing unit configured to obtain an original image, and down-sample the original image by a first sub-magnification to obtain a transition image, and down-sample the transition image by a second sub-magnification to obtain a target image; wherein the first sub-magnification and the second sub-magnification are obtained by splitting a down-sampling magnification of the original image, and the product of the first sub-magnification and the second sub-magnification is equal to the down-sampling magnification.

[0008] The second aspect of the present application provides an image processing method performed by an image processing device, the method comprising: obtaining an original image, and downsampling the original image by a first sub-scale factor to obtain a transition image, and downsampling the transition image by a second sub-scale factor to obtain a target image; wherein the first sub-scale factor and the second sub-scale factor are obtained by splitting a downsampling scale factor of the original image, and the product of the first sub-scale factor and the second sub-scale factor is equal to the downsampling scale factor.

[0009] The third aspect of the present application provides a program for performing image processing using an image processing device, the image processing device comprising a data processing unit, the program causing the data processing unit to perform: obtaining an original image, and downsampling the original image by a first sub-scale factor to obtain a transition image, and downsampling the transition image by a second sub-scale factor to obtain a target image; wherein the first sub-scale factor and the second sub-scale factor are obtained by splitting a downsampling scale factor of the original image, and the product of the first sub-scale factor and the second sub-scale factor is equal to the downsampling scale factor.

[0010] Compared with the prior art, the image processing device provided by the first aspect of the present application, after the data processing unit in the image processing device obtains the original image and the downsampling scale factor, first, the original image is downsampled by a first sub-scale factor to obtain a transition image, and then the transition image is downsampled by a second sub-scale factor to obtain a target image, wherein the first sub-scale factor and the second sub-scale factor are obtained by splitting the downsampling scale factor of the original image, and the product of the first sub-scale factor and the second sub-scale factor is equal to the downsampling scale factor. In this way, the target image after the original image is processed by twice downsampling is obtained. Compared with directly processing the original image into the target image through one downsampling process, by splitting into two downsampling processes, the original image is first processed into the transition image, and then the transition image is processed into the target image, it is possible to use all the pixel points in the image as much as possible in the first downsampling process, and thus various information in the image, including texture information, is retained, and then through the second downsampling, the final required target image is obtained, which can not only better retain the texture information of the original image in the original image downsampling process, but also make the change of the detail texture in the image continuous in the process of continuous change of the sampling scale factor, and thus improve the display effect of the filtered image.

[0011] The image processing method performed by the image processing device provided by the second aspect of the present application and the program for performing image processing using the image processing device provided by the third aspect of the present application have the same or similar beneficial effects as the image processing device provided by the first aspect of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0012] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description read in conjunction with the accompanying drawings, in which like reference numerals refer to like elements throughout. The following detailed description is presented to enable any person skilled in the art to make and use the application. Numerous specific details are described to provide a thorough understanding of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced without these specific details. In other instances, well known methods have not been described in detail in order to avoid obscuring the present application. Those skilled in the art will appreciate the

[0013] Figure 1 Structure of image processing device in an embodiment of the present application Figure 1 ;

[0014] Figure 2 Flow of image processing method performed by image processing device in an embodiment of the present application Figure 1 ;

[0015] Figure 3 Structure of image processing device in an embodiment of the present application Figure 2 ;

[0016] Figure 4 Flow of image processing method performed by image processing device in an embodiment of the present application Figure 2 ;

[0017] Figure 5 Illustration of horizontal / vertical pixel number of first pixel block in an embodiment of the present application

[0018] Figure 6 Illustration of horizontal / vertical weight corresponding to each pixel in first pixel block in an embodiment of the present application

[0019] Figure 7 Hardware configuration of image processing device in an embodiment of the present application DETAILED DESCRIPTION

[0020] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is to be understood that the present application can be embodied in various forms without being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0021] It should be noted that the technical or scientific terms used in the present application should be understood as their common meanings to those skilled in the art, unless otherwise specified.

[0022] At present, when there is a need to reduce the resolution of an image, the existing method is usually to adopt nearest neighbor down-sampling, linear interpolation down-sampling and the like to make the resolution of the image reduced from the original resolution to the target resolution. However, if the above down-sampling method is adopted to reduce the resolution of the image, the texture information in the image will be seriously lost, and thus the display effect of the down-sampled image will be reduced.

[0023] The inventors have found through deep research that the root cause of the serious loss of the texture information in the image caused by the down-sampling of the image by using the nearest neighbor, linear interpolation and the like is that the image is directly reduced from the original resolution to the target resolution at one time, and in the process of down-sampling the image, some pixel points in the image can not be used. In this way, some information in the original image will not be in the down-sampled image, and thus the texture information in the down-sampled image will be seriously lost.

[0024] Therefore, the embodiment of the present application proposes an image processing device. When it is needed to down-sample an image, the data processing unit in the image processing device does not directly reduce the image from the original resolution to the target resolution at one time, but first reduces the image from the original resolution to an intermediate resolution, and then reduces the image from the intermediate resolution to the target resolution. Since the image is first reduced from the original resolution to an intermediate resolution which is higher than the target resolution, all the pixel points in the image can be used in the process of the first down-sampling, and thus various information in the image, including the texture information, can be preserved. Then, the second down-sampling is performed to make the image reach the final target resolution. It can be seen that when it is needed to reduce the resolution of the image, the two-stage filtering method, i.e., first reducing the image from the original resolution to the intermediate resolution and then reducing the image to the target resolution, can better preserve the texture information in the image in the process of down-sampling the image, and thus the display effect of the filtered image can be improved.

[0025] In actual applications, the image processing device provided by the embodiment of the present application can be applied to various scenes in which the resolution of an image needs to be reduced. For example, the image processing device can be applied to the scenes in which the picture-in-picture needs to be used, such as television, video recording, monitoring, demonstration device, remote video and the like. For another example, the image processing device can be applied to the scenes in which the original resolution of the image is too high and the display device cannot support the image with such a high resolution. The specific use scene of the image processing device provided by the embodiment of the present application is not limited here.

[0026] Next, the specific execution process inside the image processing device provided by the embodiment of the present application will be described in detail.

[0027] Figure 1 The structure of the image processing device in the embodiment of the present application is shown in Figure 1 , and the execution process of the image processing device is shown in Figure 1As shown in FIG. 1, in the image processing device, a data processing unit 101 can be included. Through the data processing unit 101, the original image can be down-sampled in two steps, and finally a target image containing more texture information in the original image is obtained.

[0028] That is, the data processing unit in the image processing device is configured to obtain an original image, down-sample the original image by a first sub-multiplication factor to obtain a transition image, and down-sample the transition image by a second sub-multiplication factor to obtain a target image. The first sub-multiplication factor and the second sub-multiplication factor are obtained by splitting the down-sampling multiplication factor of the original image, and the product of the first sub-multiplication factor and the second sub-multiplication factor is equal to the down-sampling multiplication factor.

[0029] Figure 2 The flow of the image processing method performed by the image processing device in the embodiment of the present application is shown in FIG. 2. Figure 1 Referring to FIG. 2, the image processing method can include: Figure 2

[0030] S201: Obtain an original image.

[0031] The original image is an image that needs to be processed to reduce its resolution.

[0032] At the same time of obtaining the original image, the down-sampling multiplication factor corresponding to the original image also needs to be obtained. That is, it needs to be known that the original image is processed into what size of image.

[0033] The down-sampling multiplication factor is the specific multiplication factor by which the image needs to be down-sampled, so that the resolution of the image is reduced from the original resolution to the target resolution.

[0034] For example, assuming that the resolution of the original image is 256x256, due to actual needs, the resolution of the image needs to be reduced to 32x32. Therefore, the down-sampling multiplication factor here is

[0035] It should be noted that since the image is down-sampled, that is, the resolution of the image is reduced. Therefore, the down-sampling multiplication factor here is a number less than 1 and greater than 0. The specific value of the down-sampling multiplication factor needs to be determined according to the actual reduction of the image, which is not limited here.

[0036] In order to avoid losing too much texture information of the image in the down-sampling process, the image can be down-sampled twice. In each down-sampling, the multiplication factor of this time, that is, the first sub-multiplication factor and the second sub-multiplication factor, needs to be known.

[0037] ​The product of the first sub-multiple and the second sub-multiple is equal to the down-sampling multiple. The reason for splitting the down-sampling multiple into the first sub-multiple and the second sub-multiple in the form of a product is that: in the process of image enlargement or reduction, there is a multiple form. Therefore, when the sequential down-sampling process is split into two down-sampling processes, the product of the multiples used in the two down-sampling processes should be equal to the multiple in the previous down-sampling process.

[0038] After obtaining the down-sampling multiple, the down-sampling multiple can be directly split into the form of multiplication of two sub-multiples, so that the first sub-multiple and the second sub-multiple are obtained. Of course, the first sub-multiple can also be determined according to some preset rules first, and then the down-sampling multiple is divided by the first sub-multiple to obtain the second sub-multiple. The specific obtaining method of the first sub-multiple and the second sub-multiple is not limited here.

[0039] Since the down-sampling multiple is a positive number less than 1, and the original image is down-sampled twice, the first sub-multiple and the second sub-multiple are also positive numbers less than 1. Moreover, the first sub-multiple and the second sub-multiple are greater than the down-sampling multiple. In mathematical form, it is: r < 1, r = r1 x r2, r < r1 < 1, r < r2 < 1. Wherein, r represents the down-sampling multiple, r1 represents the first sub-multiple, and r2 represents the second multiple. In this way, the original image will not be reduced too much in the down-sampling process, thereby ensuring that the texture information of the original image is retained.

[0040] Continuing the above example, assume that the down-sampling multiple is Then, can be split into In this way, the first sub-multiple and the second sub-multiple are and respectively. Of course, can also be split into In this way, the first sub-multiple and the second sub-multiple are and respectively. Of course, can also be split into the form of multiplication of other two fractions, as long as the product of the two fractions is equal to the down-sampling multiple. For the specific numerical value of the first sub-multiple and the second sub-multiple after the down-sampling multiple is determined, it is not limited here.

[0041] S202: Down-sampling the original image using the first sub-multiple to obtain a transition image.

[0042] After determining the multiple of the first down-sampling, i.e. the first sub-multiple, the original image can be down-sampled according to the first sub-multiple, and the result obtained after down-sampling is a transition image with an intermediate resolution.

[0043] S203: down-sampling the transition image by the second sub-ratio to obtain a target image.

[0044] After the second down-sampling ratio, i.e. the second sub-ratio, is determined, the transition image can be down-sampled according to the second sub-ratio. The result obtained after the down-sampling is the target image with the target resolution.

[0045] Continuing the above example, assume that the resolution of the original image is 256x256, the first sub-ratio is 2, and the second sub-ratio is 8. The second sub-ratio is 8. After the first down-sampling, i.e. step S203, based on the first sub-ratio of 2, the resolution of the original image is reduced from 256x256 to 128x128, and thus the transition image is obtained. Then, after the second down-sampling, i.e. step S204, based on the second sub-ratio of 8, the resolution of the transition image is reduced from 128x128 to 32x32, and thus the target image with the final required resolution of 32x32 is obtained.

[0046] As to the specific manner of the down-sampling, various down-sampling interpolation manners such as bilinear interpolation and bicubic interpolation can be adopted. The specific manner of the down-sampling adopted in the embodiments of the present application is not limited herein.

[0047] Finally, it needs to be noted that the ratio value in the down-sampling ratio is a relatively low ratio. For example, the image is reduced by 8 times. If the ratio value in the down-sampling ratio is a relatively high ratio, for example, the image is reduced by 800 times, then it is meaningless to reserve the image texture information because the image reduction multiple is too large and any detailed texture information is meaningless. Therefore, the embodiments of the present application perform the down-sampling at a low ratio, and thus the texture information in the original image is reserved. The specific threshold of the low ratio is not limited herein.

[0048] ​From the above, the image processing device provided by the embodiment of the present application, through the data processing unit in the image processing device, the original image and the downsampling ratio are obtained, first, the original image is downsampled by using the first sub-ratio, and the transition image is obtained, then, the transition image is downsampled by using the second sub-ratio, and the target image is obtained, wherein the first sub-ratio and the second sub-ratio are obtained by splitting the downsampling ratio of the original image, and the product of the first sub-ratio and the second sub-ratio is equal to the downsampling ratio. In this way, the target image of the original image after two times of downsampling processing is obtained. Compared with directly processing the original image into the target image through one time of downsampling process, by splitting into two times of downsampling process, the original image is first processed into the transition image, and then the transition image is processed into the target image, so that all the pixel points in the image can be used as much as possible in the first downsampling process, and then various information in the image, including the texture information, is retained, and then the final required target image is obtained through the second downsampling, so that the texture information of the original image can be better retained in the original image downsampling process, and the change of the detail texture in the image is continuous in the process of continuous change of the sampling ratio, and then the display effect of the filtered image is improved.

[0049] Further, as a refinement and extension of the device shown in Figure 1 , the embodiment of the present application also provides an image processing device. Figure 3 For the structure of the image processing device in the embodiment of the present application Figure 2 , see Figure 3 , in the image processing device, it can include: a data processing unit 101. Through the data processing unit 101, the original image can be obtained, and the original image is downsampled by using the first sub-ratio to obtain the transition image, and the transition image is downsampled by using the second sub-ratio to obtain the target image.

[0050] And in the data processing unit 101, it can include: a pixel block calculation unit 102 and an image sampling unit 103. Through the pixel block calculation unit 102, the first pixel block horizontal / vertical pixel number in the original image for downsampling can be determined according to the first sub-ratio, and the first pixel block horizontal / vertical pixel number is greater than or equal to the inverse of the first sub-ratio. Through the image sampling unit 103, the original image can be downsampled based on the first sub-ratio and the first pixel block horizontal / vertical pixel number to obtain the transition image.

[0051] In the image sampling unit 103, a position calculation unit 104 and a pixel calculation unit 105 can be included. The position calculation unit 104 can determine the positions of the original pixel blocks corresponding to the transition pixel points in the original image according to the positions of the transition pixel points in the transition image, the first sub-multiples, and the horizontal / vertical pixel numbers of the first pixel blocks. The pixel calculation unit 105 can calculate the pixel values of the transition pixel points in the transition image based on the positions of the original pixel blocks corresponding to the transition pixel points in the original image and the horizontal / vertical pixel numbers of the first pixel blocks, and obtain the transition image.

[0052] In the image sampling unit 103, a weight determination unit 106, a pixel block determination unit 107, and a weighting unit 108 can be further included. The weight determination unit 106 can determine the weights corresponding to the pixel points in the first pixel block. The pixel block determination unit 107 can determine the original pixel blocks in the original image according to the positions of the original pixel blocks corresponding to the transition pixel points in the original image and the horizontal / vertical pixel numbers of the first pixel blocks. The weighting unit 108 can multiply the pixel values of the pixel points in each original pixel block by the weights corresponding to the pixel points in the first pixel block, add the results of the corresponding multiplications, and obtain the pixel values of the transition pixel points in the transition image.

[0053] In the weight determination unit 106, an index determination unit 109 and a weight acquisition unit 1010 can be included. The index determination unit 109 can determine the indexes of the transition pixel points in the first preset table according to the positions of the transition pixel points in the transition image, the first sub-multiples, and the length of the first preset table, and the first preset table can store the corresponding relationship between the indexes and the corresponding weights. The weight acquisition unit 1010 can determine the weights corresponding to the pixel points in the first pixel block according to the indexes of the transition pixel points in the first preset table.

[0054] Further, the positions of the transition pixel points in the transition image include horizontal coordinates and vertical coordinates, and the weights corresponding to the pixel points in the first pixel block include horizontal weights and vertical weights. Correspondingly, the weighting unit 108 can multiply the pixel values of the pixel points in each original pixel block by the horizontal weights and the vertical weights corresponding to the pixel points in the first pixel block, add the results of the corresponding multiplications, and obtain the pixel values of the transition pixel points in the transition image.

[0055] Further, as a refinement and extension of the method shown in Figure 3 , and a detailed description of the specific execution process of the device shown in Figure 4 . Figure 2 The flowchart of the image processing method executed by the image processing device in the embodiments of the present application is shown in Figure 4 , referring to Downsampling ratio (r) , the image processing method can include:

[0056] S401: Obtain an original image and a down-sampling rate.

[0057] Step S401 is the same as the specific implementation of step S201 described above, and will not be repeated here.

[0058] S402: Split the down-sampling rate into a first sub-rate and a second sub-rate.

[0059] In the process of splitting the down-sampling rate into a first sub-rate and a second sub-rate, in order to ensure the existence of texture information in the subsequent down-sampling process and to retain more texture information, the first sub-rate and the second sub-rate need to follow at least one of the following two rules.

[0060] Rule one: And is not an integer.

[0061] Rule two: r1 is greater than r2.

[0062] Wherein, r represents the down-sampling rate, r1 represents the first sub-rate, and r2 represents the second rate.

[0063] Through rule one, it can be ensured that the texture is not lost in the down-sampling process. Through rule two: it can be ensured that more texture is retained in the down-sampling process.

[0064] It should be noted here that in rule two, r1 can be set as large as possible. Since r<1, r=r1×r2, so r<r1<1, that is, r1 can be made more close to 1. The specific value of r1 is not limited here.

[0065] The specific values of the first sub-rate and the second sub-rate under different down-sampling rates are given in Table 1 below. Of course, this does not mean that the first sub-rate and the second sub-rate under a certain sampling rate can only be according to the first sub-rate and the second sub-rate given in Table 1. The specific values given here are only the values that have better texture information retention effect after a large amount of practice.

[0066] Table 1 Corresponding relationship between each down-sampling rate and corresponding first sub-rate and second sub-rate

[0067] First sub-ratio (rl) Second sub-ratio (r2) Figure 5 1 / 2 3 / 4 2 / 3 1 / 3 5 / 6 2 / 5 1 / 4 7 / 8 2 / 7 1 / 5 7 / 10 2 / 7 1 / 6 7 / 12 2 / 7 1 / 7 9 / 14 2 / 9 1 / 8 9 / 16 2 / 9

[0068] S403: Determine the number of horizontal / vertical pixels of the first pixel block in the original image for down-sampling according to the first sub-rate.

[0069] When the original image needs to be downsampled for the first time according to the first sub-magnification, it is first necessary to determine what area of ​​pixels in the original image should be aggregated to obtain a pixel in the transition image. This area is the first pixel block.

[0070] In this design, the number of horizontal / vertical pixels in the first pixel block is greater than or equal to the reciprocal of the first sub-multiplier. This ensures that every pixel in the original image can be used during the process of transforming the original image into a transition image, thereby better preserving the texture information in the original image.

[0071] Figure 3 This is a schematic diagram showing the number of horizontal / vertical pixels in the first pixel block in an embodiment of this application. See [link / reference]. Figure 6 As shown, assuming the original image size is 10×10, the first sub-magnification is... Therefore, the minimum number of horizontal / vertical pixels in the first pixel block is 2×2. This ensures that every pixel in the original image can be used, thus preserving image texture information during downsampling. Of course, the minimum number of horizontal / vertical pixels in the first pixel block can also be 4×4. The specific value of the number of horizontal / vertical pixels in the first pixel block is not limited here. However, generally, the number of horizontal / vertical pixels in the first pixel block is n×n, where n is usually an even number. This ensures that the values ​​of each pixel in the transition image are more balanced during downsampling.

[0072] S404: Based on the position of each transition pixel in the transition image, the first sub-magnification, and the number of horizontal / vertical pixels in the first pixel block, determine the position of the original pixel block corresponding to each transition pixel used for downsampling in the original image.

[0073] In other words, for each pixel in the transition image, the position of the corresponding original pixel block used for downsampling in the original image needs to be determined based on the position of each transition pixel in the transition image, the first sub-magnification, and the number of horizontal / vertical pixels in the first pixel block.

[0074] Taking the determination of the position of the original pixel block in the original image corresponding to a certain pixel point in the transition image as an example, specifically, step S404 can be determined by the following formulas (1) and (2).

[0075]

[0076]

[0077] Wherein, x_dst0, y_dst0 represent the coordinates of any one pixel point in the transition image, r1 represents the first sub-multiple, n represents the horizontal / vertical pixel number of the first pixel block, x_src0, y_src0 represent the coordinates of the first pixel in the original pixel block in the original image.

[0078] Of course, the position of each transition pixel point in the transition image, the first sub-multiple, and the horizontal / vertical pixel number of the first pixel block can also be used to determine the position of the original pixel block corresponding to each transition pixel point in the original image for downsampling in other ways. For example, coefficients are added to the above formula (1) and formula (2), or formula (1) and formula (2) are adjusted. The specific manner is not limited here.

[0079] After step S404, that is, after determining the coordinates of all pixels in the original pixel block corresponding to each transition pixel point in the original image for downsampling, the weight corresponding to each pixel point in the first pixel block needs to be determined, so that all original pixel blocks corresponding to each transition pixel point in the original image are calculated by interpolation. In this way, the calculated pixel value can be more accurate, thereby improving the accuracy of image processing.

[0080] The weight corresponding to each pixel point in the first pixel block can be pre-stored in the first preset table and obtained by querying. It can also be obtained in combination with the existing calculation formula. Whether through the preset table or the calculation formula, it can be generated according to the nearest neighbor formula, bilinear formula, bicubic formula, or polynomial formula or by experience.

[0081] In the specific implementation process, the weight corresponding to each pixel point in the first pixel block can be one weight value corresponding to one pixel point, or one weight value corresponding to one pixel point in the horizontal direction and one weight value corresponding to one pixel point in the vertical direction, and then the weight values in the two directions are multiplied to obtain the weight of the pixel point. The specific form of the weight is not limited here. When the weight is in the form of the latter, the following step S405 needs to be performed.

[0082] S405: Determine the index of each transition pixel point in the first preset table according to the position of each transition pixel point in the transition image, the first sub-multiple, and the length of the first preset table LUT1.

[0083] Wherein, the first preset table stores the corresponding relationship between each index and the corresponding weight.

[0084] Taking the weight corresponding to a certain pixel point in the transition image in the first preset table as an example, step S405 can be determined by the following formula (3) and formula (4).

[0085]

[0086]

[0087] wherein x_dst0, y_dst0 represent the coordinates of any one pixel point in the transition image, len represents the length of the first preset table, r1 represents the first sub-multiple, phase1_x, phase1_y represent the indexes of the pixel point in the first row and the first column in the transition image in the first preset table, represents the floor function.

[0088] Since the transition pixel point has a horizontal coordinate and a vertical coordinate, the indexes calculated by the above formula (3) and formula (4) contain horizontal indexes and vertical indexes.

[0089] Of course, the indexes of the transition pixel points in the first preset table can also be determined by other ways using the positions of the transition pixel points in the transition image, the first sub-multiple, and the length of the first preset table. For example, a coefficient is added in the above formula (3) and formula (4), or the formula (3) and formula (4) are adjusted. The specific way is not limited here.

[0090] S406: Determine the weights corresponding to the pixel points in the first pixel block from the first preset table according to the indexes of the transition pixel points in the first preset table.

[0091] After obtaining the indexes of the transition pixel points in the first preset table in two directions respectively, i.e. the horizontal indexes and the vertical indexes, for each pixel point, the corresponding horizontal weight and vertical weight can be found from the first preset table. Further corresponding to the original image, the horizontal weight and the vertical weight corresponding to the pixel points in the first pixel block can be obtained.

[0092] That is, phase1_x and phase1_y obtained by the above formula (3) and formula (4) can find the corresponding params0_x and params0_y in the first preset table. The parameters such as params0_x and params0_y correspond to the horizontal weight and the vertical weight corresponding to the pixel points in the first pixel block.

[0093] Figure 4 The schematic diagram of the horizontal weight and the vertical weight corresponding to the pixel points in the first pixel block in the embodiment of the present application is shown in FIG. 2. Figure 7As shown, the corresponding original image of one pixel in the transition image can be 4x4 pixels. Correspondingly, the first pixel block is 4x4. In this way, the four pixels in each row of the first pixel block correspond to the horizontal weights params0_x[0], params0_x[1], params0_x[2], and params0_x[3], respectively. The four pixels in each column of the first pixel block correspond to the vertical weights params0_y[0], params0_y[1], params0_y[2], and params0_y[3], respectively. As can be seen, the weight of the 0th pixel in the first pixel block is params0_x[0]xparams0_y[0], and the weight of the 15th pixel in the first pixel block is params0_x[3]xparams0_y[3].

[0094] S407: Determine the original pixel blocks in the original image according to the positions of the original pixel blocks corresponding to each transition pixel in the original image and the number of horizontal / vertical pixels in the first pixel block.

[0095] After determining the horizontal and vertical weights corresponding to each pixel in the first pixel block, in order to obtain a pixel in the transition image by interpolating the pixels in a block in the original image through the weights, it is necessary to determine the pixel regions in the original image corresponding to each pixel in the transition image, i.e., the original pixel blocks.

[0096] For example, assuming that a 256x256 original image needs to be transformed into a 128x128 transition image, the first image block is 4x4. For the 0th pixel in the transition image, the corresponding original pixel block in the original image is the first four pixels in the first row, the first four pixels in the second row, the first four pixels in the third row, and the first four pixels in the fourth row, i.e., 16 pixels. In this way, the original image is divided into multiple original pixel blocks similar to this, and each original pixel block after processing is each pixel in the transition image.

[0097] For example, assuming that n=4, then the all pixel coordinates of one pixel in the transition image mapped to the original pixel block in the original image are (x_src0, y_src0), (x_src1, y_src0), (x_src2, y_src0), (x_src3, y_src0), …, (x_src0, y_src3), (x_src1, y_src3), (x_src2, y_src3), (x_src3, y_src3), i.e., 16 pixels. Further, the 16 pixels are combined with the horizontal and vertical weights to perform interpolation calculation, and the pixel value at the corresponding position in the transition image can be obtained.

[0098] S408: For each original pixel block, multiply the pixel value of each pixel point in the original pixel block, the horizontal weight corresponding to each pixel point in the first pixel block, and the vertical weight corresponding to each pixel point in the first pixel block, and add the results of the corresponding multiplications to obtain the pixel value of each transition pixel point in the transition image, and further obtain the transition image.

[0099] After determining the original pixel block corresponding to each pixel point in the transition image in the original image, and the horizontal weight and the vertical weight of each pixel point in the first pixel block, for each original pixel block, multiply the pixel value of each pixel point in the original pixel block, the horizontal weight corresponding to each pixel point in the first pixel block, and the vertical weight corresponding to each pixel point in the first pixel block, and add the results of the corresponding multiplications. In this way, a pixel point in the transition image is obtained. By analogy, each target pixel block is calculated in this way, and the pixel value of each transition pixel point in the transition image is obtained.

[0100] Taking a pixel point in the transition image as an example, specifically, step S408 can be implemented by the following formula (5).

[0101] val_dst0(x_dst0_0,y_dst0_0)=

[0102] params0_x[0]×params0_y[0]×val_src0(x_src0,y_src0)+

[0103] params0_x[1]×params0_y[0]×val_src0(x_src1,y_src0)+

[0104] params0_x[2]×params0_y[0]×val_src0(x_src2,y_src0)+

[0105] …

[0106] params0_x[2]×params0_y[3]×val_src0(x_src2,y_src3)+

[0107] params0_x[3]×params0_y[3]×val_src0(x_src3,y_src3)

[0108] Formula (5)

[0109] Wherein, val_dst0(x_dst0_0, y_dst0_0) represents the pixel value of the pixel point with the coordinate (x_dst0_0, y_dst0_0) in the transition image, val_src0(x_src0, y_src0) represents the pixel value of the pixel point with the coordinate (x_src0, y_src0) in the original image, and params0_x[0] and params0_y[0] respectively represent the weight corresponding to the first column of pixel points in the original image block and the weight corresponding to the first row of pixel points, that is, the horizontal weight corresponding to the first column of pixel points in the first pixel block and the vertical weight corresponding to the first row of pixel points.

[0110] In this way, the process of down-sampling the original image according to the first sub-multiple to obtain the transition image is completed. The process of down-sampling the transition image according to the second sub-multiple to obtain the target image is similar to the specific implementation mode of steps S403-S408, and only a brief description is given below, and the detailed process parameters are described above in steps S403-S408.

[0111] S409: determining the second pixel block horizontal / vertical pixel number for down-sampling in the transition image according to the second sub-multiple.

[0112] Since the value of the second sub-multiple is smaller than the value of the second sub-multiple, the second pixel block horizontal / vertical pixel number is greater than the first pixel block horizontal / vertical pixel number. For example, the first pixel block horizontal / vertical pixel number n=4, and the second pixel block horizontal / vertical pixel number m=6.

[0113] S410: determining the position of the transition image block corresponding to each target pixel point in the transition image according to the position of each target pixel point in the target image, the second sub-multiple, and the second pixel block horizontal / vertical pixel number.

[0114] Taking the determination of the position of the transition image block corresponding to a pixel point in the transition image in the transition image as an example, specifically, step S410 can be determined by the following formula (6), formula (7).

[0115]

[0116]

[0117] Wherein, x_dst1, y_dst1 represent the coordinates of any one pixel point in the target image, r2 represents the second sub-multiple, m represents the second pixel block horizontal / vertical pixel number, and x_dst0_0, y_dst0_0 represent the coordinates of the first pixel in the transition image block in the transition image.

[0118] S411: Determine the index of each target pixel point in the second preset table according to the position of each target pixel point in the target image, the second sub-multiple, and the length of the second preset table LUT2.

[0119] For example, the corresponding weight of a certain pixel point in the target image in the second preset table is determined, and specifically, step S411 can be determined by the following formula (8) and formula (9).

[0120]

[0121]

[0122] Wherein, x_dst1_0, y_dst1_0 represents the coordinates of the pixel point of the first row and the first column in the target image, len_1 represents the length of the second preset table, r2 represents the second sub-multiple, phase2_x, phase2_y represents the index of the pixel point of the first row and the first column in the target image in the second preset table, Downward rounding is represented.

[0123] S412: Determine the corresponding weight of each pixel point in the second pixel block from the second preset table according to the index of each target pixel point in the second preset table.

[0124] S413: Determine each transition pixel block in the transition image according to the position of each target pixel point corresponding to the transition image block in the transition image and the horizontal / vertical pixel number of the second pixel block.

[0125] S414: For each transition pixel block, multiply the pixel value of each pixel point in the transition pixel block, the corresponding horizontal weight and vertical weight of each pixel point in the second pixel block, and add the corresponding multiplication result to obtain the pixel value of each target pixel point in the target image, and further obtain the target image.

[0126] For example, a certain pixel point in the target image is determined, and specifically, step S414 can be realized by the following formula (10).

[0127] val_dst1_0(x_dst1_0,y_dst1_0)=

[0128] params1_0_x[0]×params1_0_y[0]×val_dst0(x_dst0,y_dst0)+

[0129] params1_0_x[1]×params1_0_y[0]×val_dst0(x_dst1,y_dst0)+

[0130] params1_0_x[2] * params1_0_y[0] * val_dst0(x_dst2, y_dst0) + params1_0_x[2] * params1_0_y[1] * val_dst0(x_dst2, y_dst1) + params1_0_x[2] * params1_0_y[2] * val_dst0(x_dst2, y_dst2) + params1_0_x[2] * params1_0_y[3] * val_dst0(x_dst2, y_dst3)

[0131] …

[0132] params1_0_x[2] * params1_0_y[0] * val_dst0(x_dst2, y_dst0) + params1_0_x[2] * params1_0_y[1] * val_dst0(x_dst2, y_dst1) + params1_0_x[2] * params1_0_y[2] * val_dst0(x_dst2, y_dst2) + params1_0_x[2] * params1_0_y[3] * val_dst0(x_dst2, y_dst3)

[0133] params1_0_x[2] * params1_0_y[0] * val_dst0(x_dst2, y_dst0) + params1_0_x[2] * params1_0_y[1] * val_dst0(x_dst2, y_dst1) + params1_0_x[2] * params1_0_y[2] * val_dst0(x_dst2, y_dst2) + params1_0_x[2] * params1_0_y[3] * val_dst0(x_dst2, y_dst3)

[0134] Equation (10)

[0135] Wherein, val_dst1_0(x_dst1_0, y_dst1_0) represents the pixel value of the pixel point with coordinates (x_dst1_0, y_dst1_0) in the target image, val_dst0(x_dst0, y_dst0) represents the pixel value of the pixel point with coordinates (x_dst0, y_dst0) in the transition image, params1_0_x[0] and params1_0_y[0] respectively represent the weight corresponding to the first column of pixel points in the transition image block and the weight corresponding to the first row of pixel points, that is, the horizontal weight corresponding to the first column of pixel points in the second pixel block and the vertical weight corresponding to the first row of pixel points.

[0136] So far, the process of downsampling the transition image according to the second sub-magnification to obtain the target image is completed. Finally, the original image is processed into the target image.

[0137] As can be seen from the above, the image processing method provided by the embodiments of the present application divides the process of reducing the original image into two levels of downsampling scheme, the first level retains the high-frequency image information, and the second level reaches the final target resolution. This method can avoid that part of the pixel points in the original image do not participate in the downsampling process. At the same time, the downsampling magnification splitting method and the principles followed are limited, which can ensure that the downsampled image retains good texture information at low magnification, and also ensures that the downsampled image retains good contour information at high magnification. Moreover, the front and rear two levels use different width preset tables and different size pixel blocks for filtering operation, which can make the two levels of filtering smoothly transition in the process of continuously changing the reduction magnification, so as to ensure that the texture of the original image smoothly transitions in the continuous change of the magnification.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a program for executing image processing using an image processing device. The image processing device comprises a data processing unit, and the program causes the data processing unit to execute: obtaining an original image, and performing down-sampling on the original image by a first sub-magnification to obtain a transition image, and performing down-sampling on the transition image by a second sub-magnification to obtain a target image; wherein the first sub-magnification and the second sub-magnification are obtained by splitting a down-sampling magnification of the original image, and the product of the first sub-magnification and the second sub-magnification is equal to the down-sampling magnification.

[0139] Figure 7 For the hardware configuration of the image processing device in the embodiment of the present application, refer to ​ as shown in the figure, the image processing device can comprise:

[0140] A central processing unit (CPU) 701 is used as a control unit or a data processing unit to execute various processes according to programs stored in a read-only memory (ROM) 702 or a storage unit 708. For example, the CPU 701 executes processes according to the order described in the above embodiment. A random access memory (RAM) 703 stores programs and related data to be executed by the CPU 701 therein. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704.

[0141] The CPU 701 is connected to an input / output interface 705 via the bus 704, and the input / output interface 705 is connected to an input unit 706 including various switches, a keyboard, a mouse device, a microphone, a sensor, and other input components, and an output unit 707 including a display, a speaker, and other output components.

[0142] The CPU 701 executes various processes in response to instructions input from the input unit 706, and outputs the processing results to, for example, the output unit 707.

[0143] The storage unit 708 connected to the input / output interface 705 includes, for example, a hard disk or the like, and stores programs and various data to be executed by the CPU 701 therein. A communication unit 709 is used as a transmission / reception unit for Wi-Fi communication, Bluetooth (registered trademark) (BT) communication, and any other data communication via a network such as the Internet or a local area network, and communicates with external devices.

[0144] The drive 310 connected to the input / output interface 705 drives the removable medium 311 including, for example, a magnetic disk, a magneto-optical disk, and a semiconductor memory such as a memory card, to perform recording / reading of data.

[0145] It should be noted here that the above procedure embodiment description is similar to the above method or device embodiment description, and has similar beneficial effects as the method or device embodiment. For technical details not disclosed in the procedure embodiment of the present application, please refer to the description of the method or device embodiment of the present application for understanding.

[0146] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image processing device, characterized in that, The device is used in a picture-in-picture scenario, and the device includes: The data processing unit is used to acquire the original image, and downsample the original image using a first sub-magnification to use all pixels in the image during the first downsampling process, retain the texture information in the image, and obtain a transition image, and downsample the transition image using a second sub-magnification to obtain the target image; Wherein, the first sub-magnification and the second sub-magnification are obtained by decomposing the downsampling magnification of the original image, the product of the first sub-magnification and the second sub-magnification is equal to the downsampling magnification, the second sub-magnification is obtained by dividing the downsampling magnification by the first sub-magnification, the first sub-magnification is determined according to a preset rule, the reciprocal of the first sub-magnification and the reciprocal of the second sub-magnification are not integers; and the first sub-magnification is greater than the second sub-magnification. The data processing unit includes: The pixel block calculation unit is used to determine the number of horizontal / vertical pixels of the first pixel block used for downsampling in the original image according to the first sub-magnification. The first pixel block is the pixel area of ​​a pixel in the transition image obtained by aggregating pixels in the original image. The number of horizontal / vertical pixels of the first pixel block is greater than or equal to the reciprocal of the first sub-magnification. The number of horizontal / vertical pixels of the first pixel block is n×n, where n is an even number. An image sampling unit is used to downsample the original image based on the first sub-magnification and the number of horizontal / vertical pixels in the first pixel block to obtain the transition image; The image sampling unit includes: The weight determination unit is used to determine the weight corresponding to each pixel in the first pixel block; A pixel block determination unit is used to determine each original pixel block in the original image based on the position of the original pixel block corresponding to each transition pixel point in the transition image in the original image and the number of horizontal / vertical pixels of the first pixel block; The weighting unit is used to multiply the pixel value of each pixel in each original pixel block by the corresponding weight of each pixel in the first pixel block, and add the corresponding multiplication results to obtain the pixel value of each transition pixel in the transition image, thereby obtaining the transition image. The weight determination unit includes: The index determination unit is used to determine the index of each transition pixel in the first preset table based on the position of each transition pixel in the transition image, the first sub-magnification, and the length of the first preset table. The first preset table stores the correspondence between each index and its corresponding weight. The weight acquisition unit is used to determine the weight corresponding to each pixel in the first pixel block from the first preset table based on the index of each transition pixel in the first preset table.

2. The device according to claim 1, characterized in that, The image sampling unit includes: The position calculation unit is used to determine the position of the original pixel block corresponding to each transition pixel point used for downsampling in the original image based on the position of each transition pixel point in the transition image, the first sub-magnification, and the number of horizontal / vertical pixels of the first pixel block; The pixel calculation unit is used to calculate the pixel value of each transition pixel in the transition image based on the position of the original pixel block corresponding to each transition pixel in the original image and the number of horizontal / vertical pixels of the first pixel block, so as to obtain the transition image.

3. The device according to claim 1, characterized in that, The position of each transition pixel in the transition image includes horizontal and vertical coordinates, and the weight of each pixel in the first pixel block includes horizontal weight and vertical weight. The weighting unit is specifically used to multiply the pixel value of each pixel in each original pixel block by the corresponding horizontal weight and vertical weight of each pixel in the first pixel block, and then add the results of the corresponding multiplications to obtain the pixel value of each transition pixel in the transition image.

4. An image processing method executed by an image processing device, characterized in that, The method is applied to a picture-in-picture scene, and the method includes: The original image is acquired and downsampled using a first sub-magnification to utilize all pixels in the image during the first downsampling process, preserve the texture information in the image, and obtain a transition image. The transition image is then downsampled using a second sub-magnification to obtain the target image. Wherein, the first sub-magnification and the second sub-magnification are obtained by decomposing the downsampling magnification of the original image, the product of the first sub-magnification and the second sub-magnification is equal to the downsampling magnification, the second sub-magnification is obtained by dividing the downsampling magnification by the first sub-magnification, the first sub-magnification is determined according to a preset rule, the reciprocal of the first sub-magnification and the reciprocal of the second sub-magnification are not integers; and the first sub-magnification is greater than the second sub-magnification. The step of downsampling the original image using a first sub-magnification, so as to utilize all pixels in the image during the first downsampling process, preserve the texture information in the image, and obtain a transition image, includes: The number of horizontal / vertical pixels in the first pixel block used for downsampling in the original image is determined according to the first sub-magnification. The first pixel block is the pixel area of ​​a pixel in the transition image obtained by aggregating pixels in the original image. The number of horizontal / vertical pixels in the first pixel block is greater than or equal to the reciprocal of the first sub-magnification. The number of horizontal / vertical pixels in the first pixel block is n×n, where n is an even number. The original image is downsampled based on the first sub-magnification and the number of horizontal / vertical pixels in the first pixel block to obtain the transition image; The step of downsampling the original image based on the first sub-magnification and the number of horizontal / vertical pixels in the first pixel block to obtain the transition image includes: Determine the weight of each pixel in the first pixel block; Each original pixel block in the original image is determined based on the position of the original pixel block corresponding to each transition pixel point in the transition image in the original image and the number of horizontal / vertical pixels of the first pixel block; For each original pixel block, the pixel value of each pixel in the block is multiplied by the corresponding weight of each pixel in the first pixel block, and the results of the multiplication are added together to obtain the pixel value of each transition pixel in the transition image, thus obtaining the transition image. The step of determining the weights corresponding to each pixel in the first pixel block includes: Based on the position of each transition pixel in the transition image, the first sub-magnification, and the length of the first preset table, the index of each transition pixel in the first preset table is determined. The first preset table stores the correspondence between each index and its corresponding weight. The weights corresponding to each pixel in the first pixel block are determined from the first preset table based on the index of each transition pixel in the first preset table.

5. A computer program product that performs image processing using an image processing device, characterized in that, The device is used in a picture-in-picture scene, and the image processing device includes a data processing unit. The program causes the data processing unit to execute: The original image is acquired and downsampled using a first sub-magnification to utilize all pixels in the image during the first downsampling process, preserve the texture information in the image, and obtain a transition image. The transition image is then downsampled using a second sub-magnification to obtain the target image. Wherein, the first sub-magnification and the second sub-magnification are obtained by decomposing the downsampling magnification of the original image, the product of the first sub-magnification and the second sub-magnification is equal to the downsampling magnification, the second sub-magnification is obtained by dividing the downsampling magnification by the first sub-magnification, the first sub-magnification is determined according to a preset rule, the reciprocal of the first sub-magnification and the reciprocal of the second sub-magnification are not integers; and the first sub-magnification is greater than the second sub-magnification. The step of downsampling the original image using a first sub-magnification, so as to utilize all pixels in the image during the first downsampling process, preserve the texture information in the image, and obtain a transition image, includes: The number of horizontal / vertical pixels in the first pixel block used for downsampling in the original image is determined according to the first sub-magnification. The first pixel block is the pixel area of ​​a pixel in the transition image obtained by aggregating pixels in the original image. The number of horizontal / vertical pixels in the first pixel block is greater than or equal to the reciprocal of the first sub-magnification. The number of horizontal / vertical pixels in the first pixel block is n×n, where n is an even number. The original image is downsampled based on the first sub-magnification and the number of horizontal / vertical pixels in the first pixel block to obtain the transition image; The step of downsampling the original image based on the first sub-magnification and the number of horizontal / vertical pixels in the first pixel block to obtain the transition image includes: Determine the weight of each pixel in the first pixel block; Each original pixel block in the original image is determined based on the position of the original pixel block corresponding to each transition pixel point in the transition image in the original image and the number of horizontal / vertical pixels of the first pixel block; For each original pixel block, the pixel value of each pixel in the block is multiplied by the corresponding weight of each pixel in the first pixel block, and the results of the multiplication are added together to obtain the pixel value of each transition pixel in the transition image, thus obtaining the transition image. The step of determining the weights corresponding to each pixel in the first pixel block includes: Based on the position of each transition pixel in the transition image, the first sub-magnification, and the length of the first preset table, the index of each transition pixel in the first preset table is determined. The first preset table stores the correspondence between each index and its corresponding weight. The weights corresponding to each pixel in the first pixel block are determined from the first preset table based on the index of each transition pixel in the first preset table.

Images