Image processing method, image processing device and chip

Abstract

The embodiment of the present application provides an image processing method, an image processing device and a chip. The method includes: obtaining an original image, the original image includes full-color pixels and color pixels; determining multiple positions to be interpolated in the color pixel positions of the original image; based on each position to be interpolated, determining the target image block corresponding to the position to be interpolated; for each target image block, determining the target filter kernel of the target image block according to the feature information of the target image block and the pre-trained relationship model; for each position to be interpolated, filtering the target image block corresponding to the position to be interpolated according to the target filter kernel to obtain the interpolation pixel value of the position to be interpolated. Based on this scheme, there is no need to calculate color difference or color ratio, which avoids the problem of difficult calculation of color difference or color ratio in different directions when the distribution symmetry of R, G and B pixels is not strong, and achieves a better interpolation effect.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to an image processing method, an image processing device and a chip. Background Art

[0002] With the development of camera technology of electronic devices such as mobile phones, the requirements for dark light scenes are getting higher and higher. Therefore, compared with the RGB-based Bayer arrangement, the RGBW CFA (Color Filter Array) with a larger light intake has emerged, such as Figure 1 RGBW CFA shown. Since the panchromatic (W) channel has a higher signal-to-noise ratio than the red (R), green (G), and blue (B) channels and can bring in more light, the RGBW CFA has higher sensitivity, especially for better imaging performance in dark scenes.

[0003] However, compared with the traditional Bayer CFA, the RGBW CFA with the W channel has a larger period and weaker symmetry. Traditional de-mosaicing algorithms are often used for relatively symmetrical CFA structures such as Bayer. They estimate the local direction by calculating features such as color difference or color ratio in different directions, and then interpolate based on this direction. For RGBW CFA, due to the asymmetric distribution of R, G, and B pixels, it is very difficult to calculate the color difference or color ratio in different directions around them. Interpolation of RGBW CFA based on traditional de-mosaicing algorithms cannot achieve a good interpolation effect. Summary of the invention

[0004] In view of this, embodiments of the present application provide an image processing method, an image processing device, and a chip to at least partially solve the above problems.

[0005] According to a first aspect of an embodiment of the present application, there is provided an image processing method, which includes:

[0006] Acquire an original image, wherein the original image includes full-color pixels and color pixels;

[0007] Determining a plurality of positions to be interpolated in the color pixel positions of the original image;

[0008] Based on each of the positions to be interpolated, determining a target image block corresponding to the position to be interpolated;

[0009] For each of the target image blocks, determining a target filter kernel of the target image block according to feature information of the target image block and a pre-trained relationship model;

[0010] For each of the positions to be interpolated, filtering is performed on the target image block corresponding to the position to be interpolated according to the target filter core to obtain an interpolation pixel value of the position to be interpolated.

[0011] According to a second aspect of an embodiment of the present application, there is provided an image processing apparatus, comprising:

[0012] An acquisition module, used for acquiring an original image, wherein the original image includes full-color pixels and color pixels;

[0013] A first determination module, used to determine a plurality of positions to be interpolated in the color pixel positions of the original image;

[0014] A second determination module, configured to determine, based on each of the positions to be interpolated, a target image block corresponding to the position to be interpolated;

[0015] A third determination module is used to determine, for each target image block, a target filter kernel of the target image block according to feature information of the target image block and a pre-trained relationship model;

[0016] The processing module is used to perform filtering processing on the target image block corresponding to each position to be interpolated according to the target filter core to obtain the interpolation pixel value of the position to be interpolated.

[0017] According to a third aspect of an embodiment of the present application, a chip is provided, comprising: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the image processing method as described in the first aspect.

[0018] According to a fourth aspect of an embodiment of the present application, there is provided an image processing device, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the image processing method as described in the first aspect.

[0019] Based on the above scheme, by acquiring the original image, multiple positions to be interpolated are determined in the color pixel positions of the original image, and based on each position to be interpolated, the target image block corresponding to the position to be interpolated is determined. For each target image block, the target filter kernel of the target image block is determined according to the feature information of the target image block and the pre-trained relationship model, and then for each position to be interpolated, the target image block corresponding to the position to be interpolated is filtered according to the target filter to obtain the interpolation pixel value of the position to be interpolated. In this scheme, according to the feature information of the target image block corresponding to each position to be interpolated, the target filter kernel corresponding to each target image block is determined by the pre-trained relationship model, that is, the local features of the image area where each position to be interpolated is located are used to determine the target filter kernel, so as to filter the target image block corresponding to each position to be interpolated respectively, and obtain the interpolation pixel value of the position to be interpolated, without calculating the color difference or color ratio, avoiding the problem of difficult calculation of color difference or color ratio in different directions when the distribution symmetry of R, G and B pixels is not strong, and achieving a better interpolation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0021] Figure 1 A schematic diagram of the structure of an RGBW CFA provided in an embodiment of the present application;

[0022] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0023] Figure 3 A flowchart of an image processing method provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of an exemplary image block type provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of an exemplary image block type provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a method for calculating image block gradient information provided by an embodiment of the present application;

[0027] Figure 7 A flowchart of an image processing method provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of an exemplary G pixel value interpolation process provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of an exemplary two-step W pixel value interpolation process provided in an embodiment of the present application;

[0030] Fig.10 A schematic diagram of a first region of an exemplary color pixel position provided in an embodiment of the present application;

[0031] Fig.11 A schematic diagram comparing the effects of the image processing method provided in the embodiment of the present application and the traditional interpolation method.

[0032] Fig.12 A flowchart of an image processing method provided in an embodiment of the present application.

[0033] Fig.13 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application.

[0034] Fig.14 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0036] The image processing method provided in the embodiment of the present application is suitable for processing the original image collected by the image sensor. In order to facilitate understanding of the technical solution provided by the present application, the image sensor (or pixel array) is first briefly described below.

[0037] The structure of each pixel in the pixel array is similar. Generally, the structure of each pixel includes a microlens, a filter, and a photosensitive element. The microlens is located above the filter, and the filter is located above the photosensitive element. The light returned by the imaging object is focused by the microlens and then emitted, and then filtered by the filter and then emitted into the photosensitive element, and the photosensitive element converts the light signal into an electrical signal. Depending on the type of light that can be transmitted by different filters, the pixel array may include R pixels, B pixels, and G pixels. For example, an R pixel refers to only red light that is filtered by a filter and input into the photosensitive element, which is converted into an electrical signal by the photosensitive element. The principles of B pixels and G pixels are similar to those of R pixels, and will not be repeated here.

[0038] Since each pixel in the pixel array can only convert one type of light signal into an electrical signal, it is necessary to perform interpolation operations with the light signals collected by other types of pixels around it to restore the image color of the area collected by the pixel. This process is called demosaicing, which is usually completed in the image processor at the back end of the image sensor or the processor of the electronic device containing the image sensor (hereinafter collectively referred to as the processor).

[0039] In order to collect the color information of the image, it is necessary to set a color filter with a specific arrangement above the pixel sensor, which can also be called a color filter array (CFA). At present, the common CFA array is mainly based on the Bayer form of RGB array, which is characterized by its minimum repeating unit being a 2×2 four-pixel array, including 1 R pixel, 1 B pixel and 2 G pixels. The R pixel is diagonally adjacent to the B pixel, and the 2 G pixels are diagonally adjacent. Each pixel obtains one color information in RGB, and the RGB information of each pixel is restored by using a demosaicing algorithm.

[0040] Since the color-specific filters in the Bayer CFA only allow the same type of light to pass through, the amount of light reaching each pixel is reduced, thereby reducing the light sensitivity of each pixel, resulting in poor image quality in dark light scenes. With the development of camera technology in electronic devices such as mobile phones, the requirements for dark light scenes are getting higher and higher, so pixel arrays based on the RGBW CFA form have emerged, which introduce W pixels that allow all wavelengths of light into the RGB array. Since W pixels have a wider spectral response and can receive more photons, compared with Bayer CFA, RGBW CFA has a higher amount of light entering, thereby having a higher sensitivity, which improves the imaging performance in dark light scenes to a certain extent.

[0041] In order to further improve the imaging performance in dark light scenes, a related technology is mentioned. Figure 1 The RGBW CFA shown includes 8×8 minimum repeating units, each of which includes: a 4×4 square BW cluster formed by alternating 8 blue (B) filter units and 8 full-color (W) filter units, a 4×4 square RW cluster formed by alternating 8 red filter units and 8 full-color filter units, a first 4×4 square GW cluster formed by alternating 8 green (G) filter units and 8 full-color filter units, and a second 4×4 square GW cluster formed by alternating 8 G filter units and 8 W filter units, wherein the 4×4 square RW cluster is diagonally adjacent to the 4×4 square BW cluster, the first 4×4 square GW cluster is horizontally adjacent to the 4×4 square BW cluster, and the second 4×4 square GW cluster is diagonally adjacent to the first 4×4 square GW cluster.

[0042] In the image sensor, since the photosensitive units in the photosensitive unit array correspond to the filter units in the filter unit array one by one, the photosensitive units corresponding to the R filter units in the photosensitive unit array output R pixels, the photosensitive units corresponding to the G filter units output G pixels, the photosensitive units corresponding to the B filter units output B pixels, and the photosensitive units corresponding to the W filter units output W pixels, that is, the image sensor can be used to obtain the same Figure 1 The RGBW CFA shown is an RGBW CFA image (hereinafter also referred to as an RGBW image) corresponding to the RGBW CFA.

[0043] The RGBW image includes 8×8 minimum repeating units, each of which includes: a 4×4 square BW cluster formed by alternating 8 B pixels and 8 W pixels, a 4×4 square RW cluster formed by alternating 8 R pixels and 8 W pixels, a first 4×4 square GW cluster formed by alternating 8 G pixels and 8 W pixels, and a second 4×4 square GW cluster formed by alternating 8 G pixels and 8 W pixels, the 4×4 square RW cluster is diagonally adjacent to the 4×4 square BW cluster, the first 4×4 square GW cluster is horizontally adjacent to the 4×4 square BW cluster, and the second 4×4 square GW cluster is diagonally adjacent to the first 4×4 square GW cluster.

[0044] Due to the introduction of the W channel, the RGBW CFA has a larger period and weaker symmetry than the traditional Bayer CFA. Figure 1 The RGBW CFA shown in the figure has an asymmetric distribution of R, G, and B pixels. Traditional demosaicing algorithms are often used for relatively uniform and symmetrical CFA structures such as Bayer. They estimate the local direction by calculating features such as color difference or color ratio in different directions, and then interpolate based on the direction. Due to the asymmetric distribution of R, G, and B pixels of the RGBW CFA, it is difficult to calculate the color difference or color ratio in different directions around it. Interpolation of the RGBW CFA based on the traditional demosaicing algorithm cannot achieve a good interpolation effect.

[0045] To this end, an image processing method is provided in an embodiment of the present application, which can be applied to Figure 1 The asymmetric CFA structure shown in FIG. 1 is a CFA structure in which the interpolation weight is obtained by learning, and the color difference or color ratio does not need to be calculated, thereby avoiding the problem of being unable to calculate the color difference in different directions and achieving a better interpolation effect.

[0046] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0047] Figure 2The following is a flow chart of an image processing method provided in an embodiment of the present application. The image processing method can be executed by an image processor at the back end of the image sensor or other processors of the electronic device where the image sensor is located (hereinafter collectively referred to as processors). Figure 2 As shown, the method includes:

[0048] S201, obtaining an original image.

[0049] The original image includes full-color pixels and color pixels.

[0050] The original image may be a raw (RAW) image captured by the image sensor. The original image captured by the image sensor is a mosaic image consistent with the color filter array pattern on the image sensor, for example, Figure 1 The RGBW image shown has the same RGBW CFA pattern. It should be understood that the original image in this embodiment may also be an original image acquired by an image sensor with other CFAs, and this embodiment does not limit this.

[0051] A full-color pixel refers to a W pixel in an RGBW image, and a color pixel includes an R pixel, a G pixel, and a B pixel in an RGBW image. The location of a full-color pixel in an RGBW image is called a full-color pixel location, and the location of an R pixel, a G pixel, and a B pixel is called a color pixel location.

[0052] The position of each pixel in the original image (also called the pixel position) only has the pixel value of the corresponding color. For example, the W pixel position only has the W pixel value, the R pixel position only has the R pixel value, the G pixel position only has the G pixel value, and the B pixel position only has the B pixel value. The original image needs to restore the pixel values ​​of other colors of each pixel position by using a demosaicing algorithm to generate a color image that can be used for viewing.

[0053] S202: Determine a plurality of positions to be interpolated in the color pixel positions of the original image.

[0054] The position to be interpolated is a pixel position at which an interpolation pixel value of a specific color is to be interpolated among the color pixel positions.

[0055] In one implementation, step S202 includes: determining each color pixel position as a position to be interpolated. Accordingly, W pixel values ​​may be inserted into each color pixel position, so that each pixel position in the original image has W pixel values.

[0056] In another implementation, the color pixel position is divided into a first color pixel position and a non-first color pixel position, wherein the first color pixel position has a first color pixel value, and the non-first color pixel position is a color pixel position other than the first color pixel position in the color pixel position.

[0057] For example, the first color pixel position is a G pixel position, the non-first color pixel position is an R pixel position and a B pixel position, and the first color pixel position has a G pixel value. For another example, the first color pixel position is an R pixel position, the non-first color pixel position is a G pixel position and a B pixel position, and the first color pixel position has an R pixel value. For another example, the first color pixel position is a B pixel position, the non-first color pixel position is an R pixel position and a G pixel position, and the first color pixel position has a B pixel value. Accordingly, step S202 includes: determining each non-first color pixel position as a position to be interpolated. The first color pixel value can be interpolated at each non-first color pixel position, so that the original image is converted into an intermediate image with uniform and symmetrical data distribution including only the first color pixel value and the W pixel value, and then W channel interpolation is performed based on the intermediate image, so as to achieve the purpose of optimizing the W channel interpolation by better utilizing the color information in the original image, and more accurately restore the W pixel value of each color pixel position in the original image.

[0058] It should be understood that in other implementations, other pixel positions in the color pixel position of the original image may be determined as multiple positions to be interpolated according to different interpolation pixel values, and this embodiment does not limit this.

[0059] S203: Based on each position to be interpolated, determine a target image block corresponding to the position to be interpolated.

[0060] Specifically, for each position to be interpolated, an image block of a predetermined size centered at the position to be interpolated in the original image may be determined as a target image block corresponding to the position to be interpolated. Figure 1 For the original image corresponding to the RGBW CFA shown, the image block of the predetermined size may be an image block of 7×7. It should be understood that image blocks of other sizes may also be selected according to actual needs, and this embodiment does not limit this.

[0061] S204 . For each target image block, determine a target filter kernel of the target image block according to feature information of the target image block and a pre-trained relationship model.

[0062] The relationship model includes the correspondence between the feature information of different image blocks and different filter kernels. Each filter kernel in the relationship model includes an interpolation weight applied to each pixel of the corresponding image block. Accordingly, the determined target filter kernel includes an interpolation weight applied to each pixel of the target image block.

[0063] S205 . For each position to be interpolated, filter the target image block corresponding to the position to be interpolated according to the target filter core to obtain an interpolation pixel value of the position to be interpolated.

[0064] Specifically, for each position to be interpolated, a dot product operation is performed on the interpolation weights contained in the target filter kernel and applied to each pixel of the target image block corresponding to the position to be interpolated, and the pixel values ​​of each pixel of the target image block corresponding to the position to be interpolated are obtained to obtain the interpolation pixel values ​​of the pair of interpolation positions.

[0065] In the embodiment of the present application, the target filter kernel corresponding to each target image block is determined by a pre-trained relationship model based on the feature information of the target image block corresponding to each position to be interpolated, that is, the target filter kernel is determined by utilizing the local features of the image region where each position to be interpolated is located, so as to perform filtering processing on the target image block corresponding to each position to be interpolated, and obtain the interpolation pixel value of the position to be interpolated. There is no need to calculate the color difference or color ratio, which avoids the problem of difficulty in calculating the color difference or color ratio in different directions when the distribution symmetry of R, G and B pixels is not strong, and achieves a better interpolation effect.

[0066] based on Figure 2 In another embodiment of the present application, as shown in the embodiment Figure 3 As shown, step S204 includes:

[0067] Step S2041: for each target image block, obtaining an image block type and an image block texture feature of the target image block as feature information of the target image block;

[0068] Step S2042: Determine a target filter kernel corresponding to the target image block by indexing in a pre-trained relationship model according to the feature information.

[0069] Combine the following Figures 4 to 6 The specific process of step S2041 and step S2042 is explained.

[0070] In step 2041, the image block type and image block texture features of the target image block are obtained as feature information of the target image block.

[0071] First, for each target image block, the image block type of the target image block is obtained.

[0072] The image block type is used to indicate the position distribution of each color pixel in the image block. Specifically, the image block type is used to indicate the position distribution of each color pixel in the image block centered on a pixel of a specific color relative to the pixel at the center of the image block.

[0073] In one implementation, the image block type of the target image block may be acquired according to the position distribution of each color pixel in the target image block relative to the position to be interpolated.

[0074] For example, for Figure 1The original image corresponding to the RGBW CFA shown in the figure takes a 7×7 image block centered at the B pixel position, and there are Figure 4 8 image block types (i.e., type 1 to type 8) are shown. In image blocks corresponding to different image block types, the position distribution of each color pixel relative to the B pixel at the center of the image block is different. When the position to be interpolated is the B pixel position, one of the 8 image block types can be used as the image block type corresponding to the target image block at the B pixel position according to the position distribution of each color pixel in the target image block at the B pixel position relative to the position to be interpolated.

[0075] In addition, if Figure 4 As shown in , some of the image blocks corresponding to the eight image block types are symmetrical about 45 degrees and 135 degrees. Figure 5 As shown, there is symmetry between two image blocks a and b. Image block a is rotated 180 degrees and then the rotated image block a is transposed to obtain image block b. For another example, there is symmetry between four image blocks c, d, e, and f. Image block c is rotated 180 degrees and then the rotated image block c is transposed to obtain image block d; image block c is transposed to obtain image block e; image block c is transposed to obtain image block f. For another example, there is symmetry between two image blocks g and h. Image block g is rotated 180 degrees and then the rotated image block g is transposed to obtain image block h. Therefore, due to the symmetry based on the image blocks, by performing transposition operations and / or rotation operations on the image blocks, Figure 4 The eight image block types in are compressed into Figure 5 As the number of image block types decreases, the number of filter kernels stored in the relational model will also decrease accordingly, thereby reducing the storage space occupied by the relational model.

[0076] Accordingly, in one implementation of the present application, according to the position distribution of each color pixel in the target image block relative to the position to be interpolated, obtaining the image block type of the target image block may include: based on the symmetry of the target image block, performing a transformation operation on the target image block to obtain a transformed target image block; according to the position distribution of each color pixel position in the transformed target image block relative to the position to be interpolated corresponding to the target image block, obtaining the image block type of the target image block. The transformation operation includes at least one of a rotation operation and a transposition operation.

[0077] In this implementation, for example, when the position to be interpolated is a B pixel position, the position distribution of each color pixel in the target image block at the B pixel position relative to the position to be interpolated may be calculated. Figure 4 One of the three image block types shown is used as the image block type corresponding to the target image block at the B pixel position.

[0078] It should be noted that Figure 4 and Figure 5 Taking the image block centered at the B pixel position as an example, the meaning and determination method of the image block type are explained. It should be understood that for the sample image block centered at the R pixel position or the G pixel position, the meaning and determination method of the image block type are similar and will not be repeated here.

[0079] In one implementation of the present application, there is a symmetry in the distribution of R pixels and B pixels. Figure 1 In the image corresponding to the RGBW CFA shown, there is a one-to-one correspondence between the image block type of the image block corresponding to the R pixel position and the image block type of the image block corresponding to the B pixel position. That is, by replacing the B pixel in the image block corresponding to the B pixel position with the R pixel and the R pixel with the B pixel, the image block corresponding to the corresponding R pixel position in the original image can be obtained. Therefore, the image block corresponding to the B pixel position and the image block corresponding to the corresponding R pixel position can be regarded as having the same image block type. In this way, the total number of image block types in the relationship model can be reduced, thereby reducing the number of filter kernels in the relationship model, thereby further reducing the storage space occupied by the relationship model.

[0080] It should be understood that Figure 4 and Figure 5 The image block type shown is only an example. For different original images, the size of the image block, the type of the image block and the number of the image blocks may be different, which is not limited in this embodiment.

[0081] Furthermore, for each target image block, an image block texture feature of the target image block is determined.

[0082] In one implementation of the present application, obtaining the image block texture feature of the target image block includes:

[0083] Step A: Calculate the structure tensor of the target image block.

[0084] Among them, the structure tensor is a 2*2 matrix.

[0085] Specifically, when calculating the structure tensor of the target image block, the gradient information of the target image block may be calculated first, for example, the horizontal gradient and the vertical gradient of the R pixel position, the B pixel position, and the G pixel position in the target image block may be calculated. The structure tensor of the target image block is determined according to the horizontal gradient and the vertical gradient of these pixel positions.

[0086] For example, Figure 6 As shown, the horizontal gradients and vertical gradients of the 13 R pixel positions, G pixel positions and B pixel positions in the target image block can be calculated according to formula (1).

[0087]

[0088] Among them, g x and g y Respectively represent the horizontal gradient and vertical gradient of each pixel position, W left and W right W represents the W pixel value of the W pixel position adjacent to the left and the W pixel value adjacent to the right of each pixel position in the horizontal direction, respectively. up and W down They respectively represent the W pixel value of the W pixel adjacent to the upper side and the W pixel adjacent to the lower side of each R, G, and B pixel position in the vertical direction.

[0089] After obtaining the horizontal gradients and vertical gradients of the above 13 R pixel positions, G pixel positions and B pixel positions, the structure tensor of the target image block is calculated according to formula (2).

[0090]

[0091] Wherein, w is the weight of Gaussian distribution, wherein, among the horizontal gradients and vertical gradients of the 13 R pixel positions, G pixel positions, and B pixel positions, the closer the position is to the middle pixel position, the larger the corresponding weight is. S represents the structure tensor of the target image block M.

[0092] Step B: Determine the texture features of the image block according to the structure tensor of the target image block.

[0093] In one implementation of the present application, the texture feature information includes at least one of a direction feature, an intensity feature, and a consistency feature.

[0094] Among them, the directional feature represents the main direction of the gradient in the image block, and the intensity feature and consistency feature represent the intensity of the image block in the main direction of the gradient in the presence and absence of noise, respectively.

[0095] As mentioned above, the structure tensor is a 2×2 matrix S. Through mathematical operations, the two eigenvalues ​​λ1 and λ2 of the matrix S and the corresponding two eigenvectors x1 and x2 can be calculated. Among them, λ1≥λ2, that is, λ1 is the main eigenvalue of the matrix S.

[0096] Next, based on the two eigenvalues ​​λ1 and λ2 of the matrix S and the two corresponding eigenvectors x1 and x2, the directional features, intensity features and consistency features of the target image block can be calculated using formulas (3) to (5).

[0097]

[0098]

[0099]

[0100] Among them, orientation represents the directional characteristics of the target image block; strength represents the strength characteristics of the target image block; coherence represents the consistency characteristics of the target image block; x 1x They are the coordinate values ​​of the eigenvector x1 corresponding to the main eigenvalue λ1 in two dimensions.

[0101] It should be understood that, depending on the different distribution characteristics of pixels of each color in the original image captured by the image sensor, the texture feature information may include other types of features, which is not limited in this embodiment.

[0102] In step S2042, a target filter kernel corresponding to the target image block is determined by indexing in a pre-trained relationship model according to the feature information.

[0103] Specifically, each feature (such as directional feature, intensity feature and consistency feature) included in the texture feature of the image block can be quantified based on multiple preset thresholds to obtain the feature level corresponding to each feature, and the hash value corresponding to the feature information of the target image block is obtained according to these feature levels and the image block type as an index in the pre-trained relational model. Based on the index, the target filter kernel corresponding to the target image block is determined in the relational model.

[0104] In one implementation of the present application, the relationship model can be trained in the following manner:

[0105] Step A1, obtaining N sample image blocks;

[0106] Specifically, multiple sample images having the same pixel distribution image as the original image may be obtained, and the multiple sample images may have different texture features. For each sample image, according to a preset image block type, each pixel position in the sample image is traversed to sample the sample image to obtain multiple sample image blocks, thereby obtaining N sample image blocks.

[0107] Step B1, obtaining the image block type and image block texture features of the sample image blocks as feature information of each sample image block, and dividing the N sample image blocks into M categories by quantizing and grading the feature information, each category having a corresponding index, and N and M being positive integers;

[0108] Specifically, the image block type of each sample image block is obtained, and the image block texture features of each sample image block are obtained in the same manner as the method for obtaining the image block texture features in step S2041. The image block texture features of each sample image block are quantified to obtain the feature levels corresponding to each feature, and based on these feature levels and the image block type of each sample image block, a hash value corresponding to the feature information of each sample image block is obtained as its index. Based on the hash value corresponding to the feature information of each sample image block, all sample image blocks are divided into M categories.

[0109] Step C1: perform least square estimation for each category and calculate the filter kernel corresponding to the category.

[0110] Specifically, the filter kernel corresponding to each category is calculated using the sample image block of each category and the true pixel value of the pixel at the center of the sample image block.

[0111] In one implementation, for each category of image blocks, the filter kernel of each category is calculated by using the least squares estimation method as shown in formula (6).

[0112]

[0113] Among them, A i represents the i-th image block in this category, h represents the filter kernel corresponding to this category, and b i represents the true pixel value at the center of the image block, and L represents the total number of sample image blocks in this category.

[0114] In a similar manner, filter kernels of other categories are calculated to obtain the relationship model. In this implementation, the filter kernel of each category can be directly calculated by least squares estimation to obtain the relationship model. The entire calculation process is simple and time-saving.

[0115] In another implementation of the present application, the relationship model can be trained in the following manner:

[0116] A2, obtaining N sample image blocks;

[0117] B2, obtaining the image block type and image block texture features of the sample image blocks as feature information of each sample image block, and dividing the N sample image blocks into M categories by quantizing and classifying the feature information, each category having a corresponding index, and N and M are positive integers;

[0118] C2. For each category, obtain an initial filter kernel corresponding to the category, and use the initial filter kernel to perform a filtering operation on the sample image block corresponding to the category to obtain a predicted pixel value of a pixel at the center of the sample image block corresponding to the category;

[0119] D2. For each category, the initial filter kernel is adjusted according to the real pixel value and the predicted pixel value of the pixel at the center of the sample image block corresponding to the category to obtain the filter kernel corresponding to the type.

[0120] In a similar manner, other types of filter kernels can be obtained, and then the relationship model can be obtained. In this implementation, the parameters in the initial filter kernel are adjusted iteratively, so that each type of filter kernel determined is more accurate.

[0121] In the embodiment of the present application, the feature information is directly calculated based on the original image, and there is no need to perform bilinear interpolation on the original image, thereby avoiding the introduction of errors in the process of bilinear interpolation, enhancing the robustness of the feature, and reducing the computational complexity.

[0122] like Figure 7 As shown, a flow chart of another image processing method provided by an embodiment of the present application. In the embodiment of the present application, the color pixel positions in the original image are divided into first color pixel positions and non-first color pixel positions. The non-first color pixel positions are determined as positions to be interpolated. The first color pixel values ​​are interpolated at the positions to be interpolated, and the original image is converted into a uniform and symmetrical intermediate image. Thereafter, W channel interpolation is performed based on the intermediate image, and the color information in the original image is better utilized to achieve the purpose of optimizing the W channel interpolation, and the W pixel value of each color pixel position in the original image is more accurately restored. Specifically, as Figure 7 As shown, the method includes:

[0123] S701, obtaining an original image;

[0124] The original image includes full-color pixels and color pixels.

[0125] The method of obtaining the original image and the meanings of full-color pixels and color pixels are similar to those in step S201 and will not be repeated here.

[0126] Similarly, the positions where full-color pixels are located in the original image are called W pixel positions or full-color pixel positions, and the positions where R pixels, G pixels, and B pixels are located are called color pixel positions.

[0127] Wherein, the color pixel position includes a first color pixel position and a non-first color pixel position. Wherein, the first color pixel position has a first color pixel value. The non-first color pixel position is a color pixel position other than the first color pixel position in the color pixel position. For example, the first color pixel position is a G pixel position, the non-first color pixel position is an R pixel position and a B pixel position, and the first color pixel position has a G pixel value. For another example, the first color pixel position is an R pixel position, the non-first color pixel position is a G pixel position and a B pixel position, and the first color pixel position has an R pixel value. For another example, the first color pixel position is a B pixel position, the non-first color pixel position is an R pixel position and a G pixel position, and the first color pixel position has a B pixel value.

[0128] Since usually, the number of G pixels in the original image is relatively high, providing more effective information, while the number of R pixels and B pixels is relatively small, providing less effective information, firstly interpolating the G pixel values ​​at the R pixel position and the B pixel position can obtain a better interpolation effect. Therefore, in one implementation, the first color pixel position is preferably the G pixel position, and correspondingly, the position to be interpolated is the R pixel position and the B pixel position in the color pixel position, and the G pixel value is interpolated at the R pixel position and the B pixel position.

[0129] S702: Determine each non-first color pixel position in the original image as a position to be interpolated.

[0130] S703: Based on each non-first color pixel position, determine a target image block corresponding to the non-first color pixel position.

[0131] Specifically, for each non-first color pixel position, an image block of a predetermined size centered at the non-first color pixel position in the original image may be determined as a target image block corresponding to the non-first color pixel position. Figure 8 In the example given, the target image block M is a 7×7 image block centered at the B pixel position.

[0132] S704 . For each target image block, determine a target filter kernel of the target image block according to feature information of the target image block and a pre-trained relationship model.

[0133] The relationship model includes the correspondence between the feature information of different image blocks and different filter kernels. In this step, each filter kernel in the relationship model includes an interpolation weight applied to each pixel of the corresponding image block when interpolating the first color pixel value at a non-first color pixel position. Accordingly, the determined target filter kernel includes an interpolation weight applied to each pixel of the target image block when interpolating the first color pixel value at a non-first color pixel position corresponding to the target image block.

[0134] It should be noted that step S704 is similar to step S204 in the above embodiment. The specific details can be referred to the above embodiment and will not be described again here.

[0135] In this embodiment, the training process of the relationship model includes: obtaining N sample image blocks; obtaining the image block type and image block texture features of the sample image blocks as feature information of each sample image block, and dividing the N sample image blocks into M categories by quantizing and grading the feature information, each category has a corresponding index, and N and M are positive integers. Least squares estimation is performed for each category, and the filter kernel corresponding to the category is calculated. The specific training process is similar to the training process of the relationship model in the aforementioned embodiment. The specific details can be referred to the aforementioned embodiment and will not be repeated here. It should be understood that since the first color pixel value is interpolated at a non-first color pixel position in this embodiment, the pixel at the center of each sample image block has the first color pixel value during the acquisition process of the relationship model.

[0136] S705. For each non-first color pixel position, use the target filter kernel of the target image block corresponding to the non-first color pixel position to filter the target image block corresponding to the non-first color pixel position to obtain the first color pixel value of the non-first color pixel position, so that each color pixel position in the original image has the first color pixel value.

[0137] For ease of understanding, the following description is made by taking the first color pixel value as a G pixel value and the non-first color pixel position as a B pixel position in the color pixel position as an example. Figure 8 As shown, for the B pixel position with shadow added and the corresponding target image block M. When determining the target filter kernel (in Figure 8 In the example, the target filter kernel is filter kernel 1), and then a filtering operation is performed on the target image block M according to the target filter kernel to obtain a G pixel value at the B pixel position.

[0138] In a similar manner, by traversing the R pixel position and the B pixel position of the original image, the G pixel value of the R pixel position and the B pixel position of the original image can be calculated, because the G pixel position itself has a G pixel value. Therefore, Fig. 9 As shown, the asymmetric original image I1 can be interpolated into a symmetrical and uniform intermediate image I2, in which each color pixel position has a G pixel value and each panchromatic pixel position has a W pixel value. The G pixel value and the W pixel value are evenly and symmetrically distributed.

[0139] It should be understood that Figure 8 and Fig. 9 This is only an example, and in other implementations, the first color pixel position may be an R pixel position or a B pixel position.

[0140] S706. For each color pixel position in the original image, interpolation processing is performed based on first color pixel values ​​of multiple color pixel positions in a first area centered on the color pixel position in the original image and full-color pixel values ​​of multiple full-color pixel positions to obtain a full-color pixel value of the color pixel position.

[0141] Continuing with the example of the first color pixel position being the G pixel position, Fig. 9 As shown, since the G pixel values ​​and W pixel values ​​are evenly and symmetrically distributed in the intermediate image, the W pixel value can be inserted into each color pixel position with a G pixel value in the intermediate image by a conventional interpolation method to obtain the W pixel value of each color pixel position in the original image, that is, to obtain a full-resolution full-color image I 3 obtained by performing W channel interpolation on the original image.

[0142] Specifically, in one implementation, step S706 may include:

[0143] Step A, for each color pixel position in the intermediate image, determining a first area centered at the color pixel position;

[0144] Step B, determining the color difference value of each pixel position including each color pixel position and each panchromatic pixel position based on the first color pixel value of each color pixel position and the W pixel value of each panchromatic pixel position in the first area;

[0145] Step C, determining the color difference values ​​of the color pixel position at the center of the first area in four directions according to the color difference values ​​of each pixel position in the first area; and

[0146] Step D: obtaining the W pixel value of the color pixel position based on the weighted results of the color difference values ​​of the color pixel position at the center of the first area in four directions and the first color pixel value of the color pixel position.

[0147] Refer to the following Fig.10 , the above steps A to D are described in detail. Fig.10 In the figure, the color pixel position marked with G0 is the color pixel position where the W pixel value is to be inserted, and the pixel value of the color pixel position is G0. Fig.10 As shown, the determined first area includes 17 pixel positions. For each of the 13 pixel positions in the middle of the 17 pixel positions, the color difference is roughly calculated using the mean value, for example, the south (S) and north (N) directions use vertical interpolation, and the east (E) and west (W) directions use horizontal interpolation. Specifically, taking the N direction as an example, the color difference value of the color pixel position G0 and its three adjacent pixel positions in the N direction is calculated using formula (7).

[0148]

[0149]

[0150]

[0151]

[0152] Among them, cd0 to cd n3 The color difference values ​​of the color pixel position G0 and its three adjacent pixel positions in the N direction, W n1 , W n3 Respectively represent the W pixel values ​​of two W pixel positions adjacent to the color pixel position G0 in the N direction. n2 , G n4 W respectively represent the G pixel values ​​of two color pixel positions adjacent to the color pixel position G0 in the N direction. s1 Indicates the W pixel value of a W pixel position adjacent to the color pixel position G0 in the S direction.

[0153] After the color difference values ​​of the above four pixel positions are obtained, a weighted sum is performed according to the color difference values ​​of the above four pixel positions to obtain the color difference value in the N direction. The weight of each pixel position can be set based on experience. For example, the weight of the color pixel position G0 is the largest, and the weights of the pixel positions farther away from the color pixel position G0 decrease in sequence. For example, the color difference in the N direction can be calculated according to formula (8).

[0154] cd n =0.4*cd0+0.3*cd n1 +0.2*cd n2 +0.1*cd n3 (8)

[0155] In addition, the gradient in the N direction can also be calculated according to the color difference values ​​of the above four pixel positions. Specifically, the gradient w in the N direction can be calculated according to formula (9): n . Among them, ∈ in formula (9) is a minimum value to prevent the denominator from being zero.

[0156]

[0157] In a similar way, the color difference and weight in the three directions of S, W, and E are calculated, and the weights in the four directions are normalized. n +w s +w w +w e=1, and finally the color difference value of the color pixel position G0 is calculated by weighting the color difference in four directions and the weight, and the color difference value of the color pixel position G0 is added to the G pixel value of the color pixel position G0. Specifically, the W pixel value of the color pixel position G0 can be restored by formula (10).

[0158] W=G+(w n *cd n +w s *cd s +w w *cd w +w e *cd e ) (10)

[0159] In a similar manner, the W pixel value of each color pixel position in the intermediate image can be recovered.

[0160] It should be understood that the W interpolation method given here is only an example. In specific implementation, any other suitable traditional interpolation method can be used, for example, a gradient information selection method is used to interpolate along a single direction. The embodiments of the present application are not limited to this.

[0161] In the embodiment of the present application, the interpolation of the W pixel value is implemented in two steps for the original image. First, the first color pixel value is interpolated in the non-first color pixel position, and the unevenly distributed original image is constructed into an evenly distributed and symmetrical intermediate image, and then the W pixel value is interpolated based on the intermediate image, thereby achieving the purpose of optimizing the W pixel value interpolation by making good use of color information. Fig.11 A comparison diagram of the effects of the image processing method provided in the embodiment of the present application and the traditional interpolation method is shown. Fig.11 The figure on the left shows the effect of the traditional interpolation method. Fig.11 The figure on the right shows the effect of the image processing method provided by the embodiment of the present application. Fig.11 It can be seen that the image processing method provided in the embodiment of the present application can achieve better results in high-frequency details.

[0162] Fig.12 FIG. 2 is a flow chart of another image processing method according to an embodiment of the present application. In the embodiment of the present application, a one-step method is used to directly obtain the W pixel value of each color pixel position. Fig.12 As shown, the method includes:

[0163] S1201, obtaining an original image.

[0164] The original image includes full-color pixels and color pixels.

[0165] The method of obtaining the original image and the meanings of full-color pixels and color pixels are similar to those in step S201 and will not be repeated here.

[0166] Similarly, the positions where full-color pixels are located in the original image are called W pixel positions or full-color pixel positions, and the positions where R pixels, G pixels, and B pixels are located are called color pixel positions.

[0167] S1202: Determine each color pixel position in the original image as a position to be interpolated.

[0168] S1203: Based on each color pixel position, determine a target image block corresponding to the color pixel position.

[0169] Specifically, for each color pixel position, an image block of a predetermined size centered at the color pixel position in the original image may be determined as a target image block corresponding to the color pixel position.

[0170] S1204 . For each target image block, determine a target filter kernel of the target image block according to feature information of the target image block and a pre-trained relationship model.

[0171] The relationship model includes the correspondence between the feature information of different image blocks and different filter kernels. In this step, each filter kernel in the relationship model includes an interpolation weight applied to each pixel of the corresponding image block when interpolating W pixel values ​​at the color pixel positions. Accordingly, the determined target filter kernel includes an interpolation weight applied to each pixel of the target image block when interpolating W pixel values ​​at the color pixel positions corresponding to the target image block.

[0172] It should be noted that step S1204 is similar to step S204 in the above embodiment. The specific details can be referred to the above embodiment and will not be described again here.

[0173] In this embodiment, the training process of the relationship model includes: obtaining N sample image blocks; obtaining the image block type and image block texture features of the sample image blocks as feature information of each sample image block, and dividing the N sample image blocks into M categories by quantizing and grading the feature information, each category has a corresponding index, and N and M are positive integers. Least squares estimation is performed for each category, and the filter kernel corresponding to the category is calculated. The specific training process is similar to the training process of the relationship model in the aforementioned embodiment. The specific details can be referred to the aforementioned embodiment and will not be repeated here. It should be understood that since W pixel values ​​are directly interpolated at each color pixel position in this embodiment, the pixel at the center of each sample image block has W pixel values ​​during the acquisition process of the relationship model.

[0174] S1205 . For each color pixel position, use the target filter core of the target image block corresponding to the color pixel position to perform filtering processing on the target image block corresponding to the color pixel position to obtain a full-color pixel value of the color pixel position.

[0175] Since each color pixel position is a position to be interpolated, the W pixel value corresponding to each color pixel in the original image can be obtained through step S1203.

[0176] In the embodiment of the present application, the pixel value W of each color pixel position is directly obtained in one step, which is similar to the above Figure 7 Compared with the image processing method provided in the illustrated embodiment, the operation process is simpler and more convenient.

[0177] Fig.13 FIG. 2 shows a schematic diagram of the structure of an image processing device according to an embodiment of the present application. Fig.13 As shown, the image processing device 130 includes:

[0178] A first acquisition module 1301 is used to acquire an original image, where the original image includes full-color pixels and color pixels;

[0179] A first determination module 1302 is used to determine a plurality of positions to be interpolated in the color pixel positions of the original image;

[0180] A second determination module 1303 is used to determine, based on each position to be interpolated, a target image block corresponding to the position to be interpolated;

[0181] The third determination module 1304 is used to determine, for each target image block, a target filter kernel corresponding to the target image block according to feature information of the target image block and a pre-trained relationship model;

[0182] The first calculation module 1305 is used to perform filtering processing on the target image block corresponding to each position to be interpolated according to the target filter core to obtain the interpolation pixel value of the position to be interpolated.

[0183] In one implementation of the present application, the color pixel position includes a first color pixel position and a non-first color pixel position, and the first color pixel position has a first color pixel value; accordingly, the first determination module 1302 is further used to determine each non-first color pixel position as a position to be interpolated.

[0184] In another implementation of the present application, the first computing module 1305 is further used to, for each non-first color pixel position, filter the target image block corresponding to the non-first color pixel position using the target filter kernel of the target image block corresponding to the non-first color pixel position to obtain the first color pixel value of the non-first color pixel position, so that each color pixel position in the original image has a first color pixel value.

[0185] In another implementation of the present application, the image processing device also includes a second calculation module, which is used to perform interpolation processing on each color pixel position in the original image based on the first color pixel values ​​of multiple color pixel positions in a first area centered on the color pixel position in the original image and the full-color pixel values ​​of multiple full-color pixel positions to obtain the full-color pixel value of the color pixel position.

[0186] In another implementation of the present application, the first determination module 1302 is further configured to determine each color pixel position as a position to be interpolated.

[0187] In another implementation of the present application, the first calculation module 1305 is further used to filter the target image block corresponding to each color pixel position using the target filter core of the target image block corresponding to the color pixel position to obtain the full-color pixel value of the color pixel position.

[0188] In another implementation of the present application, the third determining module 1304 is further configured to:

[0189] Acquire an image block type and an image block texture feature of a target image block as feature information of the target image block;

[0190] The target filter kernel corresponding to the target image block is determined by indexing in the pre-trained relational model according to the feature information.

[0191] In another implementation of the present application, the third determination module 1304 is further configured to obtain the image block type of the target image block according to the position distribution of each color pixel in the target image block relative to the position to be interpolated.

[0192] In another implementation of the present application, the third determining module 1304 is further configured to:

[0193] Based on the symmetry of the target image block, a transformation operation is performed on the target image block to obtain a transformed target image block;

[0194] The image block type of the target image block is obtained according to the position distribution of each color pixel position in the transformed target image block relative to the to-be-interpolated position corresponding to the target image block.

[0195] In another implementation of the present application, the third determination module 1304 is further configured to perform at least one of a rotation operation and a transposition operation on the target image block.

[0196] In another implementation of the present application, the third determining module 1304 is further configured to:

[0197] Calculate the structure tensor of the target image block;

[0198] According to the structure tensor of the target image block, the texture feature of the image block is determined.

[0199] In another implementation of the present application, the image block texture feature includes at least one of a direction feature, an intensity feature, and a consistency feature.

[0200] In another implementation of the present application, the image processing device further includes a model training module, which is used to:

[0201] Obtain N sample image blocks;

[0202] Obtaining image block types and image block texture features of sample image blocks as feature information of each sample image block, and dividing N sample image blocks into M categories by quantizing and grading the feature information, each category having a corresponding index, where N and M are positive integers;

[0203] Perform least squares estimation for each category and calculate the filter kernel corresponding to the category.

[0204] Fig.14 Schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Fig.14 As shown, the image processing device 140 includes a processor 1401 and a memory 1402 .

[0205] The memory 1402 is used to store computer programs, and the processor 1401 is used to call and run the computer program stored in the memory 1402 to execute any image processing method in the method embodiments.

[0206] The image processing device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. The image processing device can be, for example, an electronic device such as a portable or mobile computing device such as a smart phone, a laptop, a tablet computer, a gaming device, or a part of an electronic device, such as a camera module in an electronic device, which is not limited in the embodiments of the present application.

[0207] An embodiment of the present application provides a chip, which includes a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes an image processing method such as any of the aforementioned methods.

[0208] The chip can be applied to the image processing device in the embodiment of the present application, and the chip can implement the corresponding processes implemented by the image processing device in each method of the embodiment of the present application, and achieve the corresponding effects. For the sake of brevity, it will not be repeated here.

[0209] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0210] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the verification code generation method described herein is implemented. In addition, when a general-purpose computer accesses a code for implementing the verification code generation method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the verification code generation method shown here.

[0211] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0212] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. An image processing method, characterized in that: include: Acquire an original image, wherein the original image includes full-color pixels and color pixels; Determining a plurality of positions to be interpolated in the color pixel positions of the original image; Based on each of the positions to be interpolated, determining a target image block corresponding to the position to be interpolated; For each of the target image blocks, determining a target filter kernel of the target image block according to feature information of the target image block and a pre-trained relationship model; For each of the positions to be interpolated, filtering is performed on the target image block corresponding to the position to be interpolated according to the target filter core to obtain an interpolation pixel value of the position to be interpolated; Wherein, determining the target filter kernel corresponding to the target image block according to the feature information of the target image block and the pre-trained relationship model comprises: Acquire an image block type and an image block texture feature of the target image block as feature information of the target image block; The target filter kernel corresponding to the target image block is determined according to the index of the feature information in a pre-trained relationship model, wherein the pre-trained relationship model contains the correspondence between the index corresponding to the feature information of different image blocks and different filter kernels.

2. The method according to claim 1, characterized in that The step of determining the position to be interpolated in the color pixel position of the original image comprises: The color pixel position includes a first color pixel position and a non-first color pixel position, the first color pixel position having a first color pixel value; Each non-first color pixel position is determined as the to-be-interpolated position.

3. The method according to claim 2, characterized in that For each of the positions to be interpolated, filtering the target image block corresponding to the position to be interpolated according to the target filter core to obtain the interpolation pixel value of the position to be interpolated includes: For each non-first color pixel position, the target image block corresponding to the non-first color pixel position is filtered using the target filter kernel of the target image block corresponding to the non-first color pixel position to obtain the first color pixel value of the non-first color pixel position, so that each color pixel position in the original image has a first color pixel value.

4. The method according to claim 3, characterized in that Also includes: For each color pixel position in the original image, interpolation processing is performed based on the first color pixel values ​​of multiple color pixel positions in a first area centered on the color pixel position in the original image and the full-color pixel values ​​of multiple full-color pixel positions to obtain the full-color pixel value of the color pixel position.

5. The method according to claim 1, characterized in that The step of determining a plurality of positions to be interpolated in the color pixel positions of the original image comprises: Each of the color pixel positions is determined as the position to be interpolated.

6. The method according to claim 5, characterized in that For each of the positions to be interpolated, filtering the target image block corresponding to the position to be interpolated according to the target filter core to obtain the interpolation pixel value of the position to be interpolated includes: For each of the color pixel positions, a target filter kernel of the target image block corresponding to the color pixel position is used to perform filtering processing on the target image block corresponding to the color pixel position to obtain a full-color pixel value of the color pixel position.

7. The method according to claim 1, characterized in that The step of obtaining the image block type of the target image block comprises: The image block type of the target image block is obtained according to the position distribution of each color pixel in the target image block relative to the position to be interpolated.

8. The method according to claim 7, characterized in that Acquiring the image block type of the target image block according to the position distribution of each color pixel in the target image block relative to the position to be interpolated, including: Based on the symmetry of the target image block, performing a transformation operation on the target image block to obtain a transformed target image block; The image block type of the target image block is obtained according to the position distribution of each color pixel position in the transformed target image block relative to the to-be-interpolated position corresponding to the target image block.

9. The method according to claim 8, characterized in that The transforming operation on the target image block comprises: At least one of a rotation operation and a transposition operation is performed on the target image block.

10. The method according to claim 1, characterized in that The acquiring of the image block texture feature of the target image block comprises: Calculating a structure tensor of the target image block; Determine the texture feature of the image block according to the structure tensor of the target image block.

11. The method according to claim 10, characterized in that The image block texture feature includes at least one of a direction feature, an intensity feature and a consistency feature.

12. The method according to claim 10, characterized in that The relationship model is trained in the following way: Obtain N sample image blocks; Obtaining an image block type and an image block texture feature of a sample image block as feature information of each sample image block, and dividing the N sample image blocks into M categories by quantizing and grading the feature information, each category having a corresponding index, wherein N and M are positive integers; A least squares estimation is performed for each category to calculate a filter kernel corresponding to the category.

13. An image processing device, characterized in that: include: An acquisition module, used for acquiring an original image, wherein the original image includes full-color pixels and color pixels; A first determination module, used to determine a plurality of positions to be interpolated in the color pixel positions of the original image; A second determination module, configured to determine, based on each of the positions to be interpolated, a target image block corresponding to the position to be interpolated; A third determination module is used to determine, for each target image block, a target filter kernel corresponding to the target image block according to feature information of the target image block and a pre-trained relationship model; A first calculation module is used for filtering the target image block corresponding to each position to be interpolated according to the target filter core to obtain an interpolation pixel value of the position to be interpolated; Among them, the third determination module is specifically used to obtain the image block type and image block texture features of the target image block as feature information of the target image block; determine the target filter kernel corresponding to the target image block by indexing in a pre-trained relationship model according to the feature information, wherein the pre-trained relationship model contains the correspondence between the indexes corresponding to the feature information of different image blocks and different filter kernels.

14. A chip, characterized in that: include: A processor is used to call and run a computer program from a memory so that a device equipped with the chip executes the image processing method as claimed in any one of claims 1 to 12.

15. An image processing device, characterized in that: The invention comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the image processing method according to any one of claims 1 to 12.

Images