Focus pixel correction method of image sensor, image signal processor and device

Abstract

The application provides a kind of image sensor focusing pixel correction method, image signal processor and imaging equipment.Method includes: obtaining the first image feature information of focusing pixel, and first image feature information includes at least one of the original pixel value and gradient direction of focusing pixel;According to the initial correction pixel value and / or predicted pixel value of focusing pixel of first image feature information;According to the initial correction pixel value and / or predicted pixel value of focusing pixel, obtain the final correction value of focusing pixel, and correct the pixel value of focusing pixel according to final correction value.The application corrects focusing pixel by making full use of the original image information of focusing pixel, can obtain better correction effect, thereby laying a good foundation for improving the final image quality;And the application does not need to increase any hardware cost, with good applicability.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for correcting the focus pixels of an image sensor, an image signal processor, and an imaging device. Background Technology

[0002] PDAF (Phase Detection Auto Focus) is one of the most commonly used technologies for improving image quality in image sensors. Its basic principle is to add dedicated PDAF pixels to the image sensor (such as a CMOS image sensor). By analyzing the distance and changes in these PDAF pixels, the phase difference of the image is detected to determine whether the lens position has reached the optimal state of sharpness, thus achieving focus.

[0003] Please see Figure 1 , Figure 1 A schematic diagram illustrating the focus pixels and normal pixels is provided. From Figure 1 As can be seen, the optical characteristics of the focusing pixel Pd and the normal pixel (image pixel, unless otherwise specified, refers to image pixels not used for focusing) P0 differ. Although phase information can be obtained from the focusing pixel Pd for fast focusing, there are certain drawbacks. This is because the signals of the focusing pixel Pd and the normal pixel P0 often differ, and this difference can affect image quality. Please refer to [link / reference]. Figure 2a , Figure 2b and Figure 2c ,in, Figure 2a This is an example of the original image of one of the scenes when there is no focus pixel; Figure 2b To and Figure 2a A schematic diagram of the original image corresponding to the same scene when there are focus pixels; Figure 2c To and Figure 2b A schematic diagram of the corrected image when using the same focus pixels in the same scene. (Comparison) Figure 2a , Figure 2b and Figure 2c It is easy to see that the signal correction effect of the focus pixels directly affects the quality of the image output.

[0004] In related technologies, when correcting focus pixels, focus pixels are often treated as completely bad pixels, and correction is performed on focus pixels using methods such as median and directional interpolation. However, the characteristics and information of the original signal of the focus pixel are not well utilized, so the correction effect of focus pixels is not very good.

[0005] It should be noted that the information disclosed in the background section of this invention is intended only to enhance the understanding of the general background of this invention, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a method for correcting the focus pixels of an image sensor, an image signal processor, and an imaging device. This invention corrects the focus pixels by making full use of the original image information of the focus pixels, thereby achieving better correction results and laying a good foundation for improving the final image quality. Moreover, this invention does not require any additional hardware costs and has good applicability.

[0007] To achieve the above objectives, the present invention provides a method for correcting the focus pixels of an image sensor, comprising:

[0008] Obtain first image feature information of the focused pixel, wherein the first image feature information includes at least one of the original pixel value and gradient direction of the focused pixel;

[0009] Based on the first image feature information, obtain the initial corrected pixel value and / or predicted pixel value of the focused pixel;

[0010] Based on the initial corrected pixel value and / or the predicted pixel value of the focused pixel, the final corrected value of the focused pixel is obtained, and the pixel value of the focused pixel is corrected based on the final corrected value.

[0011] Optionally, obtaining the initial correction pixel value of the focus pixel based on the first image feature information includes:

[0012] Based on the position information of the focus pixel and the pre-acquired focus pixel area position-correction coefficient lookup table, the correction coefficient of the focus pixel is obtained.

[0013] The initial corrected pixel value is calculated based on the original pixel value of the focused pixel and the correction coefficient.

[0014] Optionally, obtaining the correction coefficient of the focus pixel based on the position information of the focus pixel and a pre-acquired focus pixel region position-correction coefficient lookup table includes:

[0015] Based on the correspondence between the size of the image sensor and the size of the lookup table, the imaging area of ​​the image sensor is divided into several sub-image areas that correspond one-to-one with the focus pixel area positions in the lookup table.

[0016] The correction coefficients of the focused pixels are obtained using either the first or second method below:

[0017] The first method: Based on the position of the sub-image region where the position information of the focus pixel is located and the lookup table, the correction coefficient of the focus pixel is obtained;

[0018] The second method: Based on the position information of the focus pixel, obtain the four candidate sub-image regions whose center point is closest to the focus pixel;

[0019] Based on the position corresponding to each candidate sub-image region and the lookup table, candidate correction coefficients are obtained;

[0020] The four candidate correction coefficients are interpolated using bilinear interpolation to obtain the correction coefficients for the focused pixel.

[0021] Optionally, obtaining the predicted pixel value of the focused pixel based on the first image feature information of the focused pixel includes:

[0022] The predicted pixel value of the focused pixel is calculated using interpolation based on the gradient direction of the focused pixel.

[0023] Optionally, before obtaining the final corrected value of the focused pixel based on the initial corrected pixel value and the predicted pixel value of the focused pixel, the focused pixel correction method further includes:

[0024] The second image feature information of the region where the focus pixel is located is obtained, and the first weight coefficient of the initial correction value and the second weight coefficient of the predicted pixel value are obtained according to the second image feature information; wherein, the sum of the first weight coefficient and the second weight coefficient is 1, the second image feature information is one of a flat region and a textured region, and when the focus pixel is in the flat region, the first weight coefficient is less than the second weight coefficient, and when the focus pixel is in the textured region, the first weight coefficient is greater than the second weight coefficient;

[0025] The step of obtaining the final correction value of the focus pixel based on the initial correction pixel value and the predicted pixel value of the focus pixel includes:

[0026] The final correction value of the focus pixel is calculated based on the initial corrected pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient.

[0027] Optionally, obtaining the second image feature information of the region where the focus pixel is located includes:

[0028] Centered on the focused pixel, the image sampling area is determined according to the preset sampling size;

[0029] Based on the pixel values ​​of all pixels in the image sampling area, calculate the texture feature value of the image sampling area, and determine whether the texture feature value is less than a preset texture feature value: if yes, the area where the focus pixel is located is a flat area; if no, the area where the focus pixel is located is a textured area.

[0030] The texture feature values ​​include the gradient value of the image sampling area, the pixel value variance, or the frequency domain value corresponding to the frequency domain transformation.

[0031] Optionally, before calculating the final corrected value of the focused pixel based on the initial corrected pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient, the focused pixel correction method further includes:

[0032] Calculate the distance between the focused pixel and the center point of the image;

[0033] The third weight coefficient is obtained based on the distance between the focus pixel and the image center point and the pre-acquired distance-fusion weight lookup table;

[0034] The first weight coefficient and the third weight coefficient are fused together, and a fourth weight coefficient for the initial corrected pixel value is obtained based on the fusion result, and a fifth weight coefficient for the predicted pixel value is obtained based on the fourth weight coefficient.

[0035] The step of calculating the final correction value of the focus pixel based on the initial correction pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient includes:

[0036] The final correction value of the focus pixel is calculated based on the initial correction pixel value, the fourth weighting coefficient, the predicted pixel value, and the fifth weighting coefficient.

[0037] Optionally, the step of fusing the first weight coefficient and the third weight coefficient, obtaining a fourth weight coefficient for the initial corrected pixel value based on the fusion result, and obtaining a fifth weight coefficient for the predicted pixel value based on the fourth weight coefficient, includes:

[0038] Determine whether the sum of the first weight coefficient and the third weight coefficient is greater than 1. If so, set the fourth weight coefficient to 1. If not, use the sum of the first weight coefficient and the third weight coefficient as the fourth weight coefficient.

[0039] Calculate the difference between 1 and the fourth weighting coefficient, and use this difference as the fifth weighting coefficient.

[0040] To achieve the above objectives, the present invention also provides an image signal processor, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the focusing pixel correction method of the image sensor described in any of the above claims.

[0041] To achieve the above objectives, the present invention also provides an imaging device, which includes the image signal processor described above.

[0042] Compared with existing technologies, the image sensor focusing pixel correction method, image signal processor, and imaging device provided by this invention have the following advantages:

[0043] This invention achieves better correction results by fully utilizing the original image information of the focus pixels to correct them, thus laying a solid foundation for improving the final image quality. Furthermore, this invention requires no additional hardware costs and has good applicability.

[0044] Furthermore, the present invention uses an interpolation method based on the gradient direction of the focus pixel to obtain the predicted pixel value of the focus pixel, which can effectively ensure the reliability of the predicted pixel value.

[0045] Furthermore, the image sensor focus pixel correction method provided by this invention has the following advantages: when the focus pixel is in a flat region, the accuracy of the predicted pixel is very high, and the predicted pixel value is given a larger weight (reliability, i.e., the first weight coefficient is less than the second weight coefficient); while when the focus pixel is in a textured region (high-frequency region), the predicted pixel value of the current focus pixel obtained by predicting the surrounding pixels has a larger deviation, and the initial correction pixel value is given a larger weight (reliability, i.e., the first weight coefficient is greater than the second weight coefficient). This can further improve the correction effect of the focus pixel, thereby laying a good foundation for improving the quality of the final image. Furthermore, by fusing the initial correction pixel value and the predicted pixel value, the correction effect of the focus pixel can be further improved.

[0046] Furthermore, the image sensor focusing pixel correction method provided by the present invention obtains a third weight coefficient based on the distance between the focusing pixel and the optical center and a pre-acquired distance-fusion weight lookup table, and fuses the first weight coefficient and the third weight coefficient to obtain a fourth weight coefficient for the initial corrected pixel value. Thus, by fine-tuning the first weight coefficient according to the distance between the focusing image and the image center to obtain the fourth weight coefficient for the initial corrected pixel value, and then adjusting the fifth weight coefficient for the predicted pixel value according to the fourth weight coefficient, the correction effect of the focusing pixel can be further improved.

[0047] Since the image signal processor and imaging device provided by this invention belong to the same inventive concept as the image sensor focusing pixel correction method provided by this invention, the image signal processor and imaging device provided by this invention have at least all the advantages of the image sensor focusing pixel correction method provided by this invention. For details on the beneficial effects of the image signal processor and imaging device provided by this invention, please refer to the above description of the beneficial effects of the image sensor focusing pixel correction method provided by this invention, which will not be repeated here. Attached Figure Description

[0048] Figure 1 A schematic diagram of the focused pixels and normal pixels is shown;

[0049] Figure 2a This is an example of the original image of one of the scenes when there is no focus pixel;

[0050] Figure 2b To and Figure 2a A schematic diagram of the original image corresponding to the same scene when there are focus pixels;

[0051] Figure 2c To and Figure 2b A schematic diagram of the corrected image when the same focus pixels are used in the same scene;

[0052] Figure 3a This is a schematic diagram showing the distance between one pair of focal pixels and the optical center of the image sensor;

[0053] Figure 3b This diagram illustrates the correspondence between the pixel values ​​of the left and right phase focus points of the focusing pixel and the pixel values ​​of normal pixels, and the distance between them and the image center.

[0054] Figure 4 This is a structural block diagram of the image signal processor provided in Embodiment 1 of the present invention;

[0055] Figure 5 This is a structural block diagram of the imaging device provided in Embodiment 2 of the present invention;

[0056] Figure 6 This is a specific example diagram of the raw image data processing flow of the imaging device provided in Embodiment 2 of the present invention;

[0057] Figure 7 This is a schematic diagram of the overall process of the image sensor focusing pixel correction method provided in Embodiment 3 of the present invention;

[0058] Figure 8 This is a schematic diagram illustrating the correction principle of a specific example of the image sensor focusing pixel correction method provided in Embodiment 3 of the present invention.

[0059] Figure 9a One example illustrates the correspondence between the original pixel values ​​and normal pixel values ​​of the left and right phase focus points of the focused pixel and the distance between them and the image center.

[0060] Figure 9b One example illustrates the correspondence between the initial corrected pixel values ​​of the left and right phase focus points and the pixel values ​​of the normal pixels, and the distance between them and the image center, after the initial correction of the focus pixels.

[0061] Figure 10a This is a specific example of a lookup table for the location of a focus pixel region and the correction coefficient when obtaining the correction coefficient of the focus pixel using the focus pixel correction method of the image sensor provided by the present invention.

[0062] Figure 10b A schematic diagram illustrating the principle of the image sensor focus pixel correction method provided by the present invention when the correction coefficient of the focus pixel is obtained in the second manner;

[0063] Figure 11a , Figure 11b , Figure 11c and Figure 11d These are schematic diagrams of the Sobel operator's convolution kernels in four different directions when the Sobel operator is used to obtain the gradient direction of the focus pixel in the image sensor correction method provided by the present invention.

[0064] Figure 12 This is a specific example of obtaining the predicted pixel value when the gradient direction of the focused pixel is determined to be vertical, using the image sensor focusing pixel correction method provided by the present invention.

[0065] Figure 13a This is a specific example of the use of the variance method in obtaining the second image feature information of the region where the focus pixel is located when employing the focus pixel correction method of the image sensor provided by the present invention.

[0066] Figure 13b This is another specific example of using the variance method to obtain the second image feature information of the region where the focus pixel is located when employing the focus pixel correction method of the image sensor provided by the present invention.

[0067] Figure 13c A schematic diagram illustrating the principle of determining whether the area where the focus pixel is located is a flat area or a textured area;

[0068] Figure 14 This is a schematic diagram of key steps in a specific example of the image sensor focusing pixel correction method provided in Embodiment 2 of the present invention;

[0069] The accompanying figure is labeled as follows:

[0070] Image signal processor-100, processor-110, memory-120;

[0071] Normal pixels - P0, P1, P2; Focusing pixels - Pd;

[0072] Image sensor-200, pixel array-210, readout circuit-220, timing and system control module-300, output interface-400, application program-500. Detailed Implementation

[0073] The following detailed description, in conjunction with the accompanying drawings, provides a method for correcting the focus pixels of an image sensor, an image signal processor, and an imaging device according to the present invention. The advantages and features of the present invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the embodiments of the present invention. Please refer to the drawings to make the objectives, features, and advantages of the present invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the implementation conditions of the present invention. Any modifications to the structure, changes in proportions, or adjustments to the size, provided that the effects and objectives achieved by the present invention are the same or similar, should still fall within the scope of the technical content disclosed in the present invention. Specific design features of the present invention disclosed herein, including, for example, specific dimensions, orientations, positions, and shapes, will be determined in part by the specific application and usage environment. Furthermore, in the embodiments described below, the same reference numerals are sometimes used across different drawings to denote the same parts or parts having the same function, omitting repeated descriptions. In this specification, similar reference numerals and letters are used to denote similar items; therefore, once an item is defined in one figure, it need not be discussed further in subsequent figures. Furthermore, if the methods described herein involve a series of steps, and the order of these steps presented herein is not necessarily the only possible order in which they can be performed, some of the described steps may be omitted and / or other steps not described herein may be added to the method.

[0074] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The singular forms “a,” “an,” and “the” include plural objects. The term “or” is generally used to mean “and / or,” the term “several” is generally used to mean “at least one,” and the term “at least two” is generally used to mean “two or more.” Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0075] To better understand and explain this invention, the main research process for proposing this invention is described below:

[0076] For example, please see Figure 3a and Figure 3b ,in, Figure 3a This is a schematic diagram showing the distance between one pair of focal pixels and the optical center of the image sensor; Figure 3b This diagram illustrates the correspondence between the pixel values ​​of the left and right phase-detection autofocus points (LPD pixels) and normal pixels, and their distances from the image center. Figure 3a The medium gray area represents the pixel array, the blue dots represent a pair of focal pixels, and the red dots represent the optical center of the image sensor. From... Figure 3b As can be seen, although the pixel values ​​of focused pixels differ from those of normal pixels, they exhibit a certain pattern: the closer the focused pixel is to the optical center, the smaller the difference between its pixel value and that of a normal pixel during imaging; conversely, the farther the focused pixel is from the optical center, the greater the difference between its pixel value and that of a normal pixel. It should be noted that those skilled in the art will understand that there is a corresponding relationship between the distance between the focused pixel and the optical center and the distance between the position of the corresponding focused pixel in the image and the image center during imaging.

[0077] Based on the above research results, the core idea of ​​this invention is to provide a method for correcting the focus pixels of an image sensor, an image signal processor, and an imaging device. This invention can achieve better correction results by making full use of the original image information of the focus pixels to correct them, thereby laying a good foundation for improving the final image quality. Moreover, this invention does not require any additional hardware costs and has good applicability.

[0078] It should be noted that the image sensor focusing pixel correction method provided by this invention can be applied to the image signal processor and imaging device provided by this invention, and the image sensor focusing pixel correction method and image signal processor provided by this invention can be applied to the imaging device provided by this invention. It should be understood that the term "imaging device" or "imaging apparatus" or other similar terms as used herein include general imaging devices, such as, but not limited to, cameras, camcorders, mobile phones, tablet computers, educational devices, and medical imaging devices with image sensors.

[0079] In this document, Embodiment 1 describes the image signal processor provided by the present invention, Embodiment 2 describes the imaging device provided by the present invention, and Embodiment 3 describes the focusing pixel correction method of the image sensor provided by the present invention.

[0080] It should be understood that those skilled in the art should be able to comprehend that although this article uses an image signal processor to perform a focus pixel correction method for an image sensor as an example, this is not a limitation of the present invention, and the focus pixel correction method for an image sensor provided by the present invention does not impose any limitation on the execution subject.

[0081] Example 1

[0082] This embodiment provides an image signal processor (ISP). For example, please refer to [link to example]. Figure 4 , Figure 4 This is a structural block diagram of the image signal processor provided in this embodiment. Figure 4As shown, the image signal processor 100 provided in this embodiment includes a processor 110 and a memory 120. The memory 120 stores a computer program. When the computer program is executed by the processor 110, it implements the focusing pixel correction method for the image sensor provided in any embodiment of the present invention. Since the image signal processor provided in this embodiment and the focusing pixel correction method for the image sensor provided by the present invention belong to the same inventive concept, the image signal processor provided in this embodiment has at least all the advantages of the focusing pixel correction method for the image sensor provided by the present invention. For details, please refer to the relevant description of the beneficial effects of the focusing pixel correction method for the image sensor below. Here, they will not be elaborated one by one. In addition, for the specific content of the focusing pixel correction method for the image sensor, please refer to the relevant embodiments of the focusing pixel correction method for the image sensor below for understanding. To avoid redundancy, they will not be described here.

[0083] It should be noted that the image signal processor 100 provided in this embodiment is mainly used for signal conversion between the image sensor 200 (such as a CMOS sensor) and the application program (such as output to a display device). Its core function is to extract and optimize image information from the raw data output by the image sensor 200, and finally output an image that can be displayed, stored, or further processed. Specifically, the memory 120 can be used to store the computer program. The processor 110 implements various functions of the image signal processor 100 by running or executing the computer program stored in the memory 120 and calling the data stored in the memory 120. These functions include, but are not limited to, black level correction (BLC), defective pixel correction (DPC), phase defective fix (PDF), noise removal, and color adjustment.

[0084] It should be noted that, as those skilled in the art will understand, the present invention does not impose excessive limitations on the processor 110. For example, the processor 110 referred to in the present invention may be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and may also be a microcontroller unit (MCU) or other general-purpose processors. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 110 is the control center of the image signal processor 100, connecting various parts of the entire image signal processor 100 through various interfaces and lines.

[0085] It should also be noted that, as those skilled in the art will understand, the memory 120 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory 120 may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0086] Additionally, computer program code for performing the operations of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages.

[0087] It should be understood that only the parts of the image signal processor 100 related to the present invention have been described above. For more detailed information about the image signal processor 100 not mentioned in this article, please refer to the related technologies of image signal processors known to those skilled in the art. Due to space limitations, this article will not elaborate on these aspects.

[0088] Example 2

[0089] This embodiment provides an imaging device. For example, please refer to... Figure 5 , Figure 5 This is a structural block diagram of the imaging device provided in this embodiment. From... Figure 5 As can be seen, the imaging device provided in this embodiment includes the image signal processor 100, image sensor 200, and application program 700 (APP) provided by this invention. Since the imaging device provided in this embodiment and the image signal processor 100 provided by this invention belong to the same inventive concept, and the image signal processor 100 provided by this invention and the image sensor focusing pixel correction method provided by this invention belong to the same inventive concept, the imaging device provided in this embodiment possesses at least all the advantages of the image sensor focusing pixel correction method provided by this invention. For details, please refer to the following description of the beneficial effects of the image sensor focusing pixel correction method; further details will not be elaborated here.

[0090] Please continue reading Figure 5 ,like Figure 5 As shown, the imaging device further includes a timing and system control module 300 and an output interface 400; furthermore, the image sensor 200 of the imaging device includes a pixel array 210 and a readout circuit 220. Next, in conjunction with... Figure 5 The imaging process of the imaging device is briefly described as follows: Under the coordinated control of the timing and system control module 300, the pixel array 210 converts the acquired raw light signal into an electrical signal. The readout circuit 220 performs analog-to-digital conversion on the electrical signal to obtain raw image data, and sends the converted raw image data to the image signal processor 100. The image signal processor 100 performs processing on the raw image data, including but not limited to black level correction, bad pixel correction, focus pixel correction, and color adjustment, and sends the processed final image data to the output interface 400. The output interface 400 converts and adapts the final image data according to the control of the application program 500 to output the image data to a display screen, memory, or other devices.

[0091] For example, please see Figure 6 , Figure 6This is a specific example diagram illustrating the raw image data processing flow of the imaging device provided in this embodiment. For example... Figure 6 As shown, in some preferred embodiments, the image signal processor 100 processes the raw image data roughly as follows: The raw image data output by the pixel array 210 through the readout circuit 220 is divided into two paths after black level correction (BLC): one path extracts the data corresponding to the focus pixel to obtain phase detection data for use in phase detection focusing; the other path is the image data, which is first subjected to bad pixel correction (DPC), and then the focus pixel correction method of the image sensor provided by this invention is used to repair the data (e.g., pixel value) of the focus pixel to reduce the adverse effect of the focus pixel on the image data, and outputs it as image data for further image processing to obtain the final image. For more detailed information on the focus pixel correction method of the image sensor, please refer to the relevant description of the focus pixel correction method of the image sensor below, which will not be elaborated here.

[0092] Example 3

[0093] This embodiment provides a method for correcting the focus pixels of an image sensor. For example, please refer to [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic diagram illustrating the overall flow of the image sensor focus pixel correction method provided in this embodiment. From... Figure 7 As can be seen, the image sensor focus pixel correction method provided in this embodiment includes:

[0094] S100: Obtain first image feature information of the focused pixel, wherein the first image feature information includes at least one of the original pixel value and gradient direction of the focused pixel;

[0095] S200: Based on the first image feature information, obtain the initial corrected pixel value and / or predicted pixel value of the focused pixel;

[0096] S300: Obtain the final correction value of the focus pixel based on the initial correction pixel value and / or the predicted pixel value of the focus pixel, and correct the pixel value of the focus pixel based on the final correction value.

[0097] The image sensor focus pixel correction method provided by this invention first acquires the first image feature information of the focus pixel, then obtains the initial correction pixel value and / or predicted pixel value of the focus pixel based on the acquired first image feature information, and finally determines the final correction value of the focus pixel based on the initial correction pixel value and / or predicted pixel value. Therefore, this invention, by fully utilizing the original image information of the focus pixel to correct it, can achieve better correction results, thus laying a good foundation for improving the final image quality; moreover, this invention does not require any additional hardware costs and has good applicability.

[0098] For example, in some preferred embodiments, the final correction value of the focused pixel can be obtained based on the initial correction pixel value of the focused pixel, and then the pixel value of the focused pixel can be corrected based on the final correction value. Since the initial correction pixel value of the focused pixel is obtained based on the first image feature information of the focused pixel (such as the original pixel value of the focused pixel), the present invention can reduce the signal difference between the focused pixel and the normal pixel, improve the correction effect of the focused pixel, and thus lay a good foundation for improving the final image quality.

[0099] For example, in some other preferred embodiments, the final correction value of the focus pixel can be obtained based on the predicted pixel value of the focus pixel, and then the pixel value of the focus pixel can be corrected based on the final correction value. Since the predicted pixel value of the focus pixel is obtained based on the first image feature information of the focus pixel (such as the gradient direction of the focus pixel), the present invention can reduce the signal difference between the focus pixel and the normal pixel, improve the correction effect of the focus pixel, and thus lay a good foundation for improving the final image quality.

[0100] For example, please see Figure 8 , Figure 8 This is a schematic diagram illustrating the correction principle of a specific example of the image sensor focus pixel correction method provided in this embodiment. From Figure 8As can be seen, in this example, the focus pixel correction mainly includes two parts: initial correction and fine-tuning correction. Initial correction utilizes the original pixel value of the focus pixel to initially reduce the difference between the focus pixel and normal pixels. Fine-tuning correction uses the gradient direction of the focus pixel, image features of the area it is located in, and / or the distance between the focus pixel and the image center point to fine-tune the initial correction, thereby improving the correction effect of the focus pixel. Specifically, the initial correction pixel value and predicted pixel value of the focus pixel can be obtained first based on the first image feature information. Then, based on the initial correction pixel value and the predicted pixel value (for example, by performing a weighted operation on the initial correction pixel value and the predicted pixel value), the final correction value of the focus pixel is obtained, and the pixel value of the focus pixel is corrected based on the final correction value. Since the initial corrected pixel value and predicted pixel value of the focus pixel are obtained based on the first image feature information of the focus pixel (such as the original pixel value and gradient direction of the focus pixel), the present invention can further reduce the signal difference between the focus pixel and the normal pixel, significantly improve the correction effect of the focus pixel, and thus lay a good foundation for improving the final image quality.

[0101] Preferably, in some exemplary embodiments, the step S200 of obtaining the initial correction pixel value of the focus pixel based on the first image feature information includes:

[0102] S211: Obtain the correction coefficient of the focus pixel based on the position information of the focus pixel and the pre-acquired focus pixel area position-correction coefficient lookup table;

[0103] S212: Calculate the initial corrected pixel value based on the original pixel value of the focused pixel and the correction coefficient.

[0104] The image sensor focusing pixel correction method provided by this invention obtains the correction coefficient through a lookup table. This not only utilizes the correlation between the correction coefficient and the position of the focusing pixel to lay the foundation for improving the correction effect of the initial correction pixel value, but also has high correction efficiency through the lookup table method. Furthermore, the initial correction pixel value is obtained based on the correction coefficient and the original pixel value of the focusing pixel, thus it can compensate the initial correction pixel value of the focusing pixel to be close to the normal value, thereby effectively improving the correction effect of the focusing pixel.

[0105] For example, please see Figure 9a and Figure 9b ,in, Figure 9a One example illustrates the correspondence between the original pixel values ​​and normal pixel values ​​of the left and right phase focus points of the focused pixel and the distance between them and the image center. Figure 9bOne example illustrates the correspondence between the initial corrected pixel values ​​and normal pixel values ​​of the left and right phase focus points after initial focus pixel correction, and the distances between these values ​​and the image center. Figure 9a It can be seen that the original pixel values ​​of the left and right phase-detection autofocus points differ significantly from the pixel values ​​of the normal pixels. Figure 9b It can be seen that the initial corrected pixel values ​​of the left and right phase focus points of the focusing pixels are relatively close to the pixel values ​​of the normal pixels.

[0106] Preferably, in some exemplary embodiments, step S211, which involves obtaining the correction coefficient of the focus pixel based on the position information of the focus pixel and a pre-acquired focus pixel region position-correction coefficient lookup table, includes:

[0107] Based on the correspondence between the size of the image sensor and the size of the lookup table, the imaging area of ​​the image sensor is divided into several sub-image areas that correspond one-to-one with the focus pixel area positions in the lookup table.

[0108] The correction coefficients of the focused pixels are obtained using either the first or second method below:

[0109] The first method: Based on the position of the sub-image region where the position information of the focus pixel is located and the lookup table, the correction coefficient of the focus pixel is obtained;

[0110] The second method: Based on the position information of the focus pixel, obtain the four candidate sub-image regions whose center point is closest to the focus pixel;

[0111] Based on the position corresponding to each candidate sub-image region and the lookup table, candidate correction coefficients are obtained;

[0112] The four candidate correction coefficients are interpolated using bilinear interpolation to obtain the correction coefficients for the focused pixel.

[0113] Therefore, the image sensor focusing pixel correction method provided by the present invention adopts a grid design for the focusing pixel area position of the lookup table, which can not only effectively reduce the storage space occupied by the lookup table, but also reasonably set the size of the focusing pixel area of ​​the lookup table according to the need for correction accuracy, thereby effectively improving the applicability of the present invention.

[0114] It should be noted that, as those skilled in the art will understand, compared to the second method, the first method has higher correction efficiency in obtaining the initial correction pixel value of the focus pixel; and compared to the first method, the second method has a better correction effect in obtaining the initial correction pixel value of the focus pixel. In practical applications of this invention, either the first or second method can be selected to obtain the initial correction pixel value according to actual needs, and this invention does not impose any limitations on this.

[0115] For example, please see Figure 10a and Figure 10b ,in, Figure 10a This is a specific example of a lookup table for the location of a focus pixel region and the correction coefficient when obtaining the correction coefficient of the focus pixel using the focus pixel correction method of the image sensor provided by the present invention. Figure 10b This is a schematic diagram illustrating the principle of the image sensor focus pixel correction method provided by the present invention when obtaining the correction coefficient of the focus pixel using the second method. From Figure 10a As can be seen, taking a 48-megapixel image sensor as an example, the size of each focus pixel area is 512×512 pixels. Therefore, the focus pixel area position-correction coefficient lookup only requires 16×12 squares to cover the entire image. Furthermore, if the first method is used, assuming the correction coefficient corresponding to the position of the sub-image area where the focus pixel is located in the lookup table is k0, then the correction coefficient of that focus pixel is k0. If the second method is used, such as... Figure 10b As shown, if the correction coefficients corresponding to the four candidate sub-image regions whose center point is closest to the focus pixel are k1, k2, k3, and k4 respectively in the lookup table, then k1, k2, k3, and k4 are used as candidate correction coefficients. Bilinear interpolation is then used to interpolate the four candidate correction coefficients k1, k2, k3, and k4 to obtain the correction coefficient of the focus pixel. For example, the initial correction pixel value of the focus pixel can be obtained using the following formula:

[0116] gain target =bilinear interpolation by(k1~k4)

[0117] pre correction_val =rawdata target *gain target

[0118] In the above formula, gain target The correction coefficient for the focused pixel is...

[0119] bilinear interpolation by (k1~k4) represents a bilinear interpolation operation performed on the four candidate correction coefficients k1, k2, k3, and k4. correction_val The initial correction pixel value for the focused pixel, rawdata target The original pixel value of the focused pixel.

[0120] It should be noted that those skilled in the art will understand that the present invention does not impose excessive limitations on the number of candidate sub-image regions; furthermore, when performing bilinear interpolation, the weight of each candidate correction coefficient is not excessively limited. In a preferred embodiment, when calculating the correction coefficient, the weight of the candidate correction coefficient corresponding to the candidate sub-image region whose center point is closer to the focus pixel can be set larger, and the weight of the region farther away can be set smaller. Furthermore, the present invention does not impose excessive limitations on the specific implementation method for calculating the correction coefficient, and methods including but not limited to polynomial interpolation, nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation can be used.

[0121] Preferably, in some exemplary embodiments, step S200, which involves obtaining the predicted pixel value of the focus pixel based on the first image feature information of the focus pixel, includes:

[0122] S221: Based on the gradient direction of the focused pixel, the predicted pixel value of the focused pixel is calculated using an interpolation method.

[0123] Therefore, the present invention uses interpolation based on the gradient direction of the focus pixel to obtain the predicted pixel value of the focus pixel, which can effectively ensure the reliability of the predicted pixel value.

[0124] It should be noted that those skilled in the art should understand that the present invention does not impose excessive limitations on the specific implementation method for obtaining the gradient direction of the focus pixel. For example, edge detection operators, including but not limited to the Sobel operator and the Kirsch operator, can be used to perform edge detection to determine the gradient direction of the focus pixel. For more detailed information on how to determine the gradient direction of the focus pixel, please refer to relevant technologies known to those skilled in the art; due to space limitations, this will not be elaborated upon here.

[0125] For example, please see Figures 11a-11d and Figure 12 , Figure 11a , Figure 11b , Figure 11c and Figure 11d These are schematic diagrams of the Sobel operator's convolution kernels in four different directions when the Sobel operator is used to obtain the gradient direction of the focus pixel in the image sensor correction method provided by the present invention. Figure 12 This is a specific example diagram illustrating the acquisition of predicted pixel values ​​when the gradient direction of the focused pixel is determined to be vertical, using the image sensor focus pixel correction method provided by this invention. (See diagram for example.) Figure 12 As shown, if the texture direction of the focus pixel Pd detected by the Sobel operator is vertical, the predicted pixel value of the focus pixel can be calculated using the following formula:

[0126] P dir_V =(P1+P2) / 2

[0127] In the above formula, P dir_V P1 is the predicted pixel value of the focused pixel Pd, and P2 is the pixel value of the normal pixels P1 and P2, respectively.

[0128] Preferably, in some exemplary embodiments, before obtaining the final corrected value of the focused pixel based on the initial corrected pixel value and the predicted pixel value of the focused pixel, the focused pixel correction method further includes:

[0129] The second image feature information of the region where the focus pixel is located is obtained, and the first weight coefficient of the initial correction value and the second weight coefficient of the predicted pixel value are obtained according to the second image feature information; wherein, the sum of the first weight coefficient and the second weight coefficient is 1, the second image feature information is one of a flat region and a textured region, and when the focus pixel is in the flat region, the first weight coefficient is less than the second weight coefficient, and when the focus pixel is in the textured region, the first weight coefficient is greater than the second weight coefficient.

[0130] Correspondingly, step S300, which involves obtaining the final correction value of the focus pixel based on the initial correction pixel value and the predicted pixel value, includes:

[0131] The final correction value of the focus pixel is calculated based on the initial corrected pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient.

[0132] Therefore, the image sensor focus pixel correction method provided by this invention has the following advantages: when the focus pixel is in a flat region, the accuracy of the predicted pixel is very high, and a larger weight (reliability, i.e., the first weight coefficient is less than the second weight coefficient) is assigned to the predicted pixel value. However, when the focus pixel is in a textured region (high-frequency region), the predicted pixel value of the current focus pixel obtained from the surrounding pixels has a larger deviation, and a larger weight (reliability, i.e., the first weight coefficient is greater than the second weight coefficient) is assigned to the initial correction pixel value. This further improves the correction effect of the focus pixel, thus laying a good foundation for improving the quality of the final image. Furthermore, by fusing the initial correction pixel value and the predicted pixel value, the correction effect of the focus pixel can be further improved.

[0133] Furthermore, the specific values ​​of the first weighting coefficient and the second weighting coefficient can be reasonably selected based on the texture features of the textured region. Preferably, the more prominent the texture features (the more dramatic the image changes, i.e., the richer the image), the larger the value of the first weighting coefficient and the smaller the value of the second weighting coefficient; the less prominent the texture features (the flatter the image, the more monotonous the image), the smaller the value of the first weighting coefficient and the larger the value of the second weighting coefficient. For example, in some embodiments, the first weighting coefficient is determined according to whether the image region where the focused pixel is located is a flat region or a textured region, in the following way:

[0134]

[0135] Among them, 0 <a<1

[0136] In the above formula, This is the first weighting coefficient.

[0137] Preferably, in some exemplary embodiments, obtaining the second image feature information of the region where the focus pixel is located includes:

[0138] Centered on the focused pixel, the image sampling area is determined according to the preset sampling size;

[0139] Based on the pixel values ​​of all pixels in the image sampling area, calculate the texture feature value of the image sampling area, and determine whether the texture feature value is less than a preset texture feature value: if yes, the area where the focus pixel is located is a flat area; if no, the area where the focus pixel is located is a textured area.

[0140] The texture feature values ​​include the gradient value of the image sampling area, the pixel value variance, or the frequency domain value corresponding to the frequency domain transformation.

[0141] Therefore, the image sensor focusing pixel correction method provided by the present invention first determines the image sampling area based on the preset sampling size with the focusing pixel as the center, then obtains the texture feature value of the image sampling area, and compares it with the texture feature value and the preset texture feature value to determine whether the area where the focusing pixel is located is a flat area or a textured area. The implementation method is simple in logic, easy to implement and highly reliable.

[0142] For example, please see Figure 13a , Figure 13b and Figure 13c ,in, Figure 13a This is a specific example of the use of the variance method in obtaining the second image feature information of the region where the focus pixel is located when employing the focus pixel correction method of the image sensor provided by the present invention. Figure 13b This is another specific example of using the variance method to obtain the second image feature information of the region where the focus pixel is located when employing the focus pixel correction method of the image sensor provided by the present invention. Figure 13c This is a schematic diagram illustrating the principle of determining whether the area where the focus pixel is located is a flat area or a textured area. Figure 13a and Figure 13b In the example, a preset sampling size of 5×5 pixels is used, and texture feature values ​​are calculated using the green channel. More specifically, Figure 13a In the image, the channel containing the focused pixel is the green channel. Figure 13b In the image, the channel containing the focused pixel is the blue channel. Further, the texture feature value is calculated using the following formula:

[0143] P′ i =sort(P i i∈1~12

[0144] avg = sum(P′) i ) / 10i∈2~11

[0145] var = sum(abs(P) i ′-avg)) / 10 i∈2~11

[0146] In the above formula, P i For the image sampling area (e.g. Figure 13a or Figure 13b The pixel value of the i-th green channel in the texture is AVG, where AVG is the average of the pixel values ​​of the other green channels after removing the maximum and minimum pixel values, and var is the texture feature value.

[0147] It should be noted that those skilled in the art should understand that the present invention does not limit the specific method for obtaining the second image feature information of the region where the focus pixel is located. For example, the second image feature information of the region where the focus pixel is located can be obtained by methods including but not limited to gradient values ​​of gradient direction information, pixel values ​​of variance information, or frequency domain transformation or frequency domain values. For more detailed information on obtaining the second image feature information of the region where the focus pixel is located, please refer to the relevant technical adaptations for obtaining image texture feature information known in the art, which will not be elaborated here.

[0148] Preferably, in some exemplary embodiments, before calculating the final corrected value of the focused pixel based on the initial corrected pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient, the focused pixel correction method further includes:

[0149] SA1: Calculate the distance between the focused pixel and the center point of the image.

[0150] SA2: Based on the distance between the focused pixel and the image center point and the pre-acquired distance-fusion weight lookup table, the third weight coefficient is obtained;

[0151] SA3: The first weight coefficient and the third weight coefficient are fused together, and a fourth weight coefficient for the initial corrected pixel value is obtained based on the fusion result, and a fifth weight coefficient for the predicted pixel value is obtained based on the fourth weight coefficient.

[0152] Correspondingly, step S300, which calculates the final correction value of the focus pixel based on the initial corrected pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient, includes:

[0153] The final correction value of the focus pixel is calculated based on the initial correction pixel value, the fourth weighting coefficient, the predicted pixel value, and the fifth weighting coefficient.

[0154] Therefore, the image sensor focusing pixel correction method provided by the present invention obtains a third weight coefficient based on the distance between the focusing pixel and the optical center and a pre-acquired distance-fusion weight lookup table, and fuses the first weight coefficient and the third weight coefficient to obtain a fourth weight coefficient of the initial corrected pixel value. Thus, by fine-tuning the first weight coefficient according to the distance between the focusing pixel and the image center to obtain the fourth weight coefficient of the initial corrected pixel value, and then adjusting the fifth weight coefficient of the predicted pixel value according to the fourth weight coefficient, the correction effect of the focusing pixel can be further improved.

[0155] It should be noted that those skilled in the art should understand that the present invention does not limit the specific implementation of calculating the distance between the focus pixel and the center point of the image.

[0156] For example, please combine Figure 3a In some exemplary embodiments, the distance between the focused pixel and the image center point can first be calculated using the following formula:

[0157] dis=abs(row-optcal_center.row)+abs(col-optcal_center.col)

[0158] In the above formula, dis is the distance between the focused pixel and the center point of the relationship (as mentioned above, it can correspond to the distance between the focused pixel in the image and the center of the image), row is the coordinate value of the focused pixel in the x-direction, optcal_center.row is the coordinate value of the optical center in the x-direction, col is the coordinate value of the focused pixel in the y-direction, and optcal_center.col is the coordinate value of the optical center in the y-direction.

[0159] Then, based on the pre-acquired distance-fusion weight lookup table, the third weight coefficient (blend ratio, hereinafter referred to as...) can be obtained by looking up the table. (Identification).

[0160] It is understood that the distance calculation method between the focusing pixel and the optical center point described above is merely an exemplary description of a preferred embodiment and not a limitation of the present invention. In other embodiments, the distance can also be obtained by conventionally calculating the distance between the two points using the coordinates of the focusing pixel and the optical center point. In other embodiments, the distance of the focusing pixel can be pre-stored in memory for direct retrieval when needed. Furthermore, those skilled in the art should understand that the present invention does not impose excessive limitations on the specific acquisition method and content of the distance-fusion weight lookup table. For example, in some preferred embodiments, it can be obtained through calibration. Furthermore, the closer the focusing pixel is to the optical center, the smaller the deviation of the original pixel value of the focusing pixel, the more accurate the result of the initial corrected pixel value of the focusing pixel, and the greater the reliability of the initial corrected pixel value; the farther the focusing pixel is from the optical center, the greater the deviation of the original pixel value of the focusing pixel, and the greater the reliability of the predicted pixel value. Therefore, this principle can be used to reasonably set the correspondence between distance and weight in the distance-fusion weight lookup table.

[0161] Preferably, in some exemplary embodiments, step SA3 involves fusing the first weighting coefficient and the third weighting coefficient, obtaining a fourth weighting coefficient for the initial corrected pixel value based on the fusion result, and obtaining a fifth weighting coefficient for the predicted pixel value based on the fourth weighting coefficient, including:

[0162] SA31: Determine whether the sum of the first weight coefficient and the third weight coefficient is greater than 1. If yes, then set the fourth weight coefficient to 1. If no, then use the sum of the first weight coefficient and the third weight coefficient as the fourth weight coefficient.

[0163] SA32: Calculate the difference between 1 and the fourth weighting coefficient, and use this difference as the fifth weighting coefficient.

[0164] Therefore, the image sensor focusing pixel correction method provided by the present invention uses the sum of the first weighting coefficient and the second weighting coefficient as the fourth weighting coefficient of the initial correction value, which is logically simple and easy to implement.

[0165] Exemplary, in some exemplary embodiments, the fourth weighting coefficient is calculated by the following formula:

[0166]

[0167] In the above formula, The fourth weighting coefficient, The third weighting coefficient, The first weighting coefficient, Indicates if If it is less than 0, the result is 0. If it is greater than 1, the result is 1; if If ≥0 and ≤1, then the result is

[0168] Correspondingly, the final correction value of the focused pixel is calculated using the following formula:

[0169]

[0170] In the above formula, corValue final corPre is the final correction value for the focused pixel, corFine is the initial correction pixel value, and corFine is the predicted pixel value. The fourth weighting coefficient, This refers to the fifth weighting coefficient.

[0171] For example, please see Figure 14 , Figure 14This diagram illustrates key steps of a specific example of the focus pixel correction method for an image sensor provided by the present invention. Figure 14 As can be seen, in this example, the entire process mainly consists of two parts: initial correction, which is primarily used to obtain initial correction pixel values; and fine-tuning correction, which is primarily used to obtain predicted pixel values ​​and calculate weight coefficients. Further, as... Figure 14 As shown, the general process of fine-tuning correction is as follows: First, the gradient direction information of the location of the focus pixel is obtained through direction detection. Then, interpolation is performed based on the gradient direction information to obtain the predicted pixel value of the focus pixel. Second, the regional features (texture region (high-frequency region) or flat region) of the image area where the focus pixel is located are detected based on gradient direction information or variance information. Third, the distance between the focus pixel and the optical center is calculated (that is, the distance between the focus pixel point corresponding to the focus pixel in the image and the image center). Finally, the weight coefficients of the initial correction pixel value and the predicted pixel value are obtained based on whether the focus pixel is in a texture region and its distance from the optical center. Finally, the final correction value of the focus pixel is obtained based on the initial correction pixel value, the predicted pixel value, and their respective weight coefficients.

[0172] In summary, compared with the prior art, the image sensor focusing pixel correction method, image signal processor, and imaging device provided by the present invention have the following advantages:

[0173] (1) By making full use of the original image information of the focus pixel to correct the focus pixel, the present invention can achieve better correction effect, thereby laying a good foundation for improving the final image quality; and the present invention does not require any additional hardware cost and has good applicability.

[0174] (2) The present invention uses an interpolation method based on the gradient direction of the focus pixel to obtain the predicted pixel value of the focus pixel, which can effectively ensure the reliability of the predicted pixel value.

[0175] (3) The image sensor focusing pixel correction method provided by this invention has the following advantages: When the focusing pixel is in a flat area, the accuracy of the predicted pixel is very high, and the predicted pixel value is given a greater weight (reliability, i.e., the first weight coefficient is less than the second weight coefficient); while when the focusing pixel is in a textured area (high-frequency area), the predicted pixel value of the current focusing pixel obtained by predicting the surrounding pixels has a large deviation, and the initial correction pixel value is given a greater weight (reliability, i.e., the first weight coefficient is greater than the second weight coefficient). This can further improve the correction effect of the focusing pixel, thereby laying a good foundation for improving the quality of the final image. Furthermore, by fusing the initial correction pixel value and the predicted pixel value, the correction effect of the focusing pixel can be further improved.

[0176] (4) The image sensor focusing pixel correction method provided by the present invention obtains a third weight coefficient based on the distance between the focusing pixel and the optical center and a pre-acquired distance-fusion weight lookup table, and fuses the first weight coefficient and the third weight coefficient to obtain a fourth weight coefficient of the initial corrected pixel value. Thus, by fine-tuning the first weight coefficient according to the distance between the focusing pixel and the image center, the fourth weight coefficient of the initial corrected pixel value is obtained. Then, the fifth weight coefficient of the predicted pixel value is obtained by adjusting the fourth weight coefficient, which can further improve the correction effect of the focusing pixel.

[0177] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0178] In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0179] The above description is merely a preferred embodiment of the image sensor focusing pixel correction method, image signal processor, and imaging device provided by the present invention, and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for correcting the focus pixels of an image sensor, characterized in that, include: Obtain first image feature information of the focused pixel, wherein the first image feature information includes at least one of the original pixel value and gradient direction of the focused pixel; Based on the first image feature information, obtain the initial corrected pixel value and / or predicted pixel value of the focused pixel; Based on the initial corrected pixel value and / or the predicted pixel value of the focused pixel, the final corrected value of the focused pixel is obtained, and the pixel value of the focused pixel is corrected based on the final corrected value.

2. The image sensor focusing pixel correction method according to claim 1, characterized in that, The step of obtaining the initial correction pixel value of the focus pixel based on the first image feature information includes: Based on the position information of the focus pixel and the pre-acquired focus pixel area position-correction coefficient lookup table, the correction coefficient of the focus pixel is obtained. The initial corrected pixel value is calculated based on the original pixel value of the focused pixel and the correction coefficient.

3. The image sensor focusing pixel correction method according to claim 2, characterized in that, The step of obtaining the correction coefficient of the focus pixel based on the position information of the focus pixel and a pre-acquired focus pixel region position-correction coefficient lookup table includes: Based on the correspondence between the size of the image sensor and the size of the lookup table, the imaging area of ​​the image sensor is divided into several sub-image areas that correspond one-to-one with the focus pixel area positions in the lookup table. The correction coefficients of the focused pixels are obtained using either the first or second method below: The first method: Based on the position of the sub-image region where the position information of the focus pixel is located and the lookup table, the correction coefficient of the focus pixel is obtained; The second method: Based on the position information of the focus pixel, obtain the four candidate sub-image regions whose center point is closest to the focus pixel; Based on the position corresponding to each candidate sub-image region and the lookup table, candidate correction coefficients are obtained; The four candidate correction coefficients are interpolated using bilinear interpolation to obtain the correction coefficients for the focused pixel.

4. The image sensor focusing pixel correction method according to claim 1, characterized in that, The step of obtaining the predicted pixel value of the focused pixel based on the first image feature information of the focused pixel includes: Based on the gradient direction of the focused pixel, the predicted pixel value of the focused pixel is calculated using an interpolation method.

5. The image sensor focusing pixel correction method according to claim 1, characterized in that, Before obtaining the final correction value of the focus pixel based on the initial correction pixel value and the predicted pixel value, the focus pixel correction method further includes: The second image feature information of the region where the focus pixel is located is obtained, and the first weight coefficient of the initial correction value and the second weight coefficient of the predicted pixel value are obtained according to the second image feature information; wherein, the sum of the first weight coefficient and the second weight coefficient is 1, the second image feature information is one of a flat region and a textured region, and when the focus pixel is in the flat region, the first weight coefficient is less than the second weight coefficient, and when the focus pixel is in the textured region, the first weight coefficient is greater than the second weight coefficient; The step of obtaining the final correction value of the focus pixel based on the initial correction pixel value and the predicted pixel value includes: The final correction value of the focus pixel is calculated based on the initial corrected pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient.

6. The image sensor focusing pixel correction method according to claim 5, characterized in that, The step of obtaining the second image feature information of the region where the focus pixel is located includes: Centered on the focused pixel, the image sampling area is determined according to the preset sampling size; Based on the pixel values ​​of all pixels in the image sampling area, calculate the texture feature value of the image sampling area, and determine whether the texture feature value is less than a preset texture feature value: if yes, the area where the focus pixel is located is a flat area; if no, the area where the focus pixel is located is a textured area. The texture feature values ​​include the gradient value of the image sampling area, the pixel value variance, or the frequency domain value corresponding to the frequency domain transformation.

7. The image sensor focusing pixel correction method according to claim 5, characterized in that, Before calculating the final correction value of the focused pixel based on the initial corrected pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient, the focused pixel correction method further includes: Calculate the distance between the focused pixel and the center point of the image; The third weight coefficient is obtained based on the distance between the focus pixel and the image center point and the pre-acquired distance-fusion weight lookup table; The first weight coefficient and the third weight coefficient are fused together, and a fourth weight coefficient for the initial corrected pixel value is obtained based on the fusion result, and a fifth weight coefficient for the predicted pixel value is obtained based on the fourth weight coefficient. The step of calculating the final correction value of the focus pixel based on the initial correction pixel value, the first weighting coefficient, the predicted pixel value, and the second weighting coefficient includes: The final correction value of the focus pixel is calculated based on the initial correction pixel value, the fourth weighting coefficient, the predicted pixel value, and the fifth weighting coefficient.

8. The image sensor focusing pixel correction method according to claim 7, characterized in that, The step of fusing the first weight coefficient and the third weight coefficient, obtaining a fourth weight coefficient for the initial corrected pixel value based on the fusion result, and obtaining a fifth weight coefficient for the predicted pixel value based on the fourth weight coefficient includes: Determine whether the sum of the first weight coefficient and the third weight coefficient is greater than 1. If so, set the fourth weight coefficient to 1. If not, use the sum of the first weight coefficient and the third weight coefficient as the fourth weight coefficient. Calculate the difference between 1 and the fourth weighting coefficient, and use this difference as the fifth weighting coefficient.

9. An image signal processor, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the focus pixel correction method for the image sensor as described in any one of claims 1 to 8.

10. An imaging device, characterized in that, Includes the image signal processor as described in claim 9.

Images