Defective pixel screening method, image sensor, and computer program product

Abstract

The application belongs to the technical field of image sensors, and provides a defective pixel screening method, an image sensor and a computer program product. The method comprises the following steps: acquiring image data of an image sensor, and screening defective pixel clusters with sizes conforming to repair rules of a defective pixel repair module from the image data; a defective pixel cluster is sequentially selected according to a preset size order, if the defective pixel cluster conforms to the repair rules after being added into a previously constructed defective pixel set, the defective pixel cluster is added into the defective pixel set, until the number of defective pixel clusters in all defective pixel sets is equal to the number of repairable defective pixel clusters of the image sensor or there is no selectable defective pixel cluster; wherein the previously constructed defective pixel set is at least two; each defective pixel set is distributed to a corresponding defective pixel repair module, and the defective pixel screening of the image sensor is completed, so that the yield can be improved while the production efficiency is maintained.

Description

Technical Field

[0001] This application belongs to the field of image sensor technology, and in particular relates to a defect pixel screening method, an image sensor, and a computer program product. Background Technology

[0002] During the production and use of image sensors, defects in the manufacturing process or errors in light signal conversion may cause some pixels to fail to record image information correctly; these pixels are called defective pixels. These defective pixels not only display abnormalities themselves but may also affect surrounding normal pixels during image processing stages, such as interpolation and filtering, leading to defective products. To ensure a high yield rate for image sensors, these problematic pixels need to be corrected using a defective pixel repair module before image processing such as interpolation and filtering.

[0003] When the defective pixel repair module performs repair tasks, its accuracy is highly dependent on the results of defective pixel screening. Therefore, how to optimize the defective pixel screening method to ensure that production efficiency is maintained while improving yield is a key issue that urgently needs to be addressed. Summary of the Invention

[0004] In view of this, embodiments of this application provide a defective pixel screening method, an image sensor, and a computer program product that can ensure improved yield while maintaining production efficiency.

[0005] A first aspect of this application provides a method for screening defective pixels in an image sensor, the image sensor including a defective pixel repair module, the method comprising:

[0006] Acquire image data from the image sensor and filter out defect pixel clusters whose size conforms to the repair rules of the defect pixel repair module;

[0007] A defective pixel cluster is selected sequentially according to a preset size order. If the defective pixel cluster meets the repair rules after being added to a pre-constructed defective pixel set, the defective pixel cluster is added to the defective pixel set until the number of defective pixel clusters in all defective pixel sets is equal to the number of defective pixel clusters that the image sensor can repair or there are no defective pixel clusters to select. The pre-constructed defective pixel set is at least two.

[0008] Each defective pixel set is assigned to a corresponding defective pixel repair module to complete the defective pixel screening of the image sensor.

[0009] A second aspect of this application provides a defective pixel screening device, comprising:

[0010] The defect pixel acquisition module is used to acquire image data from the image sensor and filter out defect pixel clusters whose size conforms to the repair rules of the defect pixel repair module;

[0011] The defective pixel screening module is used to select a defective pixel cluster in a preset size order. If the defective pixel cluster meets the repair rules after being added to a pre-built defective pixel set, the defective pixel cluster is added to the defective pixel set until the number of defective pixel clusters in all defective pixel sets is equal to the number of defective pixel clusters that the image sensor can repair. The pre-built defective pixel set is at least two.

[0012] The defect pixel set allocation module is used to allocate each defect pixel set to the corresponding defect pixel repair module, thereby completing the defect pixel screening corresponding to each defect pixel repair module.

[0013] A third aspect of this application provides an image sensor including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the defective pixel screening method provided in the first aspect of this application.

[0014] A fourth aspect of this application provides a computer program product, including a computer program that, when run, causes the defective pixel screening method provided in the first aspect of this application to be executed.

[0015] The defect pixel screening method provided in the first aspect of this application acquires image data from an image sensor and filters out defect pixel clusters whose sizes conform to the repair rules of a defect pixel repair module. A defect pixel cluster is selected sequentially according to a preset size order. If the defect pixel cluster conforms to the repair rules after being added to a pre-constructed defect pixel set, the defect pixel cluster is added to the defect pixel set until the number of defect pixel clusters in all defect pixel sets equals the number of repairable defect pixel clusters in the image sensor or there are no selectable defect pixel clusters. The pre-constructed defect pixel set is at least two. Each defect pixel set is assigned to a corresponding defect pixel repair module to complete the defect pixel screening of the image sensor. This method not only efficiently screens defect pixel clusters, ensuring production efficiency, but also prioritizes the screening of larger defect pixel clusters for repair. Furthermore, under the premise of conforming to the repair rules, it achieves optimal allocation of defect pixel clusters, improving the yield rate of the image sensor.

[0016] It is understandable that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of a defect pixel screening method provided in an embodiment of this application;

[0019] Figure 2 This is a schematic flowchart of a defect pixel screening method provided in another embodiment of this application;

[0020] Figure 3 This is a schematic flowchart of a defect pixel screening method provided in another embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the defect pixel screening device provided in the embodiments of this application;

[0022] Figure 5 This is a schematic diagram of the structure of the image sensor provided in the embodiments of this application. Detailed Implementation

[0023] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0024] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0025] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0027] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0029] The defect pixel screening method provided in this application is applied to an image sensor, which has a defect pixel repair module. This method can be executed by the image sensor's processor when running a computer program with corresponding functions. It acquires image data from the image sensor and filters out defect pixel clusters whose sizes conform to the repair rules of the defect pixel repair module. A defect pixel cluster is selected sequentially according to a preset size order. If a defect pixel cluster conforms to the repair rules after being added to a pre-constructed defect pixel set, it is added to the defect pixel set until the number of defect pixel clusters in all defect pixel sets equals the number of repairable defect pixel clusters in the image sensor or there are no more selectable defect pixel clusters. At least two pre-constructed defect pixel sets are used. Each defect pixel set is assigned to a corresponding defect pixel repair module, completing the defect pixel screening of the image sensor. This method not only efficiently screens defect pixel clusters, ensuring production efficiency, but also prioritizes the screening of larger defect pixel clusters for repair. Furthermore, under the premise of conforming to the repair rules, it achieves optimal allocation of defect pixel clusters, improving the yield rate of the image sensor.

[0030] In applications, the image sensor can be any of the following: CMOS image sensor (Complementary Metal-Oxide-Semiconductor Image Sensor), CCD sensor (Charge-Coupled Device), 3D ToF (Time of Flight) sensor, thermal image sensor, front-illuminated (FSI) sensor, back-illuminated (BSI) sensor, or stacked sensor.

[0031] like Figure 1 As shown, the defect pixel screening method provided in this application embodiment includes the following steps S101 to S103:

[0032] Step S101: Acquire image data from the image sensor and filter out defect pixel clusters whose size conforms to the repair rules of the defect pixel repair module.

[0033] In applications, image data can be digital image data obtained by an image sensor through the conversion of light signals into electrical signals. For example, digital image data can be converted RAW data that corresponds one-to-one with each pixel in the pixel array of the image sensor. It should be noted that the processing of image data is described based on the position of the corresponding pixel array, and the actual storage method can follow existing technologies.

[0034] In applications, defective pixel clusters can be detected using defective pixel detection algorithms. For example, a threshold detection algorithm can be used to compare pixel data with a threshold range; pixels exceeding the range are considered defective pixels. Alternatively, statistical methods can be used to calculate the local pixel mean and standard deviation, identifying pixels whose values ​​deviate abnormally from the mean as defective pixels. Pattern matching can also be employed, comparing pixel morphology with defective pixel forms in a pre-built database to identify defective pixels. Specific detection methods can utilize existing CIS (Corporate Identity Detection System) identification techniques.

[0035] In addition, different defect pixel repair rules impose different restrictions on the size range of defect pixel clusters. Based on the repair rules, it can be determined whether the size of each defect pixel cluster in the image falls within the range specified by the defect pixel repair rules, that is, defect pixel clusters that fall within the range specified by the repair rules are selected.

[0036] In applications, for regularly shaped defect pixel clusters, such as rectangular distributions, the dimensions in the regularity directly correspond to the dimensions of the rectangular region; for example, a rectangular defect pixel cluster with 3 pixels in the row direction and 4 pixels in the column direction has a size of 3*4. For irregular defect pixel clusters, the dimensions of the circumscribed rectangle are used. For example, the size of a cross-shaped defect pixel cluster is determined by the size of its circumscribed rectangular region.

[0037] In one embodiment, the size range of the selected defective pixel clusters is set to 1*1 to 4*4 for subsequent repair.

[0038] Step S102: Select a defective pixel cluster in a preset size order. If the defective pixel cluster meets the repair rules after being added to a pre-built defective pixel set, add the defective pixel cluster to the defective pixel set until the number of defective pixel clusters in all defective pixel sets is equal to the number of defective pixel clusters that the image sensor can repair or there are no defective pixel clusters to select. Among them, the pre-built defective pixel set is at least two.

[0039] In the application, all selected defective pixel clusters are sorted by size, and selection begins sequentially from the largest defective pixel cluster. This prioritizes defective pixel clusters that have a greater impact on image quality, ensuring that the most important issues are addressed first. Before processing, multiple defective pixel sets need to be pre-built. These defective pixel sets are containers used to store and manage defective pixel clusters of different sizes and types. Each defective pixel set can be considered an independent repair task queue for subsequent repair processing. At least two defective pixel sets are pre-built, and the number of defective pixel sets corresponds one-to-one with the number of defective pixel repair modules connected in series within the image sensor. When there are two defective pixel repair modules, there are two defective pixel sets; when there are three defective pixel repair modules, there are three defective pixel sets.

[0040] In the application, for each selected defective pixel cluster, it is necessary to determine whether it still meets the repair rules after being added to a certain defective pixel set. For example, a defective pixel repair module may only be able to repair a certain number of large defective pixel clusters, or there may be specific requirements for the distribution of defective pixel clusters. Only when a defective pixel cluster still meets these rules after being added will it be formally added to the defective pixel set. Each defective pixel cluster will first be judged whether it meets the repair rules before being added to the corresponding defective pixel set. This process continues until the number of defective pixel clusters in all defective pixel sets reaches the maximum number that the image sensor can repair, or there are no more defective pixel clusters to select. When the number of defective pixel clusters in all defective pixel sets reaches the maximum number that the image sensor can repair, or there are no more defective pixel clusters to select, the entire screening process ends, and preparations are made for the next step of defective pixel set allocation processing.

[0041] Step S103: Assign each defective pixel set to the corresponding defective pixel repair module to complete the defective pixel screening of the image sensor.

[0042] In application, after all defective pixel clusters within each defective pixel set conform to the defective pixel repair rules, the defective pixel clusters within the defective pixel set are assigned to the corresponding defective pixel repair module. Specifically, they can be assigned to the storage block (such as OTP) of the corresponding defective pixel repair module (such as RTL), so that the defective pixel repair module can read the data in its respective register and use the defective pixel repair algorithm to repair the defective pixel clusters in the register.

[0043] This application embodiment utilizes repair rules to initially screen defective pixel clusters by size, ensuring that the selected defective pixel clusters meet the repair requirements. Then, defective pixel clusters are selected sequentially from largest to smallest, prioritizing those with a greater impact on image quality. This prioritizes the treatment of important defects, significantly improving repair effectiveness and ensuring a high yield rate. Simultaneously, the pre-constructed at least two defective pixel sets and the allocation based on the repair rules not only achieve optimal allocation but also maximize the utilization of the repair module with limited repair resources. This method not only efficiently screens defective pixel clusters, ensuring production efficiency, but also prioritizes the repair of larger defective pixel clusters and achieves optimal allocation of defective pixel clusters while adhering to the repair rules, significantly improving the yield rate of image sensors.

[0044] In one embodiment, such as Figure 2 As shown, step S102 includes the following steps S201 and S204:

[0045] Step S201: Select a defective pixel cluster in descending order.

[0046] In the application, all selected defective pixel clusters are sorted according to their size, and selection is performed sequentially starting with the largest defective pixel cluster. The purpose of this is to prioritize defective pixel clusters that have a greater impact on image quality. For example, assuming the selected defective pixel clusters have the following sizes: 3*3, 2*2, 4*4, 1*1, 2*3, after sorting, the selection order will be: 4*4, 3*3, 2*3, 2*2, 1*1.

[0047] Step S202: Determine whether the defective pixel cluster meets the repair rules when it is added to the i-th defective pixel set.

[0048] In the application, i = 1, 2, ..., n-1. For each selected defective pixel cluster, it is necessary to determine whether adding it to the i-th defective pixel set still meets the repair rules. The repair rules may include, but are not limited to, factors such as the size, shape, and position of the defective pixel cluster. i represents the number of the defective pixel set currently being tried to add, and n represents the total number of pre-built defective pixel sets. i = 1, 2, ..., n-1 means that i starts from 1 and increases gradually until n-1.

[0049] Step S203: If the defective pixel cluster meets the repair rules when added to the i-th defective pixel set, then add the defective pixel cluster to the i-th defective pixel set and return to the step of selecting a defective pixel cluster in descending order.

[0050] In the application, if a defective pixel cluster meets the repair rules when added to the i-th defective pixel set, it is officially added to the i-th defective pixel set. After addition, the process returns to step S201, and the next defective pixel cluster is selected in descending order. For example, assuming the currently selected 4*4 defective pixel cluster meets the repair rules of the 1st defective pixel set, after adding it to the 1st defective pixel set, the next largest defective pixel cluster (i.e., 3*3) is selected for judgment and processing.

[0051] Step S204: If the defective pixel cluster does not meet the repair rules when added to the i-th defective pixel set, let i = i + 1, and return to the step of determining whether the defective pixel cluster meets the repair rules when added to the i-th defective pixel set.

[0052] Where i = 1, 2, ..., n-1, i is a positive integer, and n is the number of pre-constructed defect pixel sets.

[0053] In the application, if a defective pixel cluster does not meet the repair rules when added to the i-th defective pixel set, the value of i is incremented by 1, and the defective pixel cluster is attempted to be added to the next defective pixel set (i.e., the (i+1)-th defective pixel set). For example, assuming the currently selected 4x4 defective pixel cluster does not meet the repair rules of the 1st defective pixel set, i is changed from 1 to 2, and the 4x4 defective pixel cluster is attempted to be added to the 2nd defective pixel set. If adding it to the 2nd defective pixel set also does not meet the repair rules, i is continued to be incremented by 1 until a suitable defective pixel set is found or all defective pixel sets cannot accommodate the defective pixel cluster.

[0054] This application embodiment selects defective pixel clusters in order of size and determines whether they meet the repair rules after being added to the corresponding defective pixel set. It prioritizes the processing of larger defective pixel clusters that have a greater impact on image quality. At the same time, through the dynamic allocation mechanism of multiple defective pixel sets, it avoids the problem of incomplete repair caused by the capacity limitation of a single defective pixel set. This not only improves the screening and repair efficiency of defective pixel clusters and increases production efficiency, but also achieves the optimal allocation of defective pixel clusters under the premise of meeting the repair rules, significantly improving the yield of image sensors.

[0055] In one embodiment, step S301 is further included:

[0056] Step S301: If the defective pixel cluster does not meet the repair rules when added to the nth defective pixel set, then add the defective pixel cluster to the pre-created deletion set.

[0057] In the application, if the currently selected defective pixel cluster cannot be added to any defective pixel set, it is considered an unrepairable defective pixel cluster. In this case, the defective pixel cluster is added to a pre-created deletion set. The deletion set stores those unrepairable defective pixel clusters for further processing or recording. For example, suppose the currently selected 4x4 defective pixel cluster fails to meet the repair rules when attempting to be added to the 1st, 2nd, and 3rd defective pixel sets due to various reasons (such as quantity limitations, positional conflicts, etc.), then this 4x4 defective pixel cluster will be added to the deletion set.

[0058] In one embodiment, step S202 specifically involves:

[0059] If in the i-th defective pixel set there are no other defective pixel clusters within a specified pixel cell from the defective pixel cluster, and there are no other defective pixel clusters within a single defective pixel repair cycle along the row direction in the specified row area centered on the defective pixel cluster, and there are no other defective pixel clusters in the specified row area centered on the defective pixel cluster or there are other defective pixel clusters not exceeding a preset number, then the defective pixel cluster is determined to meet the repair rules when added to the i-th defective pixel set.

[0060] In application, the condition that no other defective pixel clusters exist within a specified pixel cell of the currently selected defective pixel cluster ensures a safe distance around it, preventing it from getting too close to other defective pixel clusters and resulting in poor repair performance. For example, assuming the specified pixel cell is 3 pixels, then there should be no other defective pixel clusters within a 3-pixel distance of the current defective pixel cluster. The 3 pixels surrounding the current defective pixel cluster can be the left or right side along the row direction, or the top or bottom side along the column direction.

[0061] In application, the condition that no other defective pixel clusters exist within a single defective pixel repair cycle along the row direction within a specified row area centered on the defective pixel cluster ensures that the repair of the current defective pixel cluster is not affected by other defective pixel clusters in the same row direction. For example, assuming the specified row area is 5 rows of pixels and a defective pixel repair cycle is 10 pixels, then within the row area of ​​5 rows above and below the current defective pixel cluster, there should be no other defective pixel clusters within a distance of 10 pixels along the row direction.

[0062] In the application, the specified row area centered on the defective pixel cluster must contain no other defective pixel clusters or contain no more than a preset number of other defective pixel clusters. This condition further ensures that within the specified row area of ​​the current defective pixel cluster, either there are no other defective pixel clusters, or there are no more than a preset number of other defective pixel clusters. For example, assuming the specified row area is 5 rows of pixels and the preset number is 2, then within the row area of ​​5 rows above and below the current defective pixel cluster, a maximum of 2 other defective pixel clusters are allowed.

[0063] This application's embodiments, through multi-dimensional conditional judgments, ensure that each defective pixel cluster conforms to strict repair rules when added to the defective pixel set, thereby improving the accuracy and effectiveness of repair. Specifically, by ensuring a safe distance around the defective pixel cluster, interference from other defective pixel clusters during the repair process is avoided, improving the quality of the repair effect; by limiting the number of other defective pixel clusters within the repair cycle in the row direction, the effectiveness of repair is further improved; by setting a preset number of other defective pixel clusters, a small number of defective pixel clusters are allowed within a certain range, increasing the flexibility of the repair rules and adapting to repair needs in different scenarios. This application's embodiments, through strict repair rules, ensure that the repair of each defective pixel cluster is effective, significantly improving the yield rate of image sensors and reducing the defect rate caused by improper repair, thereby improving overall production efficiency and product quality.

[0064] In one embodiment, step S101 is followed by:

[0065] The defective pixel clusters are sorted in descending order.

[0066] In the application, after the initial screening of defective pixel clusters, these defective pixel clusters are further sorted to facilitate subsequent selection in order of size. For example, assuming the screened defective pixel clusters have the following sizes: 3*3, 2*2, 4*4, 1*1, 2*3, after sorting, the selection order will be: 4*4, 3*3, 2*3, 2*2, 1*1.

[0067] In one embodiment, step S401 is also included:

[0068] If there are defective pixel clusters of the same size, then sort them in ascending order of distance value, where the distance value is the distance between the largest defective pixel cluster and the defective pixel cluster.

[0069] In applications, to further optimize the processing order, these defective pixel clusters can be sorted according to the distance between them and the largest defective pixel cluster. Specifically, the distance between each defective pixel cluster and the largest defective pixel cluster is calculated, and then they are sorted in ascending order of distance value. For example, suppose there are two 3*3 defective pixel clusters A and B, and a 4*4 largest defective pixel cluster C. Calculate the distance between A and B and C. Assuming the distance between A and C is 10 pixels, and the distance between B and C is 5 pixels, then after sorting, B will be placed before A.

[0070] In one embodiment, the distance between the largest defective pixel cluster and the defective pixel cluster is the distance between the center of the outer rectangle of the largest defective pixel cluster and the defective pixel cluster.

[0071] In applications, to calculate distances more accurately, the center of the bounding rectangle of the largest defective pixel cluster can be used as a reference point. Specifically, the Euclidean distance between the center of the bounding rectangle of the largest defective pixel cluster and the center of the bounding rectangle of each defective pixel cluster is calculated. For example, assuming the coordinates of the center of the bounding rectangle of the largest defective pixel cluster C are (10, 10), the coordinates of the center of the bounding rectangle of defective pixel cluster A are (15, 15), and the coordinates of the center of the bounding rectangle of defective pixel cluster B are (12, 12), then:

[0072] The distance between A and C is 1 pixel;

[0073] The distance between B and C is The number of pixels is 1, so B will be placed before A.

[0074] This application embodiment sorts defective pixel clusters in descending order, and when defective pixel clusters of the same size exist, further sorts them in ascending order according to their distance from the largest defective pixel cluster. This ensures that defective pixel clusters that have a greater impact on image quality are processed first, significantly improving the yield of the image sensor.

[0075] In one embodiment, a defective pixel cluster is a single defective pixel or a plurality of consecutive defective pixels.

[0076] In applications, a defective pixel cluster can be a single defective pixel or multiple consecutive defective pixels. When a defective pixel has no adjacent defective pixels, it is considered an independent defective pixel, i.e., a 1*1 defective pixel cluster. When multiple defective pixels are connected to form a continuous array, the array is considered a single defective pixel cluster. For example, two adjacent defective pixels can be considered as one defective pixel cluster. If a defective pixel has an adjacent defective pixel in both its row and column directions, forming an L-shaped defective pixel array, then this L-shaped defective pixel array is considered a defective pixel cluster.

[0077] In one embodiment, there are two defective pixel repair modules, and the defective pixel repair rules are as follows:

[0078] Rule 1: Supports defective pixels ranging from a minimum of 1x1 to a maximum of 4x4.

[0079] Rule 2: The defect pixel regions adjacent to defect pixels are spaced 4 pixels apart in the x-direction and 4 pixels apart in the y-direction.

[0080] Rule 3: (Current processing row of defective pixels + 4 rows below) There are a maximum of 3 defective pixels in 5 rows. The defective pixel information is arranged in ascending order of x and y.

[0081] Rule 4: For adjacent defective pixels in the same row, only one defective pixel is supported per cycle.

[0082] A defect pixel correction method is provided, such as Figure 3 As shown, the process includes the following steps S1 to S16:

[0083] Step S1: Begin.

[0084] Step S2: Locate defective pixel clusters from 1*1 to 4*4, and find the center of the largest defective pixel cluster in the image, Rect_Center.

[0085] In the application, defective pixel clusters with sizes ranging from 1*1 to 4*4 are filtered from the image data. The largest defective pixel cluster in the image is then identified from these filtered clusters. For example, if there is a 4*4 largest defective pixel cluster C, its center (Rect_Center) is determined. When the largest defective pixel cluster C itself is a regular rectangular structure, this center is the actual center of the largest defective pixel cluster C. When the largest defective pixel cluster C is an irregular defective pixel cluster, the center of the bounding rectangle (4*4) of the largest defective pixel cluster C is taken as the center of the largest defective pixel cluster C. The center (Rect_Center) can be represented in coordinate form, such as (10, 12), where the horizontal axis is along the row direction, with each pixel as a unit, and 10 represents 10 pixels along the row direction. Similarly, the vertical axis is along the column direction, and 12 represents 12 pixels along the column direction. The origin of the coordinate system can be set to any position in the image, such as the start or end point of the image.

[0086] Step S3: Sort all defective pixel clusters by size, then sort them in ascending order by distance from Rect_Center, and the set is ClusterSetAll.

[0087] In the application, assuming the selected defective pixel clusters have the following sizes: 3*3, 2*2, 2*2, 4*4, 1*1, 2*3, after sorting by size, the order is: 4*4, 3*3, 2*3, 2*2, 2*2, 1*1. Two defective pixel clusters have the same size, both 2*2 and 2*2. These are then further sorted in ascending order based on their distance from the Rect_Center. Specifically, for the two 2*2 defective pixel clusters A and B, and the largest 4*4 defective pixel cluster C, the distances between A, B, and C are calculated. Assuming the distance between A and C is 10 pixels, and the distance between B and C is 5 pixels, then after sorting, B will be placed before A. All sorted defective pixel clusters are then added to the set ClusterSetAll.

[0088] Step S4: Create collections OTPSet1, OTPSet2, and DelSet.

[0089] In the application, sets OTPSet1, OTPSet2, and DelSet are pre-created. OTPSet1 is assigned to the first defective pixel repair module so that the first defective pixel repair module can repair defective pixel clusters within OTPSet1. OTPSet2 is assigned to the second defective pixel repair module so that the second defective pixel repair module can repair defective pixel clusters within OTPSet2. DelSet is a deletion set used to store all defective pixel clusters that cannot be repaired.

[0090] Step S5: Set OtpTmp = OTPSet1.

[0091] Step S6: Take the defective pixel clusters Cls from ClusterSetAll in sequence.

[0092] Step S7: Determine whether there are other defective pixel clusters in the four pixels above, below, left, and right of the defective pixel cluster Cls in OtpTmp. If there are, proceed to step S10; otherwise, proceed to step S8.

[0093] Step S8: Determine whether there is a defective pixel cluster within the range of the four rows above the defective pixel cluster Cls, the row containing the defective pixel cluster, and the four rows below it in OtpTmp. If there is, proceed to step S10; otherwise, proceed to step S9.

[0094] Step S9: Determine whether the number of other defective pixel clusters within the range of four rows above the defective pixel cluster Cls, the row containing the defective pixel cluster, and four rows below it in OtpTmp exceeds 3. If yes, proceed to step S10; otherwise, proceed to step S13.

[0095] In application, the order of steps S7 to S9 can be arbitrarily adjusted, as long as each step is executed. Specifically, if in the i-th defective pixel set, there are no other defective pixel clusters within a specified pixel cell from the defective pixel cluster, and there are no other defective pixel clusters within one defective pixel repair cycle along the row direction in the specified row area centered on the defective pixel cluster, and there are no other defective pixel clusters or there are no more than a preset number of other defective pixel clusters in the specified row area centered on the defective pixel cluster, then the defective pixel cluster is determined to meet the repair rules when added to the i-th defective pixel set.

[0096] Step S10: Determine whether the current OtpTmp is OTPSet1. If yes, proceed to step S11; otherwise, proceed to step S12.

[0097] Step S11: Set OtpTmp = OTPSet2 and execute step S7.

[0098] Step S12: Add the defective pixel cluster Cls to the set DelSet, and delete the defective pixel cluster Cls from ClusterSetAll. Then proceed to step S14.

[0099] Step S13: Add the defective pixel cluster Cls to the set OtpTmp, and delete the defective pixel cluster Cls from ClusterSetAll. Then proceed to step S14.

[0100] In the application, steps S5, S6, and S10-S13 are used to select defective pixel clusters in descending order and determine whether they meet the repair rules after being added to the corresponding defective pixel set. Specifically: if a defective pixel cluster does not meet the repair rules when added to the i-th defective pixel set, the value of i is incremented by 1, and the defective pixel cluster is attempted to be added to the next defective pixel set (i.e., the (i+1)-th defective pixel set). For example, assuming the currently selected 4*4 defective pixel cluster does not meet the repair rules of the first defective pixel set, i is changed from 1 to 2, and the 4*4 defective pixel cluster is attempted to be added to the second defective pixel set. If adding to the second defective pixel set also does not meet the repair rules, i is incremented by 1 again until a suitable defective pixel set is found or all defective pixel sets cannot accommodate the defective pixel cluster. If the currently selected defective pixel cluster cannot be added to any defective pixel set, the defective pixel cluster is considered an unrepairable defective pixel cluster. At this time, the defective pixel cluster is added to a pre-created deletion set. The deletion set is a special set used to store those unrepairable defective pixel clusters for further processing or recording. For example, if the currently selected 4x4 defective pixel cluster fails to meet the repair rules when trying to add it to the 1st, 2nd, and 3rd defective pixel sets for various reasons (such as quantity limitations, positional conflicts, etc.), then this 4x4 defective pixel cluster will be added to the deletion set.

[0101] Step S14: Determine whether the sum of OTPSet1 + OTPSet2 is equal to M or whether ClusterSetAll is empty. If yes, proceed to step S15; otherwise, proceed to step S5.

[0102] In application, the sum of OTPSet1 + OTPSet2 equals M. For example, in this case, M equals 2, meaning that the defective pixel clusters allocated to the defective pixel repair module have reached the repair limit of the image sensor. At this point, no more defective pixel clusters will be added to either OTPSet1 or OTPSet2. Furthermore, if ClusterSetAll is empty, it is assumed that all defective pixel clusters have been allocated to either OTPSet1 or OTPSet2. Although the repair limit of the image sensor has not been reached, the allocation can still be completed.

[0103] Step S15: Output OTPSet1, OTPSet2, and DelSet.

[0104] In the application, OTPSet1 is assigned to the first defective pixel repair module, and OTPSet2 is assigned to the second defective pixel repair module. The unrepairable defective pixel clusters stored in DelSet can also be further processed or recorded later.

[0105] Step S16, End.

[0106] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0107] This application also provides a defective pixel screening device for performing the steps described in the defective pixel screening method embodiments above. The defective pixel screening device may be a virtual appliance in the image sensor, run by the image sensor's processor, or it may be the image sensor itself.

[0108] like Figure 4 As shown, the defective pixel screening device 100 provided in this application embodiment includes:

[0109] The defect pixel acquisition module 101 is used to acquire image data from the image sensor and filter out defect pixel clusters whose size conforms to the repair rules of the defect pixel repair module.

[0110] The defect pixel screening module 102 is used to select a defect pixel cluster in a preset size order. If the defect pixel cluster meets the repair rules after being added to a pre-built defect pixel set, the defect pixel cluster is added to the defect pixel set until the number of defect pixel clusters in all defect pixel sets is equal to the number of defect pixel clusters that the image sensor can repair. The pre-built defect pixel set is at least two.

[0111] The defect pixel set allocation module 103 is used to allocate each defect pixel set to the corresponding defect pixel repair module, thereby completing the defect pixel screening corresponding to each defect pixel repair module.

[0112] In applications, the modules in the defective pixel screening device can be software program modules, or they can be implemented by different logic circuits integrated in the processor, or they can be implemented by multiple distributed processors.

[0113] like Figure 5 As shown, this application embodiment also provides an image sensor 200, including: at least one processor 201 ( Figure 5 The diagram shows only one processor, memory 202, and computer program 203 stored in memory 202 and executable on at least one processor 201. When processor 201 executes computer program 203, it implements the steps in the various method embodiments described above.

[0114] In applications, image sensors may include, but are not limited to, processors and memory. Those skilled in the art will understand that... Figure 5 This is merely an example of an image sensor and does not constitute a limitation on image sensors. It may include more or fewer components than illustrated, or combine certain components, or use different components.

[0115] In applications, the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0116] In applications, the memory may be an internal storage unit of the image sensor in some embodiments, such as the hard drive or RAM of the image sensor. In other embodiments, the memory may be an external storage device of the image sensor, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., mounted on the image sensor. Furthermore, the memory may include both internal and external storage units of the image sensor. The memory is used to store operating systems, applications, boot loaders, data, and other programs, such as program code for computer programs. The memory can also be used to temporarily store data that has been output or will be output.

[0117] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0118] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0119] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the various method embodiments above.

[0120] This application provides a computer program product that, when run on an image sensor, enables the image sensor to perform the steps described in the various method embodiments above.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / image sensor, a recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0122] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0123] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0124] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for screening defective pixels in an image sensor, characterized in that, The image sensor includes a defective pixel repair module, and the method includes: Acquire image data from the image sensor and filter out defect pixel clusters whose size conforms to the repair rules of the defect pixel repair module; A defective pixel cluster is selected sequentially according to a preset size order. If the defective pixel cluster meets the repair rules after being added to a pre-constructed defective pixel set, the defective pixel cluster is added to the defective pixel set until the number of defective pixel clusters in all defective pixel sets is equal to the number of defective pixel clusters that the image sensor can repair or there are no defective pixel clusters to select. The pre-constructed defective pixel set is at least two. Each defective pixel set is assigned to a corresponding defective pixel repair module to complete the defective pixel screening of the image sensor.

2. The defect pixel screening method as described in claim 1, characterized in that, The step of selecting a defective pixel cluster sequentially according to a preset size order, and if the defective pixel cluster meets the repair rules after being added to a pre-constructed defective pixel set, then adding the defective pixel cluster to the defective pixel set includes: Select a cluster of defective pixels in descending order; Determine whether the defective pixel cluster conforms to the repair rules when it is added to the i-th defective pixel set; If the defective pixel cluster meets the repair rules when added to the i-th defective pixel set, then the defective pixel cluster is added to the i-th defective pixel set, and the step of selecting a defective pixel cluster in descending order is returned; If the defective pixel cluster does not meet the repair rules when added to the i-th defective pixel set, then let i = i + 1, and return to the step of determining whether the defective pixel cluster meets the repair rules when added to the i-th defective pixel set; Where i = 1, 2, ..., n-1, i is a positive integer, and n is the number of pre-constructed defect pixel sets.

3. The defect pixel screening method as described in claim 2, characterized in that, Also includes: If the defective pixel cluster does not meet the repair rules when added to the nth defective pixel set, then the defective pixel cluster is added to the pre-created deletion set.

4. The defect pixel screening method as described in claim 2, characterized in that, The step of determining whether the defective pixel cluster meets the repair rules when added to the i-th defective pixel set includes: If, in the i-th defective pixel set, there are no other defective pixel clusters within a specified pixel cell from the defective pixel cluster, and there are no other defective pixel clusters within a defective pixel repair cycle along the row direction in the specified row area centered on the defective pixel cluster, and there are no other defective pixel clusters or there are no more than a preset number of other defective pixel clusters in the specified row area centered on the defective pixel cluster, then it is determined that the defective pixel cluster meets the repair rules when added to the i-th defective pixel set.

5. The defect pixel screening method as described in claim 1, characterized in that, After acquiring image data from the image sensor and filtering out defective pixel clusters whose sizes conform to the repair rules of the defective pixel repair module, the process further includes: The defective pixel clusters are sorted in descending order.

6. The defect pixel screening method as described in claim 5, characterized in that, Also includes: If there are defective pixel clusters of the same size, then the defective pixel clusters of the same size are sorted in ascending order of distance value, where the distance value is the distance between the largest defective pixel cluster and the defective pixel cluster.

7. The defect pixel screening method as described in claim 6, characterized in that, The distance between the largest defective pixel cluster and the defective pixel cluster is the distance between the center of the outer rectangle of the largest defective pixel cluster and the defective pixel cluster.

8. The defect pixel screening method as described in claim 1, characterized in that, The defective pixel cluster can be a single defective pixel or multiple consecutive defective pixels.

9. An image sensor, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the defect pixel screening method as described in any one of claims 1 to 8.

10. A computer program product, characterized in that, Includes a computer program, which, when run, causes the method as described in any one of claims 1-8 to be performed.

Images