Defective pixel detection method and electronic device

Abstract

The present application provides a method and electronic device for detecting bad pixels, relating to the field of image processing technology. The method comprises: an electronic device, with a pixel to be detected as the center, obtains the pixel value of each of m pixels in a first pixel matrix that are in the same color channel as the pixel to be detected, to obtain m pixel values; m is a positive integer. The electronic device determines a first pixel threshold based on the m pixel values; if a first pixel value among the m pixel values ​​is greater than the first pixel threshold, the electronic device determines that the first pixel corresponding to the first pixel value is a bad pixel; after the electronic device compensates the first pixel, the electronic device determines whether the pixel to be detected is a bad pixel. This method can detect three adjacent bad pixels in a RAW domain image caused by defects in the image sensor, thereby improving the accuracy of detection.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on January 29, 2024, with application number 202410124205.4 and invention name “A method and electronic device for detecting bad pixels”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of image processing technology, and in particular to a method for detecting bad pixels and an electronic device. Background Art

[0003] With the advancement of science and technology and manufacturing processes, image sensors used in cameras in electronic devices, as photoelectric sensors, have greatly satisfied people's needs for image acquisition and processing. However, during manufacturing and actual use, defects in the material structure, manufacturing process, and aging of image sensors can cause bad pixels in the images they output, affecting the quality of the images. Summary of the Invention

[0004] The present application provides a bad pixel detection method and electronic device, which can detect three adjacent bad pixels existing in a RAW domain image caused by defects in an image sensor, thereby improving the detection accuracy.

[0005] This application adopts the following technical solutions:

[0006] In a first aspect, a method for detecting bad pixels is provided, which is applied to an electronic device, wherein the electronic device includes a camera, which is used to capture a RAW domain image, wherein the RAW domain image includes multiple pixels; wherein the electronic device sequentially uses every two adjacent pixels in the multiple pixels as pixels to be detected, and detects whether there are bad pixels in the multiple pixels. Specifically, the method may include:

[0007] The electronic device obtains the pixel value of each of m pixels in a first pixel matrix that share the same color channel as the pixel to be detected, with the pixel to be detected as the center, to obtain m pixel values, where m is a positive integer. The electronic device determines a first pixel threshold based on the m pixel values. If a first pixel value among the m pixel values ​​is greater than the first pixel threshold, the electronic device determines that the first pixel corresponding to the first pixel value is a defective pixel. After compensating the first pixel, the electronic device determines whether the pixel to be detected is a defective pixel.

[0008] Based on the first aspect, the electronic device first detects the bad pixels around the pixel to be detected with the pixel to be detected as the center; then, after compensating for the surrounding bad pixels, it detects whether the pixel to be detected is a bad pixel, thereby detecting three adjacent bad pixels in the RAW domain image, thereby improving the detection accuracy.

[0009] In one design, if the pixel values ​​corresponding to the pixel to be detected are all greater than a first pixel threshold, the electronic device determines that the pixel to be detected is a bad pixel. This allows the electronic device to directly use the first pixel threshold to detect whether the pixel to be detected is a bad pixel, reducing detection complexity and power consumption.

[0010] In one design, after detecting that a pixel to be detected is a defective pixel, the electronic device replaces the pixel value corresponding to the pixel to be detected with a first pixel threshold to compensate for the pixel to be detected; alternatively, the electronic device replaces the pixel value corresponding to the pixel to be detected with the second largest value among m pixel values ​​to compensate for the pixel to be detected. In this way, by compensating the pixel to be detected, image quality can be improved, thereby enhancing the user experience.

[0011] The pixel value corresponding to the pixel to be detected refers to the pixel values ​​of two adjacent pixels corresponding to the pixel to be detected.

[0012] In one design, an electronic device can determine whether a pixel to be detected is a bad pixel using the following method. Specifically, the electronic device, centered on the pixel to be detected, obtains the pixel value of each of n pixels in a second pixel matrix that share the same color channel as the pixel to be detected, thereby obtaining n pixel values; n is a positive integer, and the n pixel values ​​include the value of the first pixel to be replaced. The electronic device then determines a second pixel threshold based on the n pixel values. If the pixel value corresponding to the pixel to be detected is greater than the second pixel threshold, the electronic device determines that the pixel to be detected is a bad pixel.

[0013] In this way, by determining whether the pixel to be detected is a bad pixel in the above manner, the detection accuracy can be improved.

[0014] In one design method, after the electronic device determines that the pixel to be detected is a bad pixel, the electronic device replaces the pixel value corresponding to the pixel to be detected with a second pixel threshold to compensate for the pixel to be detected; or, the electronic device replaces the pixel value corresponding to the pixel to be detected with the maximum value of n pixel values ​​to compensate for the pixel to be detected.

[0015] In this way, by compensating the pixel values ​​of two adjacent pixels corresponding to the pixel to be detected, the image quality can be improved, thereby improving the user experience.

[0016] In one design approach, the electronic device can replace the first pixel value with a first pixel threshold to compensate for the first pixel; or the electronic device can replace the first pixel value with the second largest value among m pixel values ​​to compensate for the first pixel. In this way, by compensating for the first pixel, image quality can be improved, thereby enhancing the user experience.

[0017] In one design, the electronic device can determine the first pixel threshold in the following manner. Specifically, the electronic device determines the first pixel threshold based on a first variable value, a second variable value, and the second largest value among m pixel values; wherein the first variable value is any value in the range [1, 2]; and the second variable value is related to the number of bits of the pixel, or the second variable value is zero.

[0018] In this way, the first pixel threshold determined in this way is more accurate, thereby improving the accuracy of detection.

[0019] In one design approach, the first pixel threshold satisfies the following expression:

[0020] K1=Fmax*SMP+TBP_offset;

[0021] Wherein, K1 represents the first pixel threshold, Fmax represents the first variable value, SMP represents the second maximum value, and TBP_offse represents the second variable value.

[0022] In one design, the electronic device can determine the second pixel threshold in the following manner. Specifically, the electronic device determines the second pixel threshold based on a third variable value, a fourth variable value, and a maximum value among n pixel values; wherein the third variable value is any value in the range [1, 2]; the fourth variable value is related to the number of bits of the pixel; or the fourth variable value is zero.

[0023] In this way, the second pixel threshold determined in this way is more accurate, thereby improving the accuracy of detection.

[0024] In one design, the second pixel threshold satisfies the following expression:

[0025] K2=Fmax*MP+offset;

[0026] Wherein, K2 represents the second pixel threshold, Fmax represents the third variable value, MP represents the maximum value, and offset represents the fourth variable value.

[0027] In one design, multiple pixels included in a RAW domain image are divided into P pixel units, each of which includes one red pixel, two green pixels, and one blue pixel. This pixel unit can also be called an RGGB pixel unit, where P ≥ 1.

[0028] In a second aspect, an electronic device is provided, which has the function of implementing any one of the functions described in the first aspect. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0029] In a third aspect, an electronic device is provided, comprising: a memory, one or more processors, and a camera; the memory stores computer program code, the computer program code comprising computer instructions; when the computer instructions are executed by the processor, the electronic device executes the above-mentioned first aspect or any one of the methods in the first aspect.

[0030] In a fourth aspect, a chip system is provided, which includes: at least one processor and an interface, the interface being used to receive instructions and transmit them to at least one processor; at least one processor executes the instructions so that the electronic device executes any one of the methods described in the first aspect above.

[0031] In a fifth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, which, when executed on a computer, enables the computer to execute any one of the methods described in the first aspect.

[0032] In a sixth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any one of the methods described in the first aspect.

[0033] Among them, the technical effects brought about by any implementation method in the second to sixth aspects can refer to the technical effects brought about by different implementation methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of an interface for displaying dark spots in an image provided by an embodiment of the present application;

[0035] Figure 2 A schematic diagram of another interface for displaying dark spots in an image provided by an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0037] Figure 4 A schematic structural diagram of a camera provided in an embodiment of the present application;

[0038] Figure 5 A schematic structural diagram of an image sensor provided in an embodiment of the present application;

[0039] Figure 6A schematic structural diagram of a pixel unit provided in an embodiment of the present application;

[0040] Figure 7 A schematic diagram of a software framework of an electronic device provided in an embodiment of the present application;

[0041] Figure 8 A schematic diagram of an ISP processing process provided in an embodiment of the present application;

[0042] Figure 9 Schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 1 ;

[0043] Figure 10 Schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 2 ;

[0044] Figure 11 Schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 3 ;

[0045] Figure 12 Schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 4 ;

[0046] Figure 13 Schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 5 ;

[0047] Figure 14 Schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 6 ;

[0048] Figure 15 A schematic diagram comparing a bad pixel before and after elimination provided in an embodiment of the present application;

[0049] Figure 16 A schematic structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] Before describing the embodiments of the present application, the technical terms involved in the present application are first explained.

[0051] Bad pixels: Due to the process (or usage time) of the image sensor, there will be some defective pixels in the RAW domain image output by the image sensor. These pixels may be darker or brighter than the adjacent pixels, and are therefore called bad pixels (or bad pixels).

[0052] In some embodiments, bad pixels can be categorized as single bad pixels, double bad pixels, and bad pixel clusters based on the number of adjacent bad pixels on the same color channel. A single bad pixel refers to a single bad pixel within a certain range on the same color channel. A double bad pixel refers to two adjacent, consecutive bad pixels within a certain range on the same color channel. A bad pixel cluster refers to three or more adjacent, consecutive bad pixels within a certain range on the same color channel.

[0053] In actual implementation, bad pixels will affect the shooting effect, such as affecting the image quality, and thus affecting the user's shooting experience. For example, when shooting in a dark environment, if there is a bad pixel in the image sensor, the captured image will show a bright spot. Or, when shooting in a bright environment, if there is a bad pixel in the image sensor, the captured image will show a dark spot. For example, Figure 1 As shown, the captured image includes three dark spots.

[0054] It should be noted that if there are bad pixel clusters on the image sensor, the captured image will show a large area of ​​dark spots (or bright spots). Figure 2 As shown, a large area of ​​dark spots appears on the captured image.

[0055] Based on the above, an embodiment of the present application provides a method for detecting bad pixels, which is applied to electronic devices including cameras. This method can improve the quality of captured images and further improve user experience by detecting bad pixels existing in the RAW domain images captured by the image sensor and compensating for the bad pixels.

[0056] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0057] The bad pixel detection method provided in the embodiments of the present application can be applied to electronic devices with camera functions. The electronic devices may include, for example, mobile phones, action cameras (GoPro), digital cameras, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, in-vehicle devices, ultra-mobile personal computers (UMPCs), netbooks, cellular phones, personal digital assistants (PDAs), augmented reality (AR) and virtual reality (VR) devices. The embodiments of the present application do not impose any particular restrictions on the specific form of the electronic devices.

[0058] For example, Figure 3 A schematic structural diagram of the electronic device 100 is shown.

[0059] Among them, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a camera 193 and a display screen 194, etc.

[0060] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0061] The processor 110 may include one or more processing units, for example, the processor 110 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0062] Among them, the controller can be the nerve center and command center of the electronic device 100.

[0063] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0064] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an I2C interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface 130, etc.

[0065] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0066] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0067] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can execute the above instructions stored in the internal memory 121, so that the electronic device 100 performs the methods provided in some embodiments of the present application, as well as various functional applications and data processing. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system; the program storage area may also store one or more applications (such as a gallery, contacts, etc.). The data storage area may store data created during the use of the electronic device 100 (such as photos, contacts, etc.). In addition, the internal memory 121 may include a high-speed random access memory; it may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, universal flash storage (UFS), etc. In other embodiments, the processor 110 executes the instructions stored in the internal memory 121 and / or the instructions stored in the memory provided in the processor, so that the electronic device 100 performs the methods provided in the embodiments of the present application, as well as various functional applications and data processing.

[0068] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0069] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 can receive input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160.

[0070] The power management module 141 may be used to monitor performance parameters such as battery capacity, battery cycle count, battery charge voltage, battery discharge voltage, and battery health status (eg, leakage, impedance), etc.

[0071] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0072] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0073] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include one or more filters, switches, power amplifiers, low noise amplifiers (LAN), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be placed in the processor 110. In some embodiments, at least some of the functions of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0074] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate one or more communication processing modules. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0075] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0076] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0077] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-LED, a Micro-OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0078] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0079] The ISP is used to process data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, color, and other factors. The ISP can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0080] Camera 193 is used to capture still images or videos. The lens generates an optical image of an object and projects it onto a photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or other format. In some embodiments, electronic device 100 may include one or N cameras 193, where N is a positive integer greater than one.

[0081] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0082] Of course, the electronic device 100 provided in the embodiment of the present application may also include one or more devices such as a positioning module 181, a button 190, a motor 191, an indicator 192 and a SIM card interface [1~N] 195, without limitation.

[0083] In some embodiments of the present application, the camera 193 can be used as a rear camera and be disposed on one side of the housing of the electronic device 100. Alternatively, in other embodiments of the present application, the camera 193 can be used as a front camera and be disposed on one side of the display screen 194 of the electronic device 100.

[0084] For example, the camera 193 may include Figure 4 The image sensor 20 is shown, and an optical lens 30 is provided on the incident light side of the image sensor 20 . The optical lens 30 can allow external light to be incident on the image sensor 20 .

[0085] For example, the image sensor 20 may include: Figure 5 The filter 200 shown. The filter 200 may include filter units 210 arranged in an array. The filter unit 210 may include at least three color filter blocks for transmitting three primary colors of light, such as red (R), blue (B), and green (G), respectively, such as a red filter block R, a blue filter block B, and a green filter block G. Since the human eye is more sensitive to green, the number of green filter blocks G in the same filter unit 210 may be greater than the number of other color filter blocks. For example, when a filter unit 210 is arranged in a 2×2 matrix, any of the filter units 210 may include one red filter block R, one blue filter block B, and two green filter blocks G.

[0086] Based on this, the red filter block R is used to convert the Figure 4 The red light from the optical lens 30 in the image is transmitted, while the remaining light is filtered out. The blue filter block B is used to transmit the blue light from the optical lens 30 and filter out the remaining light. The green filter block G is used to transmit the green light from the optical lens 30 and filter out the remaining light. In this way, the light from the optical lens 30 can be separated into three primary colors (RGB) after passing through the above-mentioned color filter blocks, so that the electronic device 100 can obtain RGB domain images.

[0087] For example, Figure 5As shown, the image sensor 20 also includes multiple photoelectric conversion elements 220 (or photosensitive elements). In one filter unit 210, the position of each color filter block can correspond to p (p ≥ 1, such as p = 1 or p = 4) photoelectric conversion elements 220. Taking p = 1 as an example, each color filter block can cover one photoelectric conversion element 220. The photoelectric conversion element 220 can be a photodiode, which is used to convert light passing through the color filter block into an electrical signal.

[0088] For example, the photodiode can be manufactured by using a charge coupled device (CCD) process. Figure 4 The photodiode (light represented by arrows in the figure) is converted into an electrical signal, which is then converted into a digital image signal through an analog-to-digital conversion circuit. Alternatively, the photodiode can be manufactured using a complementary metal oxide semiconductor (CMOS) process. The photodiode manufactured using the CMOS process has both N-type and P-type semiconductors. The current generated by these two semiconductors through the complementary effect can be recorded and interpreted by the processing chip and converted into a digital image signal through the digital-to-analog conversion circuit.

[0089] For example, when p=1, Figure 5 As shown, the image sensor 20 may include a color filter 200 and a plurality of photoelectric conversion elements 220. A color filter block in the color filter 200 corresponds to the position of a photoelectric conversion element 220. When the color filter block is the red filter block R, the blue filter block B, or the green filter block G, of the light incident on the color filter block, only light of the same color as the color of the color filter block can pass through and enter the photoelectric conversion element 220 corresponding to the position of the color filter block, where it is converted into an electrical signal.

[0090] In this case, each photoelectric conversion element 220 performs analog-to-digital conversion on the collected electrical signals, obtaining digital image signals that are output via the image sensor 20 as a RAW domain image. The RAW domain image includes multiple pixels, and the digital image signal corresponding to each of the multiple pixels is converted from the electrical signal collected by the aforementioned color filter block and the photoelectric conversion element 220 corresponding to the position of the color filter block. In this embodiment of the present application, a color filter block and the photoelectric conversion element 220 corresponding to the position of the color filter block can constitute a pixel.

[0091] Because a color filter block, such as the blue filter block B, allows only one color of light (e.g., blue light) to pass through, the digital image signal obtained by the pixel in the RAW domain image, which is formed by the blue filter block of the image sensor 20 and the photoelectric conversion element 220 corresponding to the position of the blue filter block, only has one color information (or one color channel), such as the blue channel. This color channel is consistent with the color of the blue filter block.

[0092] For example, Figure 6 As shown, the arrangement of multiple pixels included in the RAW domain image can be called a Bayer array. A Bayer array pixel unit includes one red pixel (R), two green pixels (G), and one blue pixel (B). This pixel unit can also be called an RGGB pixel unit. Exemplarily, the multiple pixels included in the RAW domain image can be divided into P pixel units, each of which includes one red pixel, two green pixels, and one blue pixel.

[0093] The methods in the following embodiments can all be implemented in the electronic device 100 having the above hardware structure. In order to make the technical solution of the present application clearer and easier to understand, the following embodiments describe the technical solution provided by the embodiments of the present application in detail in combination with the software structure of the electronic device 100.

[0094] Figure 7 This is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0095] The software structure of the electronic device can adopt a layered architecture, event-driven architecture, micro-kernel architecture, microservice architecture or cloud architecture. The embodiment of the present application takes the layered architecture as an example to illustrate the software structure of the electronic device.

[0096] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the software structure of the electronic device is divided from top to bottom into: application layer (application, APP), application framework layer (framework, FWK), hardware abstraction layer (hardwareabstraction layer, HAL) and kernel layer (kernel). Figure 7 As shown, the layered architecture can also include Android runtime and system library. For ease of understanding, in the embodiment of this application, Figure 7 Also shown are some Figure 3 Components of the electronic device 100, such as the camera 193, the display screen 194, etc.

[0097] The above application layer can include a series of application packages. Figure 7 As shown, the application package may include APP1, APP2, and APP3, etc. In some embodiments, the application package may include some applications with shooting functions (such as camera applications, etc.). When the electronic device runs the camera application, the electronic device may start the camera and capture RAW domain images through the camera.

[0098] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes predefined functions. The application framework layer provides programming services to the application layer through the API interface. Figure 7 As shown, the application framework layer includes a camera service framework.

[0099] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0100] The core library consists of two parts: one is the functional functions that Java voice needs to call, and the other is the Android core library.

[0101] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files in the application layer and application framework layer as binary files. The virtual machine performs functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0102] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0103] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0104] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support multiple audio encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0105] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing and layer processing.

[0106] A 2D graphics engine is a drawing engine for 2D drawings.

[0107] The hardware abstraction layer is an interface layer between the kernel layer and the hardware, which can be used to abstract the hardware. Figure 7 As shown, the hardware abstraction layer includes the camera interface.

[0108] The kernel layer provides the underlying drivers for various hardware of electronic devices. For example, Figure 7 As shown, the kernel layer includes a camera driver module.

[0109] The following combination Figure 7 The software structure shown here exemplifies the solution of an embodiment of the present application. In some embodiments, when the camera application is launched, the camera application triggers a command to start the camera and calls an API interface provided by the application framework layer to send the command to the camera service framework. The camera service framework calls the camera interface in the hardware abstraction layer and sends the command to the camera driver module. The camera driver module is used to drive camera 193 to capture RAW domain images.

[0110] The camera 193 transmits the collected RAW domain image to the camera interface in the hardware abstraction layer. The camera interface is used to process the RAW domain image and transmit the processed RAW domain image to the display screen 194.

[0111] For example, an ISP may be provided in the camera interface. The camera interface may process the RAW domain image through the ISP. For example, Figure 8 As shown, ISP can process the RAW domain image as follows.

[0112] It is understood that the various processing steps of the ISP illustrated in the embodiments of this application do not constitute specific limitations of this application. In other embodiments of this application, the ISP may include more or fewer steps than shown, or may combine or split certain steps, or arrange the steps differently.

[0113] For example, Figure 8 As shown, black level correction (BLC) 201 is used to correct the black level. Black level refers to the video signal level when no line of light is output on a calibrated display device. Black level correction is performed because, on the one hand, image sensors have dark current, which causes voltage output issues even in the absence of light. On the other hand, because image sensors lack precision during analog-to-digital conversion (for example, a 10-bit pixel value has a valid range of 0 to 1023), the image sensor may be unable to convert information close to zero.

[0114] Bad pixel correction (BPC) 202 is used to detect and correct bad pixels in RAW domain images. Specifically, the bad pixels can be detected by obtaining the pixel values ​​of the pixels surrounding the pixel to be detected, and then performing certain processing. Based on this, the pixel values ​​of the bad pixels can be compensated to achieve the purpose of bad pixel correction.

[0115] Lens shading correction (LSC) 203 is used to eliminate the problem that the color and brightness values ​​around the image are inconsistent with the image center due to the lens optical system.

[0116] Noise reduction (NR) 204 refers to RAW-domain noise reduction. RAW-domain noise reduction is used to reduce noise in an image. Image noise can affect the user's visual experience, and noise reduction can improve image quality to a certain extent.

[0117] Automatic white balance (AWB) 205 is used to restore white objects to white at any color temperature. Due to the influence of color temperature, white will appear yellowish at low color temperatures and bluish at high color temperatures. The purpose of white balance is to ensure that white objects have R=G=B at any color temperature, thus appearing white.

[0118] Color interpolation (Demosaic) 206 is used to make each pixel contain three components of RGB at the same time.

[0119] The color correction matrix (CCM) 207 is used to calibrate the accuracy of colors other than white.

[0120] Global tone mapping (GTM) 208 is used to map the brightness and dynamic range of a high-dynamic range (HDR) image to a low dynamic range (LDR) image, enabling the HDR image to be displayed on an LDR display. The mapping process uses the same transformation function for each pixel in the image, creating a one-to-one mapping relationship.

[0121] Color space transform (CST) 209 is used to convert the RGB domain image into a YUV domain image.

[0122] Color noise reduction (NR Chroma) 210 is used to reduce the hue and saturation noise of YUV color space images (UV). Noise in images can affect the user's visual experience, and noise reduction can improve the image quality to a certain extent.

[0123] Color space transform (CST) 211 is used to convert a YUV domain image into an RGB domain image.

[0124] Local tone mapping (LTM) 212 is used to map the brightness and dynamic range of the HDR image to the standard of the LDR image, so that the high dynamic range HDR image can be displayed on an LDR display. The mapping process divides the image into blocks, and different transformation functions are used for different blocks.

[0125] The gamma processing (Gamma) 213 is used to adjust the brightness value, contrast, dynamic range, etc. of the image by adjusting the gamma curve.

[0126] A two-dimensional color lookup table (TDL) 214 is used to perform color correction on the image.

[0127] Color space transform (CST) 215 is used to convert an RGB domain image into a YUV domain image.

[0128] Sharpening 216 enhances the image clarity by enhancing the edges in the image.

[0129] It should be noted that, in the embodiment of the present application, the electronic device can implement a solution for detecting bad pixels and compensating for the bad pixels in the bad pixel correction 202 .

[0130] The following describes the bad pixel detection process provided in the embodiments of the present application with reference to the accompanying drawings. It should be understood that the following embodiments are merely some examples of the present application and do not constitute a limitation to the present application.

[0131] Generally speaking, in electronic devices that have already left the factory, the maximum number of adjacent bad pixels on the same color channel in the image sensor is three. This is because electronic devices undergo factory inspection before leaving the factory. If the number of adjacent bad pixels on the same color channel is four or more, the electronic device is considered a defective product and will not be sold. Therefore, in actual implementation, bad pixels can include single bad pixels, double bad pixels, and bad pixel clusters (such as triple bad pixels).

[0132] In some embodiments of the present application, for a RAW domain image including multiple pixels captured by a camera (including the aforementioned image sensor 20), when detecting whether the multiple pixels include a single bad pixel, the electronic device sequentially uses each of the multiple pixels as a pixel to be detected to determine whether the pixel to be detected is a single bad pixel. Correspondingly, when detecting whether the multiple pixels include a double bad pixel, the electronic device sequentially uses each two adjacent pixels in the multiple pixels as pixels to be detected to determine whether the pixel to be detected is a double bad pixel.

[0133] It should be noted that in actual implementation, for a single RAW image, the electronic device sequentially detects whether single and double bad pixels exist in multiple pixels included in the RAW image. Specifically, the electronic device first detects each pixel as a single bad pixel. After detecting all pixels, the electronic device then detects whether each pair of adjacent pixels is a double bad pixel.

[0134] The following embodiments respectively introduce the specific implementation process of detecting single bad pixels and double bad pixels in electronic devices. It should be noted that the following embodiments of the present application use the example of detecting whether a hot pixel is a bad pixel as an example.

[0135] (1) Single bad point

[0136] For example, Figure 9 As shown in (1), for a pixel to be detected in the RAW domain image, with the pixel to be detected (e.g., point A) as the center, the pixel value of each of m (e.g., 8) pixels in the first pixel matrix (e.g., a 5×5 matrix) that are in the same color channel (e.g., red channel) as the pixel to be detected is obtained to obtain m (i.e., 8) pixel values. m is a positive integer.

[0137] It should be noted that Figure 9 The first pixel matrix is ​​a 5×5 matrix for illustration purposes only and does not constitute a limitation of the present application. Of course, the first pixel matrix can also be a 3×3 matrix or a 7×7 matrix without limitation. The value of m is not specifically limited and is subject to actual settings. For example, the value of m is related to the size of the color channel and the first pixel matrix. For example, if the first pixel matrix is ​​a 3×3 green channel, m is 4 and the like, which is not described in detail.

[0138] Furthermore, in the embodiment of the present application, in order to reduce the computing power of the electronic device and lower the power consumption of the electronic device, the first pixel matrix can be set as small as possible, such as a 3×3 matrix or a 5×5 matrix.

[0139] For example, Figure 9The pixel value of each pixel shown is related to the number of bits of the pixel. For example, if the number of bits of the pixel is 8 bits, the pixel value is 0 to 255. For another example, if the number of bits of the pixel is 10 bits, the pixel value is 0 to 1023. For another example, if the number of bits of the pixel is 18 bits, the pixel value is 0 to 26143. In the embodiments of the present application, the pixel value of 18 bits is used as an example for illustration.

[0140] It's important to note that the number of bits per pixel is closely related to image quality. The number of bits determines the number of different colors or grayscale levels a pixel can represent—the color depth of the image. A higher bit count allows for a greater number of colors to be represented, resulting in a richer image and higher quality. For example, a pixel with 1 bit can represent the colors 0 and 1, or black and white. Alternatively, an 8-bit pixel can represent 256 colors.

[0141] Afterwards, the electronic device can determine whether the pixel to be detected is a bad pixel based on the acquired pixel value of each of the m pixels in the first pixel matrix that are centered on the pixel to be detected and have the same color channel as the pixel to be detected.

[0142] In some embodiments of the present application, Figure 9 As shown in (1), for a pixel to be detected (such as point A), the electronic device can determine a pixel threshold (or second pixel threshold) based on the maximum value of the 8 pixel values ​​in the 5×5 matrix and a preset variable value. If the pixel value of the pixel to be detected is greater than the pixel threshold, the pixel to be detected is determined to be a bad pixel (i.e., a single bad pixel). If the pixel value of the pixel to be detected is less than the pixel threshold, the pixel to be detected is determined to be not a bad pixel.

[0143] For example, taking the preset variable values ​​including preset variable value 1 and preset variable value 2 as an example, the pixel threshold may satisfy the following expression (1):

[0144] K=Fmax*MP+offset (1);

[0145] Among them, K represents the pixel threshold, Fmax represents the preset variable value 1 (or the third variable value), MP represents the maximum pixel, that is, the maximum value among the 8 pixel values, and offset represents the preset variable value 2 (or the fourth variable value).

[0146] Exemplarily, the preset variable value 1 is a modifiable variable value, and the preset variable value 1 can be modified according to actual conditions. The embodiment of the present application does not limit the specific numerical value of the preset variable value 1.

[0147] Accordingly, the preset variable value 2 is a modifiable variable value that is related to the number of bits in the pixel. For example, when the number of bits in a pixel is 8 bits, the value range of the preset variable value 2 is 0 to 255. For another example, when the number of bits in a pixel is 10 bits, the value range of the preset variable value 2 is 0 to 1023. For another example, when the number of bits in a pixel is 18 bits, the value range of the preset variable 2 is 0 to 262143.

[0148] For example, assuming Fmax = 68 / 64, offset = 11413, then It can be seen that the pixel value of the pixel to be detected (point A) is 92670, which is greater than the pixel threshold K (ie, 92669.8). Therefore, the pixel to be detected, ie, point A, is determined to be a bad pixel.

[0149] In some embodiments, the electronic device may also compensate for bad pixels. For example, the electronic device may compensate for bad pixels by replacing the pixel value of the bad pixel with a pixel threshold value K (i.e., 92669.8); or, replacing the pixel value of the bad pixel with the maximum value among m pixel values ​​(i.e., 76477); or, replacing the pixel value of the bad pixel with the second largest value among m pixel values ​​(i.e., 71766); or, the electronic device may also use other pixel values ​​to replace the pixel value of the bad pixel, etc., without limitation.

[0150] For example, the electronic device can determine a pixel median, pixel mean, or pixel standard deviation based on the pixel values ​​of the eight pixels, without limitation. The electronic device then uses the pixel median, pixel mean, pixel standard deviation, etc. to replace the pixel value of the defective pixel.

[0151] Alternatively, the electronic device may replace the pixel value of the defective pixel with a preset pixel value. The preset pixel value may be a pixel value pre-configured by the electronic device and stored within the electronic device. The specific value of the preset pixel value is not limited and is subject to the actual setting. For example, the preset pixel value may be less than or equal to the maximum value of the m pixel values.

[0152] As an example, Figure 9 As shown in (2), the pixel value of the pixel to be detected is replaced by the maximum value of the m pixel values ​​(i.e., 76477).

[0153] By performing the above processing on each pixel in the RAW domain image, it is possible to detect whether a single bad pixel is included in the multiple pixels.

[0154] In summary, the electronic device can detect a single bad pixel on a RAW domain image captured by an image sensor through the above method and compensate for the single bad pixel, thereby improving the quality of the captured image and further improving the user experience.

[0155] (2) Double bad points

[0156] For example, Figure 10 As shown in (1), for two adjacent pixels to be detected in the RAW domain image, with the two adjacent pixels to be detected (such as point A and point B) as the center, the pixel value of each of m (such as 10) pixels in the first pixel matrix (such as a 5×7 matrix) that are in the same color channel (such as the blue channel) as the two adjacent pixels to be detected is obtained to obtain m (i.e., 10) pixel values.

[0157] It should be noted that, for the first pixel matrix and the example of the corresponding value of m, reference can be made to the above embodiment, which will not be described in detail here. Figure 10 The example of the pixel value of each pixel shown can refer to the above embodiment, and the number of times is not repeated. In addition, the two adjacent pixels to be detected refer to: two consecutive pixels in the same color channel.

[0158] Afterwards, the electronic device can determine whether the two adjacent pixels to be detected are bad pixels based on the pixel value of each pixel in the m pixels in the first pixel matrix that are centered on the two adjacent pixels to be detected and have the same color channel as the two adjacent pixels to be detected.

[0159] In some embodiments of the present application, Figure 10 As shown in (1), for two adjacent pixels to be detected (such as point A and point B), the electronic device can determine the pixel threshold based on the maximum value of the 10 pixel values ​​in the 5×7 matrix and the preset variable value. If the pixel values ​​of the two adjacent pixels to be detected are both greater than the pixel threshold, the two adjacent pixels to be detected are determined to be bad pixels (i.e., double bad pixels). If the pixel values ​​of the two adjacent pixels to be detected are both less than the pixel threshold, the two adjacent pixels to be detected are determined to be not bad pixels.

[0160] Exemplarily, the pixel threshold may satisfy the above expression (1). For an illustration of expression (1), reference may be made to the above embodiment, which will not be described in detail here.

[0161] For example, assuming Fmax = 68 / 64, offset = 1759, then It can be seen that the pixel value of the pixel to be detected (point A) is 59961, and the pixel value of the pixel to be detected (point B) is 73951, both of which are greater than the pixel threshold K (i.e., 59960.6). Therefore, the two adjacent pixels to be detected (points A and B) are determined to be bad pixels (i.e., double bad pixels).

[0162] In some embodiments, the electronic device can also compensate for bad pixels. For an example of compensating for bad pixels, please refer to the above embodiment and will not be described in detail.

[0163] For example, Figure 10 As shown in (2), the pixel value of the bad pixel is replaced by the maximum value of the above 10 pixel values ​​(i.e., 54778).

[0164] By performing the above processing on every two adjacent pixels in the RAW image, it is possible to detect whether the multiple pixels contain double bad pixels. Here, every two adjacent pixels refers to any two adjacent pixels in the multiple pixels included in the RAW image. For example, point A and point B, point B and the adjacent points around point B, etc., will not be detailed here.

[0165] In summary, the electronic device can detect double bad pixels on the RAW domain image captured by the image sensor through the above method and compensate for the double bad pixels, thereby improving the quality of the captured image and further improving the user experience.

[0166] In the related art, electronic devices detect single bad pixels and double bad pixels in sequence. That is, after the electronic device detects a single bad pixel, it continues to detect double bad pixels. However, in the related art, when the electronic device detects double bad pixels, due to the existence of bad pixel clusters, some double bad pixels cannot be detected and eliminated. For example, Figure 11 As shown in (1), for two adjacent pixels to be detected (such as point A and point B) in the RAW domain image, the pixel value of each of m (such as 10) pixels in the first pixel matrix (such as a 5×7 matrix) that are in the same color channel (such as the green channel) as the two adjacent pixels to be detected (such as point A and point B) is obtained with the two adjacent pixels to be detected (such as point A and point B) as the center, so as to obtain m (i.e., 10) pixel values.

[0167] For example, the electronic device can use the above expression (1) to determine whether two adjacent pixels to be detected are bad pixels. For example, assuming Fmax = 1, offset = 0, then all pixels greater than mp (maximum pixel) will be judged as bad pixels, that is, the bad pixel correction strength is the strongest. Figure 11 As shown in point A in (1), when single bad pixel detection is performed, according to expression (1), K = 1*29149+0 = 29149, the pixel value of pixel A is greater than K, so it will be judged as a bad pixel and replaced by 29149; however, when double bad pixel detection is performed, as shown in Figure 11As shown in (2), the maximum value of the 10 pixel values ​​around points A and B is 30548, so K = 1*30548 + 0 = 30548. It can be seen that the pixel values ​​of the pixels to be detected (points A and B) are both 29149, which is less than the pixel threshold K (i.e., 30548). Therefore, the pixels to be detected (points A and B) will not be determined as bad pixels.

[0168] Therefore, the three adjacent pixels to be detected are not processed, which will eventually affect the quality of the captured image and further affect the user experience.

[0169] Based on this, in some other embodiments of the present application, after the electronic device obtains the pixel value of each of m pixels in the first pixel matrix that are in the same color channel as two adjacent pixels to be detected, the electronic device first determines a first pixel threshold based on the m pixel values. If a first pixel value among the m pixel values ​​is greater than the first pixel threshold, the electronic device determines that the first pixel corresponding to the first pixel value is a defective pixel.

[0170] For example, Figure 12 As shown, with two adjacent pixels to be detected (points A and B) as the center, the pixel values ​​of 10 pixels in the green channel within a 5×7 matrix are obtained to obtain 10 pixel values. For example, the 10 pixel values ​​can be 16525, 11614, 9034, 7823, 11476, 12802, 16665, 6781, 2609, and 30548. Based on this, the electronic device can determine the first pixel threshold according to the 10 pixel values.

[0171] For example, the electronic device may determine the first pixel threshold value based on the second largest value among the 10 pixel values, the first variable value, and the second variable value. Exemplarily, the first pixel threshold value may satisfy the following expression (2):

[0172] K1=Fmax*SMP+TBP_offset (2);

[0173] Wherein, K1 represents the first pixel threshold, Fmax represents the first variable value, SMP (sub maximum pixel) represents the second maximum value among the 10 pixel values, and TBP_offset represents the second variable value.

[0174] Exemplarily, the first variable value and the second variable value are modifiable variable values, and there is no limitation on the specific numerical values ​​of the first variable value and the second variable value, which are subject to actual settings.

[0175] For example, the first variable value can be any value in the range [1, 2]. The second variable value is related to the number of bits in the pixel; alternatively, the second variable value can be zero. For an example of the second variable value, please refer to the example of the preset variable value 2 in the above embodiment, and will not be repeated here.

[0176] As an example, when the value of the first variable is 2, the value of the second variable may be zero.

[0177] For example, combined Figure 12 For example, assuming Fmax = 1 and TBP_offset = 1000, then K1 = 1*16665 + 100. It can be seen that among the 10 pixel values, a first pixel value (e.g., 30548) is greater than the first pixel threshold (i.e., 17665). Therefore, the electronic device can determine that the first pixel corresponding to the first pixel value (e.g., point C) is a defective pixel.

[0178] In some embodiments, the electronic device may further compensate for the bad pixel (ie, the first pixel). For example, the electronic device replaces the first pixel value with the first pixel threshold to compensate for the bad pixel; or Figure 13 As shown, the electronic device replaces the first pixel value with the second largest value among the above 10 pixel values.

[0179] In some embodiments of the present application, after the electronic device compensates the first pixel, the electronic device determines whether the two adjacent pixels to be detected (i.e., points A and B) are bad pixels. Alternatively, in other embodiments of the present application, if the first pixel corresponding to the first pixel value (e.g., point C) is not a bad pixel, the electronic device continues to determine whether the two adjacent pixels to be detected (points A and B) are bad pixels.

[0180] As an example, the electronic device can determine whether two adjacent pixels to be detected are bad pixels based on a first pixel threshold. If the pixel values ​​corresponding to the two adjacent pixels to be detected are both greater than the first pixel threshold, the electronic device determines that the two adjacent pixels to be detected are bad pixels. For example, Figure 12 As shown, the pixel values ​​of two adjacent pixels to be detected (point A and point B) are 30471 and 29149, respectively, both greater than the first pixel threshold (i.e., 17665). Therefore, the two adjacent pixels to be detected are bad pixels. If the pixel values ​​of the two adjacent pixels to be detected are both less than the first pixel threshold, the electronic device determines that the two adjacent pixels to be detected are not bad pixels.

[0181] On this basis, the electronic device can also compensate for the bad pixels (i.e., point A and point B). For example, the electronic device replaces the pixel values ​​corresponding to the two adjacent pixels to be detected with the first pixel threshold to compensate for the pixel to be detected. Or, Figure 13 As shown, the electronic device replaces the pixel values ​​corresponding to the two adjacent pixels to be detected with the second largest value (ie 16665) among the above 10 pixel values ​​to compensate the pixels to be detected.

[0182] As another example, after the electronic device compensates the first pixel, the electronic device may also use the above expression (1) to determine whether two adjacent pixels to be detected are bad pixels. For example, Figure 14 As shown, after the first pixel value is replaced, the pixel value of point C is 16665. Based on this, the electronic device obtains the pixel value of each of the n pixels in the second pixel matrix that are in the same color channel as the two adjacent pixels to be detected (i.e., points A and B) with the two adjacent pixels to be detected as the center, to obtain n pixel values; n is a positive integer, and the n pixel values ​​include the replaced first pixel value (i.e., the pixel value of point C after replacement, such as 16665).

[0183] Exemplarily, the electronic device determines a second pixel threshold based on n pixel values. If the pixel values ​​corresponding to two adjacent pixels to be detected are both greater than the second pixel threshold, the electronic device determines that the two adjacent pixels to be detected are bad pixels. If the pixel values ​​corresponding to the two adjacent pixels to be detected are both less than the second pixel threshold, the electronic device determines that the two adjacent pixels to be detected are not bad pixels.

[0184] For examples of the second pixel matrix and the value of n, reference may be made to the above embodiments, which will not be described in detail here.

[0185] Taking the second pixel matrix as a 5×7 matrix as an example, for example, Figure 14 As shown, for two adjacent pixels to be detected, the electronic device can determine the second pixel threshold based on the maximum value of the 10 pixel values ​​in the 5×7 matrix, as well as preset variable values ​​1 and 2. If the pixel values ​​of the two adjacent pixels to be detected are greater than the second pixel threshold, the electronic device determines that the two adjacent pixels to be detected are bad pixels.

[0186] For example, the electronic device may use the above expression (1) to determine the second pixel threshold. For example, assuming Fmax = 68 / 64, offset = 11413, It can be seen that the pixel value of the pixel to be detected (point A) is 30471, which is greater than the second pixel threshold K2 (i.e., 29119.6), and the pixel value of the pixel to be detected (point B) is 29149, which is greater than the second pixel threshold K2 (i.e., 29119.6). Therefore, the above two adjacent pixels to be detected are determined to be bad pixels.

[0187] On this basis, the electronic device replaces the pixel values ​​corresponding to two adjacent pixels with the second pixel threshold to compensate for the pixel to be detected; or, the electronic device replaces the pixel values ​​corresponding to two adjacent pixels to be detected with the maximum value of n pixel values ​​to compensate for the pixel to be detected.

[0188] In summary, the electronic device can first detect bad pixels located near two adjacent pixels to be detected within the first pixel matrix, with the two adjacent pixels being the center. After compensating for the bad pixels, the electronic device can then determine that the two adjacent pixels to be detected are bad pixels. This ensures that the electronic device can detect three adjacent bad pixels (i.e., a bad pixel cluster), thereby improving the accuracy of bad pixel cluster detection.

[0189] For example, Figure 15 As shown, by adopting the solution provided in the embodiment of the present application, the electronic device can detect the bad pixel clusters and compensate for the large dark spots caused by the bad pixel clusters, thereby eliminating the large dark spots caused by the bad pixel clusters, improving the picture quality, and further improving the user experience.

[0190] In addition, in an embodiment of the present application, due to the aging of the image sensor, there may be four (or more) adjacent bad pixels in the RAW domain image collected by the image sensor. Exemplarily, for the detection method of four (or more) adjacent bad pixels, the pixel values ​​of each pixel around the four adjacent pixels to be detected are obtained with the four adjacent pixels to be detected as the center, and the pixel threshold is determined based on the pixel value obtained. If the pixel values ​​of the four adjacent pixels to be detected are all greater than the pixel threshold, the electronic device determines that the four adjacent pixels to be detected are bad pixels. If the pixel values ​​of the four adjacent pixels to be detected are all less than the pixel threshold, the electronic device determines that the four adjacent pixels to be detected are not bad pixels. Its specific implementation process can refer to the above embodiment and will not be repeated here.

[0191] In addition, in the embodiment of the present application, the detection of whether there are bad pixels and compensation for bad pixels in a scene with bright spots in a dark environment are used as an example. For a scene with dark spots in a bright environment, expression (3) can be used to detect whether there are bad pixels.

[0192] Fmin*MP+offset(3); wherein Fmin represents the first variable value, MP represents the minimum pixel value (i.e., the minimum pixel value around the pixel to be detected), and offset represents the second variable value.

[0193] The principle of detecting whether a dark pixel is a bad pixel and compensating for the bad pixel is similar to the principle of detecting a bright pixel. The specific implementation process can be referred to the above embodiment and will not be repeated here.

[0194] As an example, if the pixel value of the pixel to be detected is less than the minimum pixel value, it is determined that the pixel to be detected is a dark point (ie, a bad pixel in a bright environment).

[0195] It should be noted that the contents recorded in each embodiment of the present application can explain the technical solutions in other embodiments of the present application, and the technical features recorded in each embodiment can also be applied in other embodiments, and the technical features in other embodiments can be combined to form a new solution. This application only exemplifies several embodiments for illustration, and does not mean that this application is limited to this.

[0196] The present application provides an electronic device that may include a camera, a memory, and one or more processors; the memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs the various functions or steps in the above embodiments. The structure of the electronic device can refer to the above Figure 3 The structure of the electronic device 100 is shown.

[0197] The present application also provides a chip system for use in electronic devices. Figure 16 As shown, the chip system 1100 includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 can be Figure 3 The processor 110 shown is, on this basis, the interface circuit 1102 can be, for example, an interface circuit between the processor 1101 and an external memory; or an interface circuit between the processor 1101 and an internal memory.

[0198] The processor 1101 and the interface circuit 1102 can be interconnected via lines. For example, the interface circuit 1102 can be used to receive signals from other devices (such as the memory of the electronic device 100). For another example, the interface circuit 1102 can be used to send signals to other devices (such as the processor 1101). Exemplarily, the interface circuit 1102 can read instructions stored in the memory and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the electronic device can perform the various functions or steps performed by the mobile phone in the above embodiment. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiments of the present application.

[0199] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the various functions or steps executed by the electronic device in the above method embodiment.

[0200] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the functions or steps executed by the electronic device in the above method embodiment.

[0201] It should be noted that the terms "first" and "second" in the specification, claims and drawings of this application are used to distinguish different objects, rather than to describe a specific order. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "multiple" means two or more.

[0202] Furthermore, the terms "include," "comprise," and "have," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0203] It should be understood that in this application, "at least one (item)" refers to one or more. "Multiple" refers to two or more. "At least two (items)" refers to two or three and more than three. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple. “When” and “if” both mean that corresponding measures will be taken under certain objective circumstances. They do not limit the time, nor do they require any judgment action when they are implemented, nor do they mean that there are other limitations.

[0204] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner to facilitate understanding.

[0205] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules according to the system, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0207] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0208] In addition, the functional units in various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0209] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0210] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for detecting bad pixels, characterized in that: The method is applied to an electronic device, the electronic device including a camera, the camera being used to capture an original RAW domain image, the RAW domain image including a plurality of pixels; wherein the electronic device sequentially uses every two adjacent pixels in the plurality of pixels as pixels to be detected, and detects whether there are bad pixels in the plurality of pixels; the method comprising: The electronic device obtains the pixel value of each of m pixels in the first pixel matrix that are in the same color channel as the pixel to be detected, with the pixel to be detected as the center, to obtain m pixel values; m is a positive integer; The electronic device determines a first pixel threshold according to the m pixel values; If a first pixel value among the m pixel values ​​is greater than the first pixel threshold, the electronic device determines that a first pixel corresponding to the first pixel value is a bad pixel; After the electronic device compensates the first pixel, the electronic device determines whether the pixel to be detected is a bad pixel.

2. The method according to claim 1, characterized in that The electronic device determines that the pixel to be detected is a bad pixel, including: If the pixel values ​​corresponding to the pixels to be detected are all greater than the first pixel threshold, the electronic device determines that the pixels to be detected are bad pixels.

3. The method according to claim 2, characterized in that The method further comprises: The electronic device replaces the pixel value corresponding to the pixel to be detected with the first pixel threshold to compensate for the pixel to be detected; or, The electronic device replaces the pixel value corresponding to the pixel to be detected with the second largest value among the m pixel values ​​to compensate for the pixel to be detected.

4. The method according to any one of claims 1 to 3, characterized in that The electronic device compensating the first pixel includes: The electronic device replaces the first pixel value with the first pixel threshold; or, The electronic device replaces the first pixel value with the second largest value among the m pixel values.

5. The method according to claim 4, characterized in that The electronic device determines whether the pixel to be detected is a bad pixel, including: The electronic device obtains, with the pixel to be detected as the center, a pixel value of each of n pixels in a second pixel matrix that are in the same color channel as the pixel to be detected, to obtain n pixel values; n is a positive integer, and the n pixel values ​​include the first pixel value to be replaced; The electronic device determines a second pixel threshold according to the n pixel values; If the pixel value corresponding to the pixel to be detected is greater than the second pixel threshold, the electronic device determines that the pixel to be detected is a bad pixel.

6. The method according to claim 5, characterized in that The method further comprises: The electronic device replaces the pixel value corresponding to the pixel to be detected with the second pixel threshold to compensate for the pixel to be detected; or, The electronic device replaces the pixel value corresponding to the pixel to be detected with the maximum value of the n pixel values ​​to compensate for the pixel to be detected.

7. The method according to any one of claims 1 to 3, 5 or 6, characterized in that The electronic device determines a first pixel threshold according to the m pixel values, including: The electronic device determines the first pixel threshold according to the first variable value, the second variable value and the second largest value among the m pixel values; The first variable value is any value in [1, 2]; the second variable value is related to the number of bits of the pixel, or the second variable value is zero.

8. The method according to claim 7, characterized in that The first pixel threshold satisfies the following expression: ； in, represents the first pixel threshold, represents the first variable value, represents the second largest value, Represents the second variable value.

9. The method according to claim 5 or 6, characterized in that The electronic device determines a second pixel threshold according to the n pixel values, including: The electronic device determines the second pixel threshold according to the third variable value, the fourth variable value and the maximum value of the n pixel values; The third variable value is any value in [1, 2]; the fourth variable value is related to the number of bits of the pixel; or the fourth variable value is zero.

10. The method according to claim 9, characterized in that The second pixel threshold satisfies the following expression: ； in, represents the second pixel threshold, represents the third variable value, represents the maximum value, Represents the value of the fourth variable.

11. The method according to any one of claims 1-3, 5-6, 8 or 10, characterized in that The plurality of pixels are divided into P pixel units, each of the P pixel units includes one red pixel, two green pixels and one blue pixel; P≥1.

12. An electronic device, characterized in that: include: memory and one or more processors, cameras; The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 11.

13. A chip system, characterized in that: include: at least one processor and an interface; The interface is used to receive instructions and transmit them to the at least one processor; The at least one processor executes the instructions so that the electronic device performs the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 11.

Images