Dead pixel detection method and electronic equipment

Abstract

The invention provides a dead pixel detection method and electronic equipment, and relates to the technical field of image processing, and the method comprises the steps: taking a to-be-detected pixel as a center through the electronic equipment, obtaining the pixel value of each pixel in m pixels which are in the same color channel with the to-be-detected pixel in a first pixel matrix, so as to obtain m pixel values; m is a positive integer. The electronic equipment determines a first pixel threshold value according to the m pixel values; if a first pixel value greater than a first pixel threshold exists in the m pixel values, the electronic equipment determines a first pixel corresponding to the first pixel value as a dead pixel; after the electronic equipment compensates the first pixel, the electronic equipment determines whether the to-be-detected pixel is the dead pixel or not, so that three adjacent dead pixel pixels existing on the RAW domain image caused by the defect of the image sensor can be detected, and the detection accuracy is improved.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on January 29, 2024, with application number 202410124205.4 and invention name “A method and electronic device for detecting bad pixels”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The embodiments of the present application relate to the field of image processing technology, and more particularly to a method for detecting bad pixels and an electronic device. Background Art

[0003] With the development of science and technology and manufacturing processes, the image sensor used in the camera of electronic equipment, as a photoelectric sensor, has greatly met people's needs for image acquisition and processing. However, during the production and actual use process, due to defects in the material structure of the image sensor, defects in the manufacturing process, aging and other problems, the pixels on the image output by the image sensor may be bad, affecting the shooting effect. Summary of the invention

[0004] The present application provides a bad pixel detection method and electronic device, which can detect three adjacent bad pixels existing on a RAW domain image caused by defects of an image sensor, thereby improving the detection accuracy.

[0005] This application adopts the following technical solutions:

[0006] In a first aspect, a method for detecting bad pixel is provided, which is applied to an electronic device, wherein the electronic device includes a camera, and the camera is used to collect a RAW domain image, wherein the RAW domain image includes a plurality of pixels; wherein the electronic device sequentially uses every two adjacent pixels in the plurality of pixels as pixels to be detected, and detects whether there are bad pixel pixels in the plurality of pixels. Specifically, the method may include:

[0007] The electronic device takes the pixel to be detected as the center, obtains the pixel value of each pixel in the first pixel matrix and in the same color channel as the pixel to be detected, so as to obtain m pixel values; m is a positive integer. The electronic device determines the first pixel threshold value according to the m pixel values; if there is a first pixel value greater than the first pixel threshold value among the m pixel values, the electronic device determines that the first pixel corresponding to the first pixel value is a bad pixel. After the electronic device compensates the first pixel, the electronic device determines whether the pixel to be detected is a bad pixel.

[0008] Based on the first aspect, the electronic device first detects the bad pixels around the pixel to be detected with the pixel to be detected as the center; then after compensating for the surrounding bad pixels, it detects whether the pixel to be detected is a bad pixel, thereby detecting three adjacent bad pixels in the RAW domain image, thereby improving the detection accuracy.

[0009] In one design, if the pixel values ​​corresponding to the pixels to be detected are all greater than the first pixel threshold, the electronic device determines that the pixels to be detected are bad pixels. In this way, the electronic device can directly use the first pixel threshold to detect whether the pixels to be detected are bad pixels, which reduces the complexity of detection and also reduces power consumption.

[0010] In one design, after detecting that the pixel to be detected is a bad pixel, the electronic device replaces the pixel value corresponding to the pixel to be detected with the first pixel threshold to compensate the pixel to be detected; or, the electronic device replaces the pixel value corresponding to the pixel to be detected with the second largest value among m pixel values ​​to compensate the pixel to be detected. In this way, by compensating the pixel to be detected, the image quality can be improved, thereby improving the user experience.

[0011] The pixel value corresponding to the pixel to be detected refers to: the pixel values ​​of two adjacent pixels corresponding to the pixel to be detected.

[0012] In one design, the electronic device can determine whether the pixel to be detected is a bad pixel in the following manner. Specifically, the electronic device takes the pixel to be detected as the center, obtains the pixel value of each pixel in the second pixel matrix of n pixels in the same color channel as the pixel to be detected, so as to obtain n pixel values; n is a positive integer, and the n pixel values ​​include the first pixel value to be replaced. Then, the electronic device determines the second pixel threshold value based on the n pixel values; if the pixel value corresponding to the pixel to be detected is greater than the second pixel threshold value, the electronic device determines that the pixel to be detected is a bad pixel.

[0013] In this way, by determining whether the pixel to be detected is a bad pixel in the above manner, the accuracy of detection can be improved.

[0014] In one design method, after the electronic device determines that the pixel to be detected is a bad pixel, the electronic device replaces the pixel value corresponding to the pixel to be detected with a second pixel threshold to compensate for the pixel to be detected; or, the electronic device replaces the pixel value corresponding to the pixel to be detected with the maximum value of n pixel values ​​to compensate for the pixel to be detected.

[0015] In this way, by compensating the pixel values ​​of two adjacent pixels corresponding to the pixel to be detected, the image quality can be improved, thereby improving the user experience.

[0016] In one design, the electronic device may replace the first pixel value with the first pixel threshold to compensate for the first pixel; or the electronic device may replace the first pixel value with the second largest value among the m pixel values ​​to compensate for the first pixel. In this way, by compensating for the first pixel, the image quality can be improved, thereby improving the user experience.

[0017] In one design, the electronic device can determine the first pixel threshold in the following manner. Specifically, the electronic device determines the first pixel threshold according to the first variable value, the second variable value, and the second largest value among the m pixel values; wherein the first variable value is any value in [1, 2]; the second variable value is related to the number of bits of the pixel, or the second variable value is zero.

[0018] In this way, the first pixel threshold determined in this way is more accurate, thereby improving the accuracy of detection.

[0019] In one design, the first pixel threshold satisfies the following expression:

[0020] K1=Fmax*SMP+TBP_offset;

[0021] Among them, K1 represents the first pixel threshold, Fmax represents the first variable value, SMP represents the second maximum value, and TBP_offse represents the second variable value.

[0022] In one design, the electronic device can determine the second pixel threshold in the following manner. Specifically, the electronic device determines the second pixel threshold according to the third variable value, the fourth variable value, and the maximum value of n pixel values; wherein the third variable value is any value in [1, 2]; the fourth variable value is related to the number of bits of the pixel; or the fourth variable value is zero.

[0023] In this way, the second pixel threshold determined in this way is more accurate, thereby improving the accuracy of detection.

[0024] In one design, the second pixel threshold satisfies the following expression:

[0025] K2=Fmax*MP+offset;

[0026] Among them, K2 represents the second pixel threshold, Fmax represents the third variable value, MP represents the maximum value, and offset represents the fourth variable value.

[0027] In one design, a plurality of pixels included in a RAW domain image are divided into P pixel units, each of the P pixel units includes a red pixel, two green pixels and a blue pixel. The pixel unit may also be referred to as an RGGB pixel unit, where P≥1.

[0028] In a second aspect, an electronic device is provided, which has the function of implementing any one of the functions in the first aspect, and the function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0029] According to a third aspect, an electronic device is provided, comprising: a memory, one or more processors, and a camera; the memory stores computer program code, the computer program code comprising computer instructions; when the computer instructions are executed by the processor, the electronic device executes the method described in the first aspect or any one of the methods described in the first aspect.

[0030] In a fourth aspect, a chip system is provided, which includes: at least one processor and an interface, the interface being used to receive instructions and transmit them to at least one processor; at least one processor executes the instructions so that the electronic device executes any one of the methods described in the first aspect above.

[0031] In a fifth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer can execute any of the methods described in the first aspect.

[0032] In a sixth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect.

[0033] Among them, the technical effects brought about by any implementation method in the second to sixth aspects can refer to the technical effects brought about by different implementation methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of an interface for displaying dark spots in an image provided by an embodiment of the present application;

[0035] Figure 2 A schematic diagram of another interface for displaying dark spots in an image provided by an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of the structure of a camera provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram of the structure of an image sensor provided in an embodiment of the present application;

[0039] Figure 6A schematic diagram of the structure of a pixel unit provided in an embodiment of the present application;

[0040] Figure 7 A schematic diagram of a software framework of an electronic device provided in an embodiment of the present application;

[0041] Figure 8 A schematic diagram of an ISP processing process provided in an embodiment of the present application;

[0042] Fig. 9 A schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 1 ;

[0043] Fig.10 A schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 2 ;

[0044] Fig.11 A schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 3 ;

[0045] Fig.12 A schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 4 ;

[0046] Fig.13 A schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 5 ;

[0047] Fig.14 A schematic diagram of the principle of a bad pixel detection method provided in an embodiment of the present application Figure 6 ;

[0048] Fig.15 A schematic diagram showing a comparison between a bad pixel before and after elimination provided in an embodiment of the present application;

[0049] Fig.16 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] Before describing the embodiments of the present application, the technical terms involved in the present application are first explained.

[0051] Bad pixels: Due to the process (or usage time) of the image sensor, there will be some defective pixels in the RAW domain image output by the image sensor. These pixels may be darker or brighter than the adjacent pixels, and are therefore called bad pixels (or bad pixels).

[0052] In some embodiments, bad pixels can be divided into single bad pixels, double bad pixels and bad pixel clusters according to the number of adjacent bad pixels on the same color channel. A single bad pixel refers to a single bad pixel within a certain range on the same color channel. A double bad pixel refers to two adjacent and continuous bad pixels within a certain range on the same color channel. A bad pixel cluster refers to three or more adjacent and continuous bad pixels within a certain range on the same color channel.

[0053] In actual implementation, bad pixels will affect the shooting effect, such as affecting the image quality, and thus affecting the user's shooting experience. For example, when shooting in a dark environment, if there is a bad pixel in the image sensor, the captured image will show a bright spot. Or, when shooting in a bright environment, if there is a bad pixel in the image sensor, the captured image will show a dark spot. For example, Figure 1 As shown, the captured image includes three dark spots.

[0054] It should be noted that if there are bad pixel clusters on the image sensor, the captured image will show a large area of ​​dark spots (or bright spots). Figure 2 As shown, a large area of ​​dark spots appears on the captured image.

[0055] Based on the above, an embodiment of the present application provides a method for detecting bad pixels, which is applied to an electronic device including a camera. The method can improve the quality of captured pictures and further improve the user experience by detecting bad pixels existing in the RAW domain image captured by the image sensor and compensating for the bad pixels.

[0056] The technical solution provided by the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

[0057] A method for detecting a bad pixel provided in an embodiment of the present application can be applied to an electronic device with a shooting function. The electronic device may be, for example, a mobile phone, a sports camera (GoPro), a digital camera, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, etc. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device.

[0058] For example, Figure 3 A schematic structural diagram of an electronic device 100 is shown.

[0059] Among them, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a camera 193 and a display screen 194, etc.

[0060] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0061] The processor 110 may include one or more processing units, for example, the processor 110 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0062] Among them, the controller can be the nerve center and command center of the electronic device 100.

[0063] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0064] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an I2C interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface 130, etc.

[0065] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0066] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0067] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can execute the above instructions stored in the internal memory 121, so that the electronic device 100 performs the methods provided in some embodiments of the present application, as well as various functional applications and data processing. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system; the program storage area can also store one or more applications (such as a gallery, contacts, etc.). The data storage area can store data (such as photos, contacts, etc.) created during the use of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory; it may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, universal flash storage (UFS), etc. In other embodiments, the processor 110 executes the instructions stored in the internal memory 121, and / or the instructions stored in the memory provided in the processor, so that the electronic device 100 performs the methods provided in the embodiments of the present application, as well as various functional applications and data processing.

[0068] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0069] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 can receive input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160.

[0070] The power management module 141 may be used to monitor performance parameters such as battery capacity, battery cycle times, battery charging voltage, battery discharging voltage, and battery health status (eg, leakage, impedance).

[0071] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0072] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0073] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include one or more filters, switches, power amplifiers, low noise amplifiers (LAN), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be placed in the processor 110. In some embodiments, at least some of the functions of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0074] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating one or more communication processing modules. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0075] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0076] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0077] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-LED, Micro-LED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0078] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0079] Among them, ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, color, etc. of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0080] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0081] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0082] Of course, the electronic device 100 provided in the embodiment of the present application may also include one or more devices such as a positioning module 181, a button 190, a motor 191, an indicator 192 and a SIM card interface [1~N] 195, without limitation.

[0083] In some embodiments of the present application, the camera 193 can be used as a rear camera and arranged on one side of the housing of the electronic device 100. Alternatively, in other embodiments of the present application, the camera 193 can be used as a front camera and arranged on one side of the display screen 194 of the electronic device 100.

[0084] For example, the camera 193 may include Figure 4 The image sensor 20 shown in the figure and the optical lens 30 disposed on the incident light side of the image sensor 20 . The optical lens 30 can allow external light to be incident on the image sensor 20 .

[0085] For example, the image sensor 20 may include: Figure 5 The filter 200 shown. The filter 200 may include filter units 210 arranged in an array. The filter unit 210 may include at least three color filter blocks for transmitting three primary colors of light, such as red (red, R) light, blue (blue, B) light, and green (green, G) light, respectively, such as red filter block R, blue filter block B, and green filter block G. Since the human eye is more sensitive to green, in the same filter unit 210, the number of green filter blocks G may be more than the number of other color filter blocks. For example, in the case where a filter unit 210 is arranged in the form of a 2×2 matrix, any of the filter units 210 may include a red filter block R, a blue filter block B, and two green filter blocks G.

[0086] Based on this, the red filter block R is used to convert the Figure 4 The red light in the light from the optical lens 30 is transmitted, and the rest of the light is filtered out. The blue filter block B is used to transmit the blue light in the light from the optical lens 30, and filter out the rest of the light. The green filter block G is used to transmit the green light in the light from the optical lens 30, and filter out the rest of the light. In this way, the light from the optical lens 30 can be separated into three primary colors (RGB) after passing through the above-mentioned color filter blocks, so that the electronic device 100 can obtain an RGB domain image.

[0087] For example, Figure 5As shown, the image sensor 20 also includes a plurality of photoelectric conversion elements 220 (or referred to as photosensitive elements). In the one filter unit 210, the position of each color filter block may correspond to p (p ≥ 1, such as p = 1 or p = 4) photoelectric conversion elements 220. Taking p = 1 as an example, each color filter block may cover one photoelectric conversion element 220. The photoelectric conversion element 220 may be a photodiode, which is used to convert the light passing through the color filter block into an electrical signal.

[0088] Exemplarily, the photodiode can be manufactured by using a charge coupled device (CCD) process. Figure 4 The photodiode (indicated by the arrow in the figure) is converted into an electrical signal, and then the electrical signal is converted into a digital image signal through an analog-to-digital conversion circuit. Alternatively, the photodiode can be prepared using a complementary metal oxide semiconductor (CMOS) process. The photodiode prepared by the CMOS process has both N-type and P-type semiconductors. The current generated by the complementary effect of these two semiconductors can be recorded and interpreted by the processing chip, and converted into a digital image signal through a digital-to-analog conversion circuit.

[0089] For example, when p=1, Figure 5 As shown, the image sensor 20 may include a filter 200 and a plurality of photoelectric conversion elements 220, wherein a color filter block in the filter 200 corresponds to the position of a photoelectric conversion element 220. When the color filter block is the red filter block R, the blue filter block B or the green filter block G, of the light incident on the color filter block, only the light consistent with the color of the color filter block can pass through and enter the photoelectric conversion element 220 corresponding to the position of the color filter block, and is converted into an electrical signal under the action of the photoelectric conversion element 220.

[0090] In this case, each photoelectric conversion element 220 converts the collected electrical signal into digital image signals after analog-to-digital conversion, and outputs the digital image signals as RAW domain images through the image sensor 20. The RAW domain image includes multiple pixels, and the digital image signals corresponding to each of the multiple pixels are converted by the above-mentioned one color filter block and the electrical signal collected by the photoelectric conversion element 220 corresponding to the position of the color filter block. In the embodiment of the present application, one color filter block and the photoelectric conversion element 220 corresponding to the position of the color filter block constitute one pixel.

[0091] Since the color filter block, for example, the blue filter block B, only allows light of one color (for example, blue light) to pass through, the digital image signal obtained by the pixel composed of the blue filter block of the image sensor 20 and the photoelectric conversion element 220 corresponding to the position of the blue filter block in the RAW domain image only has one color information (or one color channel), such as a blue channel. The color channel is consistent with the color of the blue filter block.

[0092] For example, Figure 6 As shown, the arrangement of the multiple pixels included in the RAW domain image can be called a Bayer array. The Bayer array is a pixel unit including a red pixel (R), two green pixels (G) and a blue pixel (B), and the pixel unit can also be called an RGGB pixel unit. Exemplarily, the multiple pixels included in the RAW domain image can be divided into P pixel units, each of the P pixel units includes a red pixel, two green pixels and a blue pixel.

[0093] The methods in the following embodiments can all be implemented in the electronic device 100 having the above hardware structure. In order to make the technical solution of the present application clearer and easier to understand, the following embodiments are combined with the software structure of the electronic device 100 to introduce the technical solution provided by the embodiments of the present application in detail.

[0094] Figure 7 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0095] The software structure of the electronic device may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture or a cloud architecture. The embodiment of the present application takes a layered architecture as an example to exemplify the software structure of the electronic device.

[0096] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the software structure of the electronic device is divided from top to bottom into: application layer (application, APP), application framework layer (framework, FWK), hardware abstraction layer (hardwareabstraction layer, HAL) and kernel layer (kernel). Figure 7 As shown, the layered architecture may also include an Android runtime and a system library. For ease of understanding, in the embodiments of the present application, Figure 7 It also shows some Figure 3 Components of the electronic device 100, such as a camera 193, a display screen 194, etc.

[0097] The above application layer may include a series of application packages. Figure 7 As shown, the application package may include APP1, APP2, and APP3, etc. In some embodiments, the application package may include some applications with shooting functions (such as camera applications, etc.). When the electronic device runs the camera application, the electronic device may start the camera and collect RAW domain images through the camera.

[0098] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes predefined functions. The application framework layer provides programming services to the application layer through the API interface. Figure 7 As shown, the application framework layer includes a camera service framework.

[0099] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0100] The core library consists of two parts: one part is the functional functions that Java voice needs to call, and the other part is the Android core library.

[0101] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0102] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0103] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0104] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0105] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing, etc.

[0106] A 2D graphics engine is a drawing engine for 2D drawings.

[0107] The hardware abstraction layer is an interface layer between the kernel layer and the hardware, and can be used to abstract the hardware. Figure 7 As shown, the hardware abstraction layer includes a camera interface.

[0108] The kernel layer provides low-level drivers for various hardware of electronic devices. For example, Figure 7 As shown, the kernel layer includes a camera driver module.

[0109] Combine the following Figure 7 The software structure shown in the figure exemplifies the scheme of the embodiment of the present application. In some embodiments, when the camera application is started, the camera application triggers the instruction to start the camera, and calls the API interface provided by the application framework layer to send the instruction to start the camera to the camera service framework. The camera service framework calls the camera interface in the hardware abstraction layer and sends the instruction to start the camera to the camera driver module. The camera driver module is used to drive the camera 193 to collect RAW domain images.

[0110] The camera 193 transmits the collected RAW domain image to the camera interface in the hardware abstraction layer. The camera interface is used to perform image processing on the RAW domain image and transmit the processed RAW domain image to the display screen 194.

[0111] For example, an ISP may be provided in the camera interface. The camera interface may perform image processing on the RAW domain image through the ISP. For example, Figure 8 As shown, ISP can process the RAW domain image as follows.

[0112] It is to be understood that the various processing procedures of the ISP illustrated in the embodiments of the present application do not constitute specific limitations on the present application. In other embodiments of the present application, the ISP may include more or fewer steps than shown, or combine certain steps, or split certain steps, or arrange different steps.

[0113] For example, Figure 8 As shown, black level correction (BLC) 201 is used to correct the black level. Black level refers to the video signal level when there is no bright output on a display device that has been calibrated. Black level correction is performed on the one hand because the image sensor has dark current, which causes the image sensor to have voltage output problems even when there is no light. On the other hand, because the image sensor is not accurate enough when performing analog-to-digital conversion, taking 10 bits as an example, the effective range of the pixel value of each pixel is 0 to 1023, and the image sensor may not be able to convert information close to 0.

[0114] Bad pixel correction (BPC) 202 is used to detect bad pixels on the RAW domain image and correct the bad pixels. Specifically, the pixel to be detected can be taken as the center, the pixel values ​​of the pixels around the pixel to be detected can be obtained, and then certain processing can be performed to detect the bad pixels. On this basis, the pixel values ​​of the bad pixels can be compensated to achieve the purpose of correcting the bad pixels.

[0115] Lens shading correction (LSC) 203 is used to eliminate the problem that the color and brightness values ​​around the image are inconsistent with the image center due to the lens optical system.

[0116] Noise reduction (NR) 204 refers to RAW domain noise reduction. RAW domain noise reduction is used to reduce noise in an image. Noise in an image can affect the user's visual experience, and noise reduction can improve the image quality of the image to a certain extent.

[0117] The automatic white balance (AWB) 205 is used to restore white objects to white at any color temperature. Due to the influence of color temperature, white will be yellowish at low color temperatures and blueish at high color temperatures. The purpose of white balance is to make white objects R=G=B at any color temperature, so that they appear white.

[0118] Color interpolation (Demosaic) 206 is used to make each pixel contain three components of RGB at the same time.

[0119] The color correction matrix (CCM) 207 is used to calibrate the accuracy of colors other than white.

[0120] Global tone mapping (GTM) 208 is used to map the brightness and dynamic range of a high-dynamic range (HDR) image to a low dynamic range (LDR) image standard, so that the high-dynamic range HDR image can be displayed on an LDR display. During the mapping process, each pixel of the image uses the same transformation function, which is a one-to-one mapping relationship.

[0121] Color space transform (CST) 209 is used to convert the RGB domain image into a YUV domain image.

[0122] The color noise reduction (NR Chroma) 210 is used to reduce the hue and saturation noise of the YUV color space image (UV). The noise in the image will affect the user's visual experience, and the image quality can be improved to a certain extent through noise reduction.

[0123] Color space transform (CST) 211 is used to convert a YUV domain image into an RGB domain image.

[0124] Local tone mapping (LTM) 212 is used to map the brightness and dynamic range of the HDR image to the standard of the LDR image, so that the high dynamic range HDR image can be displayed on the LDR display. During the mapping process, the image is divided into blocks, and different blocks use different transformation functions.

[0125] The gamma processing (Gamma) 213 is used to adjust the brightness value, contrast, dynamic range, etc. of the image by adjusting the gamma curve.

[0126] A two-dimensional color lookup table (TDL) 214 is used to perform color correction on the image.

[0127] Color space transform (CST) 215 is used to convert the RGB domain image into a YUV domain image.

[0128] Sharpening 216 improves image clarity by enhancing the edges in the image.

[0129] It should be noted that in the embodiment of the present application, the electronic device can implement a solution for detecting bad pixel and compensating the bad pixel in the bad pixel correction 202 .

[0130] The following is an example of the bad pixel detection process provided in the embodiments of the present application in conjunction with the drawings of the specification. It can be understood that the following embodiments are merely some examples of the present application and do not constitute a limitation of the present application.

[0131] Generally speaking, in electronic devices that have already left the factory, the number of adjacent bad pixels on the same color channel in the image sensor is at most three. This is because the electronic devices have been tested before leaving the factory. If the number of adjacent bad pixels on the same color channel is four or more, the electronic device is considered to be a substandard product and will not be sold in the future. Therefore, in actual implementation, bad pixel pixels may include single bad pixels, double bad pixels, and bad pixel clusters (such as three bad pixels).

[0132] In some embodiments of the present application, for a RAW domain image including multiple pixels captured by a camera (including the above-mentioned image sensor 20), when detecting whether a single bad pixel is included in the multiple pixels, the electronic device sequentially uses each pixel in the multiple pixels as a pixel to be detected to determine whether the pixel to be detected is a single bad pixel. Correspondingly, when detecting whether a double bad pixel is included in the multiple pixels, the electronic device sequentially uses each two adjacent pixels in the multiple pixels as pixels to be detected to determine whether the pixel to be detected is a double bad pixel.

[0133] It should be noted that, in actual implementation, for the same RAW domain image, the electronic device sequentially detects whether there are single bad pixels and double bad pixels in the multiple pixels included in the RAW domain image. That is to say, the electronic device first uses each pixel as a pixel to be detected to detect whether the pixel to be detected is a single bad pixel. After detecting all pixels, the electronic device then uses every two adjacent pixels as pixels to be detected to detect whether the pixel to be detected is a double bad pixel.

[0134] The following embodiments respectively introduce the specific implementation process of detecting a single bad pixel and a double bad pixel of an electronic device. It should be noted that the following embodiments of the present application are illustrated by taking the detection of whether a hot pixel is a bad pixel as an example.

[0135] (I) Single bad point

[0136] For example, Fig. 9 As shown in (1), for a pixel to be detected in the RAW domain image, the pixel to be detected (such as point A) is taken as the center, and the pixel value of each pixel in the m (such as 8) pixels in the first pixel matrix (such as a 5×5 matrix) that are in the same color channel (such as the red channel) as the pixel to be detected is obtained to obtain m (i.e. 8) pixel values. m is a positive integer.

[0137] It should be noted that Fig. 9 The first pixel matrix is ​​a 5×5 matrix as an example for illustration, which does not constitute a limitation of the present application. Of course, the first pixel matrix can also be a 3×3 matrix, or a 7×7 matrix, without limitation. There is no specific limitation on the value of m, which is subject to the actual setting. Exemplarily, the value of m is related to the size of the color channel and the first pixel matrix. For example, in the green channel of the first pixel matrix of 3×3, m is 4, etc., which will not be repeated.

[0138] In addition, in the embodiment of the present application, in order to reduce the computing power of the electronic device and reduce the power consumption of the electronic device, the first pixel matrix can be set as small as possible, such as a 3×3 matrix or a 5×5 matrix.

[0139] For example, Fig. 9The pixel value of each pixel shown is related to the bit number of the pixel. For example, if the bit number of the pixel is 8 bits, the pixel value is 0 to 255. For another example, if the bit number of the pixel is 10 bits, the pixel value is 0 to 1023. For another example, if the bit number of the pixel is 18 bits, the pixel value is 0 to 26143. In the embodiment of the present application, the pixel bit number of 18 bits is used as an example for illustration.

[0140] It should be noted that the number of bits of a pixel has a certain relationship with the image quality. The number of bits determines the number of different colors or grayscale levels that a pixel can represent, that is, the color depth of the image. The higher the number of bits, the more colors can be represented, the richer the image colors, and the higher the image quality. For example, if the number of bits of a pixel is 1 bit, it can represent two colors, 0 and 1, that is, black and white. For another example, if the number of bits of a pixel is 8 bits, it can represent 256 colors.

[0141] Afterwards, the electronic device can determine whether the pixel to be detected is a bad pixel based on the acquired pixel value of each pixel in the m pixels in the first pixel matrix that are centered on the pixel to be detected and have the same color channel as the pixel to be detected.

[0142] In some embodiments of the present application, Fig. 9 As shown in (1), for the pixel to be detected (such as point A), the electronic device can determine the pixel threshold (or second pixel threshold) according to the maximum value of the 8 pixel values ​​in the 5×5 matrix and the preset variable value. If the pixel value of the pixel to be detected is greater than the pixel threshold, the pixel to be detected is determined to be a bad pixel (i.e., a single bad pixel). If the pixel value of the pixel to be detected is less than the pixel threshold, the pixel to be detected is determined not to be a bad pixel.

[0143] Exemplarily, taking the preset variable value including the preset variable value 1 and the preset variable value 2 as an example, the pixel threshold may satisfy the following expression (1):

[0144] K = Fmax*MP+offset (1);

[0145] Among them, K represents the pixel threshold, Fmax represents the preset variable value 1 (or the third variable value), MP represents the maximum pixel, that is, the maximum value among the 8 pixel values, and offset represents the preset variable value 2 (or the fourth variable value).

[0146] Exemplarily, the preset variable value 1 is a modifiable variable value, and the preset variable value 1 can be modified according to actual conditions. The embodiment of the present application does not limit the specific numerical value of the preset variable value 1.

[0147] Correspondingly, the preset variable value 2 is a modifiable variable value, and the preset variable value 2 is related to the bit number of the pixel. For example, when the bit number of the pixel is 8 bits, the value range of the preset variable value 2 is 0 to 255. For another example, when the bit number of the pixel is 10 bits, the value range of the preset variable value 2 is 0 to 1023. For another example, when the bit number of the pixel is 18 bits, the value range of the preset variable 2 is 0 to 262143.

[0148] For example, assuming Fmax = 68 / 64, offset = 11413, then It can be seen that the pixel value of the pixel to be detected (point A) is 92670, which is greater than the pixel threshold K (ie, 92669.8). Therefore, the pixel to be detected, ie, point A, is determined to be a bad pixel.

[0149] In some embodiments, the electronic device may also compensate for bad pixels. Exemplarily, the electronic device may compensate for bad pixels by replacing the pixel value of the bad pixel with the pixel threshold K (i.e., 92669.8); or, replacing the pixel value of the bad pixel with the maximum value among m pixel values ​​(i.e., 76477); or, replacing the pixel value of the bad pixel with the second largest value among m pixel values ​​(i.e., 71766); or, the electronic device may also use other pixel values ​​to replace the pixel value of the bad pixel, etc., without limitation.

[0150] For example, the electronic device can determine the pixel median, pixel mean, or pixel standard deviation based on the pixel values ​​of the above 8 pixels, without limitation. Then, the electronic device uses the pixel median, pixel mean, pixel standard deviation, etc. to replace the pixel value of the bad pixel.

[0151] Alternatively, the electronic device may also use a preset pixel value to replace the pixel value of the bad pixel. The preset pixel value may be a pixel value pre-configured by the electronic device and stored in the electronic device. The specific value of the preset pixel value is not limited and is subject to the actual setting. Exemplarily, the preset pixel value may be less than or equal to the maximum value of the m pixel values.

[0152] As an example, Fig. 9 As shown in (2), the pixel value of the pixel to be detected is replaced by the maximum value of the m pixel values ​​(ie, 76477).

[0153] By performing the above processing on each pixel in the RAW domain image, it is possible to detect whether a single bad pixel is included in a plurality of pixels.

[0154] In summary, the electronic device can detect a single bad pixel on a RAW domain image captured by an image sensor through the above method and compensate for the single bad pixel, thereby improving the quality of the captured image and further improving the user experience.

[0155] (2) Double bad points

[0156] For example, Fig.10 As shown in (1), for two adjacent pixels to be detected in the RAW domain image, the pixel value of each of m (e.g., 10) pixels in the first pixel matrix (e.g., 5×7 matrix) that are in the same color channel (e.g., blue channel) as the two adjacent pixels to be detected (e.g., point A and point B) is obtained with the two adjacent pixels to be detected (e.g., point A and point B) as the center, so as to obtain m (i.e., 10) pixel values.

[0157] It should be noted that, for the first pixel matrix and the example description of the corresponding value of m, reference can be made to the above embodiment, which will not be described in detail here. Fig.10 The example of the pixel value of each pixel shown can refer to the above embodiment, and the number of times is not repeated. In addition, the two adjacent pixels to be detected refer to: two continuous pixels in the same color channel.

[0158] Afterwards, the electronic device can determine whether two adjacent pixels to be detected are bad pixels based on the pixel value of each pixel in the m pixels in the first pixel matrix that are centered on the two adjacent pixels to be detected and have the same color channel as the two adjacent pixels to be detected.

[0159] In some embodiments of the present application, Fig.10 As shown in (1), for two adjacent pixels to be detected (such as point A and point B), the electronic device can determine the pixel threshold value according to the maximum value of the 10 pixel values ​​in the 5×7 matrix and the preset variable value. If the pixel values ​​of the two adjacent pixels to be detected are both greater than the pixel threshold value, the two adjacent pixels to be detected are determined to be bad pixels (i.e., double bad pixels). If the pixel values ​​of the two adjacent pixels to be detected are both less than the pixel threshold value, the two adjacent pixels to be detected are determined not to be bad pixels.

[0160] Exemplarily, the pixel threshold may satisfy the above expression (1). For an illustration of the expression (1), reference may be made to the above embodiment, which will not be described in detail here.

[0161] For example, assuming Fmax = 68 / 64, offset = 1759, then It can be seen that the pixel value of the pixel to be detected (point A) is 59961, and the pixel value of the pixel to be detected (point B) is 73951, both of which are greater than the pixel threshold K (i.e., 59960.6). Therefore, the two adjacent pixels to be detected (point A and point B) are determined to be bad pixels (i.e., double bad pixels).

[0162] In some embodiments, the electronic device may also compensate for bad pixels. For an example of compensating for bad pixels, reference may be made to the above embodiments, which will not be described in detail.

[0163] For example, Fig.10 As shown in (2), the pixel value of the bad pixel is replaced by the maximum value of the above 10 pixel values ​​(i.e., 54778).

[0164] By performing the above processing on every two adjacent pixels in the RAW domain image, it is possible to detect whether the multiple pixels include double bad pixels. Among them, every two adjacent pixels refer to any two adjacent pixels among the multiple pixels included in the RAW domain image. For example, point A and point B, point B and the adjacent points around point B, etc., which will not be described in detail.

[0165] In summary, the electronic device can detect double bad pixels on the RAW domain image captured by the image sensor through the above method and compensate for the double bad pixels, thereby improving the quality of the captured picture and further improving the user experience.

[0166] In the related art, the electronic device detects single bad pixels and double bad pixels in sequence. That is, after the electronic device detects a single bad pixel, it continues to detect double bad pixels. However, in the related art, when the electronic device detects double bad pixels, due to the existence of bad pixel clusters, some double bad pixels cannot be detected and eliminated. For example, Fig.11 As shown in (1), for two adjacent pixels to be detected (such as point A and point B) in the RAW domain image, the pixel value of each pixel in m (such as 10) pixels of the same color channel (such as green channel) as the two adjacent pixels to be detected in the first pixel matrix (such as 5×7 matrix) is obtained with the two adjacent pixels to be detected (such as point A and point B) as the center, so as to obtain m (i.e. 10) pixel values.

[0167] For example, the electronic device can use the above expression (1) to determine whether two adjacent pixels to be detected are bad pixels. For example, assuming Fmax = 1, offset = 0, then all pixels greater than mp (maxmium pixel) will be judged as bad pixels, that is, the bad pixel correction strength is the strongest. Fig.11 As shown in point A in (1), when single bad pixel detection is performed, according to expression (1), K = 1*29149+0 = 29149, the pixel value of pixel A is greater than K, so it will be judged as a bad pixel and replaced by 29149; however, when double bad pixel detection is performed, Fig.11As shown in (2), the maximum value of the 10 pixel values ​​around the pixel points A and B is 30548, so K = 1*30548+0 = 30548. It can be seen that the pixel values ​​of the pixels to be detected (points A and B) are both 29149, which is less than the pixel threshold K (i.e. 30548). Therefore, the pixels to be detected (points A and B) will not be determined as bad pixels.

[0168] Therefore, the above three adjacent pixels to be detected are not processed, which will eventually affect the quality of the captured picture and further affect the user experience.

[0169] Based on this, in some other embodiments of the present application, after the electronic device obtains the pixel value of each pixel in the m pixels of the same color channel as the two adjacent pixels to be detected in the first pixel matrix, the electronic device first determines the first pixel threshold value according to the m pixel values. If there is a first pixel value greater than the first pixel threshold value among the m pixel values, the electronic device determines that the first pixel corresponding to the first pixel value is a bad pixel.

[0170] For example, Fig.12 As shown, taking two adjacent pixels to be detected (point A and point B) as the center, the pixel values ​​of 10 pixels of the green channel in the 5×7 matrix are obtained to obtain 10 pixel values. For example, the 10 pixel values ​​may be 16525, 11614, 9034, 7823, 11476, 12802, 16665, 6781, 2609 and 30548. On this basis, the electronic device may determine the first pixel threshold according to the 10 pixel values.

[0171] For example, the electronic device may determine the first pixel threshold value according to the second largest value among the 10 pixel values, the first variable value and the second variable value. Exemplarily, the first pixel threshold value may satisfy the following expression (2):

[0172] K1=Fmax*SMP+TBP_offset (2);

[0173] Among them, K1 represents the first pixel threshold, Fmax represents the first variable value, SMP (sub maximum pixel) represents the sub-maximum value among the 10 pixel values, and TBP_offset represents the second variable value.

[0174] Exemplarily, the first variable value and the second variable value are modifiable variable values, and there is no limitation on the specific numerical values ​​of the first variable value and the second variable value, which are subject to actual settings.

[0175] Exemplarily, the first variable value may be any value in [1, 2]. The second variable value is related to the number of bits of the pixel; or, the second variable value is zero. For an example of the second variable value, reference may be made to the example of the preset variable value 2 in the above embodiment, which will not be repeated here.

[0176] As an example, when the first variable value is 2, the second variable value may be zero.

[0177] Exemplary, combined Fig.12 For example, assuming that Fmax=1, TBP_offset=1000, then K1=1*16665+100. It can be seen that among the above 10 pixel values, there is a first pixel value (such as 30548) that is greater than the first pixel threshold (i.e., 17665), then the electronic device can determine that the first pixel (such as point C) corresponding to the first pixel value is a bad pixel.

[0178] In some embodiments, the electronic device may also compensate for the bad pixel (ie, the first pixel). Exemplarily, the electronic device replaces the first pixel value with the first pixel threshold to compensate for the bad pixel; or Fig.13 As shown, the electronic device replaces the first pixel value with the second largest value among the above 10 pixel values.

[0179] In some embodiments of the present application, after the electronic device compensates the first pixel, the electronic device determines whether the two adjacent pixels to be detected (i.e., point A and point B) are bad pixels. Alternatively, in other embodiments of the present application, if the first pixel (e.g., point C) corresponding to the first pixel value is not a bad pixel, the electronic device continues to determine whether the two adjacent pixels to be detected (point A and point B) are bad pixels.

[0180] As an example, the electronic device may determine whether two adjacent pixels to be detected are bad pixels based on a first pixel threshold. If the pixel values ​​corresponding to the two adjacent pixels to be detected are both greater than the first pixel threshold, the electronic device determines that the two adjacent pixels to be detected are bad pixels. Fig.12 As shown, the pixel values ​​of the two adjacent pixels to be detected (point A and point B) are 30471 and 29149 respectively, both of which are greater than the first pixel threshold (i.e., 17665). Therefore, the two adjacent pixels to be detected are bad pixels. If the pixel values ​​of the two adjacent pixels to be detected are both less than the first pixel threshold, the electronic device determines that the two adjacent pixels to be detected are not bad pixels.

[0181] On this basis, the electronic device can also compensate for the bad pixel (i.e., point A and point B). Exemplarily, the electronic device replaces the pixel values ​​corresponding to two adjacent pixels to be detected with the first pixel threshold to compensate for the pixel to be detected. Or, Fig.13 As shown, the electronic device replaces the pixel values ​​corresponding to two adjacent pixels to be detected with the second largest value (ie, 16665) among the above 10 pixel values ​​to compensate for the pixels to be detected.

[0182] As another example, after the electronic device compensates the first pixel, the electronic device may also use the above expression (1) to determine whether two adjacent pixels to be detected are bad pixels. Fig.14 As shown, after the first pixel value is replaced, the pixel value of point C is 16665. On this basis, the electronic device takes two adjacent pixels to be detected (i.e., point A and point B) as the center, obtains the pixel value of each pixel in the second pixel matrix of n pixels in the same color channel as the two adjacent pixels to be detected, so as to obtain n pixel values; n is a positive integer, and the n pixel values ​​include the replaced first pixel value (i.e., the pixel value of point C after replacement, such as 16665).

[0183] Exemplarily, the electronic device determines the second pixel threshold value based on n pixel values. If the pixel values ​​corresponding to two adjacent pixels to be detected are both greater than the second pixel threshold value, the electronic device determines that the two adjacent pixels to be detected are bad pixels. If the pixel values ​​corresponding to two adjacent pixels to be detected are both less than the second pixel threshold value, the electronic device determines that the two adjacent pixels to be detected are not bad pixels.

[0184] For examples of the second pixel matrix and the value of n, reference may be made to the above embodiments, which will not be described in detail here.

[0185] Taking the second pixel matrix as a 5×7 matrix as an example, illustratively, Fig.14 As shown, for two adjacent pixels to be detected, the electronic device can determine the second pixel threshold value according to the maximum value of the 10 pixel values ​​in the 5×7 matrix, and the preset variable value 1 and the preset variable value 2. If the pixel values ​​of the two adjacent pixels to be detected are greater than the second pixel threshold value, the electronic device determines that the two adjacent pixels to be detected are bad pixels.

[0186] For example, the electronic device may determine the second pixel threshold using the above expression (1). For example, assuming that Fmax=68 / 64, offset=11413, It can be seen that the pixel value of the pixel to be detected (point A) is 30471, which is greater than the second pixel threshold K2 (i.e. 29119.6), and the pixel value of the pixel to be detected (point B) is 29149, which is greater than the second pixel threshold K2 (i.e. 29119.6), so the above two adjacent pixels to be detected are determined to be bad pixels.

[0187] On this basis, the electronic device replaces the pixel values ​​corresponding to two adjacent pixels with the second pixel threshold to compensate for the pixel to be detected; or, the electronic device replaces the pixel values ​​corresponding to two adjacent pixels to be detected with the maximum value of n pixel values ​​to compensate for the pixel to be detected.

[0188] In summary, the electronic device can first detect the bad pixel of the two adjacent pixels to be detected in the first pixel matrix with the two adjacent pixels to be detected as the center; after the electronic device compensates for the bad pixel, the electronic device determines that the two adjacent pixels to be detected are bad pixel. In this way, it can be ensured that the electronic device can detect three adjacent bad pixel pixels (i.e., bad pixel clusters), thereby improving the accuracy of detecting bad pixel clusters.

[0189] For example, Fig.15 As shown, by adopting the solution provided in the embodiment of the present application, the electronic device can detect the bad pixel clusters and compensate for the large dark spots caused by the bad pixel clusters, thereby eliminating the large dark spots caused by the bad pixel clusters, improving the picture quality and further improving the user experience.

[0190] In addition, in an embodiment of the present application, due to the aging of the image sensor, there may be four (or more) adjacent bad pixels in the RAW domain image collected by the image sensor. Exemplary, for the detection method of four (or more) adjacent bad pixels, the pixel values ​​of each pixel around the four adjacent pixels to be detected are obtained with the four adjacent pixels to be detected as the center, and the pixel threshold is determined based on the obtained pixel value. If the pixel values ​​of the four adjacent pixels to be detected are all greater than the pixel threshold, the electronic device determines that the four adjacent pixels to be detected are bad pixels. If the pixel values ​​of the four adjacent pixels to be detected are all less than the pixel threshold, the electronic device determines that the four adjacent pixels to be detected are not bad pixels. The specific implementation process can refer to the above embodiment, which will not be repeated here.

[0191] In addition, in the embodiment of the present application, the detection of whether there are bad pixels and compensation of bad pixels in a scene with bright spots in a dark environment are used as an example. For a scene with dark spots in a bright environment, expression (3) can be used to detect whether there are bad pixels.

[0192] Fmin*MP+offset(3); wherein Fmin represents the first variable value, MP represents the minimum pixel value (i.e., the minimum pixel value around the pixel to be detected), and offset represents the second variable value.

[0193] The principle of detecting whether a dark spot (dead pixel) is a bad pixel and compensating for the bad pixel is similar to the principle of detecting the bright spot. The specific implementation process can refer to the above embodiment and will not be repeated here.

[0194] As an example, if the pixel value of the pixel to be detected is less than the minimum pixel value, the pixel to be detected is determined to be a dark point (ie, a bad pixel in a bright environment).

[0195] It should be noted that the contents recorded in each embodiment of the present application can explain the technical solutions in other embodiments of the embodiments of the present application, and the technical features recorded in each embodiment can also be applied in other embodiments to form new solutions by combining the technical features in other embodiments. The present application only exemplarily lists several embodiments for illustration, and does not mean that the present application is limited to this.

[0196] The present application provides an electronic device, which may include a camera, a memory, and one or more processors; the memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs the functions or steps in the above embodiments. The structure of the electronic device can refer to the above Figure 3 The structure of the electronic device 100 is shown.

[0197] The present application also provides a chip system for use in electronic devices. Fig.16 As shown, the chip system 1100 includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 may be Figure 3 The processor 110 is shown. On this basis, the interface circuit 1102 can be, for example, an interface circuit between the processor 1101 and an external memory; or an interface circuit between the processor 1101 and an internal memory.

[0198] The processor 1101 and the interface circuit 1102 can be interconnected via lines. For example, the interface circuit 1102 can be used to receive signals from other devices (such as the memory of the electronic device 100). For another example, the interface circuit 1102 can be used to send signals to other devices (such as the processor 1101). Exemplarily, the interface circuit 1102 can read the instructions stored in the memory and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the electronic device can execute the various functions or steps executed by the mobile phone in the above embodiment. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiments of the present application.

[0199] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes each function or step executed by the electronic device in the above method embodiment.

[0200] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the electronic device in the above method embodiment.

[0201] It should be noted that the terms "first" and "second" in the specification, claims and drawings of this application are used to distinguish different objects rather than to describe a specific order. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "multiple" means two or more.

[0202] In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0203] It should be understood that in the present application, "at least one (item)" refers to one or more. "Multiple" refers to two or more. "At least two (items)" refers to two or three and more than three. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple. “When” and “if” both mean that corresponding measures will be taken under certain objective circumstances. It does not limit the time, nor does it require any judgment when it is implemented, nor does it mean that there are other limitations.

[0204] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0205] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules according to the system, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0206] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0207] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0208] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0209] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0210] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for detecting a bad pixel, characterized in that: The invention is applied to an electronic device, wherein the electronic device comprises a camera, and the camera is used to collect an original RAW domain image, wherein the RAW domain image comprises a plurality of pixels; wherein the electronic device sequentially uses every two adjacent pixels among the plurality of pixels as pixels to be detected, and detects whether there are bad pixels among the plurality of pixels; and the method comprises: The electronic device takes the pixel to be detected as the center, and obtains the pixel value of each pixel in the m pixels in the first pixel matrix that are in the same color channel as the pixel to be detected, so as to obtain m pixel values; m is a positive integer; The electronic device determines a first pixel threshold according to the m pixel values; If there is a first pixel value among the m pixel values ​​that is greater than the first pixel threshold, the electronic device determines that a first pixel corresponding to the first pixel value is a bad pixel; After the electronic device compensates the first pixel, the electronic device determines whether the pixel to be detected is a bad pixel.

2. The method according to claim 1, characterized in that The electronic device determines that the pixel to be detected is a bad pixel, including: If the pixel values ​​corresponding to the pixels to be detected are all greater than the first pixel threshold, the electronic device determines that the pixels to be detected are bad pixels.

3. The method according to claim 2, characterized in that The method further comprises: The electronic device replaces the pixel value corresponding to the pixel to be detected with the first pixel threshold to compensate for the pixel to be detected; or, The electronic device replaces the pixel value corresponding to the pixel to be detected with the second largest value among the m pixel values ​​to compensate for the pixel to be detected.

4. The method according to claim 1, characterized in that: The electronic device determines whether the pixel to be detected is a bad pixel, including: The electronic device takes the pixel to be detected as the center, obtains the pixel value of each pixel in the n pixels in the second pixel matrix that are in the same color channel as the pixel to be detected, so as to obtain n pixel values; n is a positive integer, and the n pixel values ​​include the first pixel value to be replaced; The electronic device determines a second pixel threshold according to the n pixel values; If the pixel value corresponding to the pixel to be detected is greater than the second pixel threshold, the electronic device determines that the pixel to be detected is a bad pixel.

5. The method according to claim 4, characterized in that The method further comprises: The electronic device replaces the pixel value corresponding to the pixel to be detected with the second pixel threshold to compensate for the pixel to be detected; or, The electronic device replaces the pixel value corresponding to the pixel to be detected with the maximum value of the n pixel values ​​to compensate for the pixel to be detected.

6. The method according to any one of claims 1 to 5, characterized in that The electronic device compensates the first pixel, including: The electronic device replaces the first pixel value with the first pixel threshold; or, The electronic device replaces the first pixel value with the second largest value among the m pixel values.

7. The method according to any one of claims 1 to 6, characterized in that The electronic device determines a first pixel threshold according to the m pixel values, including: The electronic device determines the first pixel threshold according to the first variable value, the second variable value and the second largest value among the m pixel values; The first variable value is any value in [1, 2]; the second variable value is related to the number of bits of the pixel, or the second variable value is zero.

8. The method according to claim 7, characterized in that The first pixel threshold satisfies the following expression: K1=Fmax*SMP+TBP_offset; Among them, K1 represents the first pixel threshold, Fmax represents the first variable value, SMP represents the second largest value, and TBP_offset represents the second variable value.

9. The method according to claim 4 or 5, characterized in that: The electronic device determines a second pixel threshold according to the n pixel values, including: The electronic device determines the second pixel threshold according to the third variable value, the fourth variable value and the maximum value of the n pixel values; The third variable value is any value in [1, 2]; the fourth variable value is related to the number of bits of a pixel; or the fourth variable value is zero.

10. The method according to claim 9, characterized in that The second pixel threshold satisfies the following expression: K2=Fmax*MP+offset; Among them, K2 represents the second pixel threshold, Fmax represents the third variable value, MP represents the maximum value, and offset represents the fourth variable value.

11. The method according to any one of claims 1 to 10, characterized in that The plurality of pixels are divided into P pixel units, each of the P pixel units includes one red pixel, two green pixels and one blue pixel; P≥1.

12. An electronic device, characterized in that: include: memory and one or more processors, camera; The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method as claimed in any one of claims 1 to 11.

13. A chip system, characterized in that: include: at least one processor and an interface; The interface is used to receive instructions and transmit them to the at least one processor; The at least one processor executes the instructions so that the electronic device performs the method as claimed in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 11.

15. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

Images