Image processing method and electronic device

Abstract

This application provides an image processing method and an electronic device, relating to the field of terminal technology. The method includes: acquiring crosstalk information corresponding to each of multiple photosensitive modules, the crosstalk information including crosstalk parameters corresponding to each of multiple photosensitive objects in the photosensitive modules, the crosstalk parameters indicating crosstalk signals from surrounding photosensitive objects adjacent to the photosensitive object, and the crosstalk parameters being determined based on the detected photosensitive objects. For any one of the multiple photosensitive modules, compensation is performed on the detection signals corresponding to each of the multiple photosensitive objects in the photosensitive module according to the crosstalk information corresponding to the photosensitive module, to obtain corrected signals corresponding to each of the multiple photosensitive objects in the photosensitive module. A target image is generated based on the corrected signals corresponding to each of the multiple photosensitive objects. In this way, by setting the detection method for photosensitive objects for the image sensor, the problem of grid noise in the final output image can be avoided.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method and an electronic device. Background Technology

[0002] Today, image sensors play a crucial role in end devices. The core function of an image sensor is to convert light signals into electrical signals.

[0003] In traditional implementations, when light is accurately incident on the centrally symmetrical position of the image sensor, the sensor generates a uniform electrical signal. These uniform electrical signals, after processing, can be converted into a digital image.

[0004] However, in certain scenarios, the electrical signals received by the sensor may be non-uniform. If image processing is performed based on non-uniform electrical signals, the resulting image will contain grid noise, thus affecting image quality. Summary of the Invention

[0005] This application provides an image processing method and an electronic device, applicable to the field of terminal technology. The technical solution of this application can be applied to situations where non-ideal factors exist. By setting a photosensitive object to be detected on the image sensor, and then extracting and correcting crosstalk information based on the detected photosensitive object, the problem of grid noise in the final output image can be avoided.

[0006] In a first aspect, embodiments of this application propose an image processing method applied to a terminal device. The image sensor in the terminal device includes multiple photosensitive modules, each including multiple photosensitive objects; at least some of the multiple photosensitive modules are first-type photosensitive modules, each including a detection photosensitive object. The detection photosensitive object has a photosensitive state opposite to that of adjacent surrounding photosensitive objects, either being photosensitive or not photosensitive. The method includes:

[0007] The crosstalk information corresponding to each of the multiple photosensitive modules is obtained. The crosstalk information includes the crosstalk parameters corresponding to each of the multiple photosensitive objects in the photosensitive module. The crosstalk parameters are used to indicate the crosstalk signals of the surrounding photosensitive objects adjacent to the photosensitive object to the photosensitive object. The crosstalk parameters are determined according to the detected photosensitive object.

[0008] For any one of the multiple photosensitive modules, the detection signals corresponding to each of the multiple photosensitive objects in the photosensitive module are compensated according to the crosstalk information corresponding to the photosensitive module, so as to obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module.

[0009] The target image is generated based on the correction signals corresponding to each of the multiple photosensitive objects.

[0010] In this implementation, the image sensor is divided into modules to define the photosensitive objects for detection, thus differentiating them from other normal photosensitive objects and facilitating the extraction of crosstalk information. By defining these photosensitive objects, crosstalk information is obtained for each photosensitive object in multiple photosensitive modules. This crosstalk information can be used to identify the presence of non-ideal factors. The extracted crosstalk information can then be used to compensate for the detection signals corresponding to each photosensitive object in the photosensitive modules, resulting in a corrected signal. Correcting the image using this corrected signal avoids the problem of grid noise in the final output image.

[0011] In one possible implementation, crosstalk information corresponding to each of the multiple photosensitive modules is obtained, including:

[0012] For any type I photosensitive module, obtain the crosstalk information corresponding to the type I photosensitive module;

[0013] Based on the crosstalk information corresponding to each of the first-type photosensitive modules, the crosstalk information corresponding to the remaining photosensitive modules other than the first-type photosensitive modules is determined.

[0014] In this implementation, by classifying the photosensitive modules containing the photosensitive object to be detected as the first type of photosensitive modules, the crosstalk information corresponding to each of the remaining photosensitive modules can be determined using the first type of photosensitive modules.

[0015] In one possible implementation, for any one of the multiple photosensitive modules, the photosensitive module includes N photosensitive units, the photosensitive unit includes M photosensitive sub-units, and the M photosensitive sub-units correspond to their respective photosensitive colors;

[0016] The photosensitive subunit includes at least one photosensitive pixel, and the photosensitive unit includes a total of T photosensitive pixels, where N, M and T are all integers greater than or equal to 1.

[0017] The photosensitive object is either a photosensitive pixel or a photosensitive subunit.

[0018] In this implementation, by dividing the photosensitive object into two cases—photosensitive pixels and photosensitive sub-units—the detection of the photosensitive object can be set for different types of image sensors. By setting the detection of the photosensitive object for different image sensors, the application scope of the technical method of this application can be expanded.

[0019] In one possible implementation, when the photosensitive object is a photosensitive pixel, the number of photosensitive objects detected in the first type of photosensitive module is T; or...

[0020] When the photosensitive object is a photosensitive subunit, the number of photosensitive objects detected in the first type of photosensitive module is M.

[0021] In this implementation, by classifying the photosensitive objects, the detection photosensitive objects can be set according to the two types of photosensitive objects, realizing the setting of detection photosensitive objects for different types of image sensors and expanding the application of this solution.

[0022] In one possible implementation, among the N photosensitive units of the first type of photosensitive module, at least some of the photosensitive units are special photosensitive units, and multiple photosensitive objects are distributed in each special photosensitive unit;

[0023] Furthermore, the location of the photosensitive object contained in each special photosensitive unit is different within the photosensitive unit.

[0024] In this implementation, by constraining the positions of the photosensitive objects contained in the special photosensitive unit to be different within the photosensitive unit, the non-repeatability of the photosensitive object setting can be achieved, reducing manufacturing costs.

[0025] In one possible implementation, when the detected photosensitive object is not photosensitive, for any surrounding photosensitive object adjacent to the detected photosensitive object, only one of the multiple photosensitive objects adjacent to the surrounding photosensitive object is the detected photosensitive object.

[0026] In this implementation, by constraining that only one of the multiple photosensitive objects adjacent to the photosensitive object is a detection photosensitive object, it can correspond to the case where no photosensitive objects are set around the second photosensitive object. Thus, by setting a variable of a detection photosensitive object, the consistency of other photosensitive conditions can be achieved except for the difference of the variable, so that the crosstalk signal of the first mapping object to the second photosensitive object can be calculated.

[0027] In one possible implementation, when the photosensitive object detected in the first type of photosensitive module is not photosensitive, the crosstalk information corresponding to the first type of photosensitive module is obtained, including:

[0028] For any photosensitive object to be detected, based on the detection signals of each of the surrounding photosensitive objects adjacent to the photosensitive object, the crosstalk signals of the photosensitive module located at the first position in each photosensitive unit to each of the adjacent surrounding photosensitive objects are determined. The first position is the position of the photosensitive object in the photosensitive unit.

[0029] Based on the crosstalk signals of the photosensitive modules at multiple locations within each photosensitive unit to the adjacent surrounding photosensitive objects, the crosstalk parameters corresponding to each photosensitive object in the first type of photosensitive module are determined, so as to obtain the crosstalk information corresponding to the first type of photosensitive module.

[0030] In this implementation, by determining that a photosensitive object within a photosensitive unit meets a first position, the crosstalk signal of that photosensitive object to each of its adjacent surrounding photosensitive objects can be determined. By determining the crosstalk signal of a photosensitive object at the first position in any photosensitive unit to each of its adjacent surrounding photosensitive objects, the crosstalk signal of a photosensitive object at the first position in other photosensitive units to each of its adjacent surrounding photosensitive objects can be determined, thereby determining the crosstalk parameters corresponding to each photosensitive object in the first type of photosensitive module.

[0031] In one possible implementation, determining the crosstalk signal of the photosensitive object located at a first position within each photosensitive unit to each of the adjacent surrounding photosensitive objects, based on the detection signals of each of the surrounding photosensitive objects adjacent to the detected photosensitive object, includes:

[0032] For any first photosensitive object adjacent to the photosensitive object to be detected, a first detection signal of the first photosensitive object is acquired, and a second detection signal of the second photosensitive object is acquired. The position of the second photosensitive object in the photosensitive unit is the same as the position of the first photosensitive object in the photosensitive unit, and there is no photosensitive object to be detected among the multiple photosensitive objects adjacent to the second photosensitive object.

[0033] Based on the difference between the first detection signal and the second detection signal, the crosstalk signal of the first mapped photosensitive object to the second photosensitive object is determined. The positional relationship between the first mapped photosensitive object and the second photosensitive object is the same as the positional relationship between the detected photosensitive object and the first photosensitive object.

[0034] Based on the crosstalk signal of the first mapped photosensitive object to the second photosensitive object, the crosstalk signal of the photosensitive object located at the first position in each photosensitive unit to each of the adjacent surrounding photosensitive objects is determined, wherein the position of the first mapped photosensitive object in the photosensitive unit is equal to the first position.

[0035] In this implementation, by calculating the difference between the first detection signal and the second detection signal when the photosensitive object is not photosensitive, the crosstalk signal of the first mapped photosensitive object to the second photosensitive object can be determined. This allows for the determination of the crosstalk signal of the photosensitive object at the first position within each photosensitive unit to each of its adjacent surrounding photosensitive objects. By constraining the positions of the second photosensitive object and the first mapped photosensitive object to correspond with the positions of the first photosensitive object and the detected photosensitive object, this calculation method can only be used to obtain the crosstalk signal when this corresponding positional relationship is met, ensuring the rigor of this scheme.

[0036] In one possible implementation, when detecting light-sensitive objects in the first type of photosensitive module, crosstalk information corresponding to the first type of photosensitive module is obtained, including:

[0037] For any photosensitive object to be detected, a third detection signal of the photosensitive object and a fourth detection signal of the second mapped photosensitive object are acquired. The photosensitive module to which the second mapped photosensitive object belongs is adjacent to the photosensitive module to which the photosensitive object belongs, and the position of the second mapped photosensitive object in its photosensitive module is the same as the position of the photosensitive object in its photosensitive module.

[0038] Based on the difference between the third and fourth detection signals, the crosstalk parameters corresponding to the detected photosensitive object are determined.

[0039] Based on the crosstalk parameters corresponding to the detected photosensitive object, the crosstalk parameters corresponding to each photosensitive module located at the second position in each photosensitive unit are determined to obtain the crosstalk information corresponding to the first type of photosensitive module. The second position is the position of the detected photosensitive object in its respective photosensitive unit.

[0040] In this implementation, by calculating the difference between the third and fourth detection signals based on the photosensitive object's photosensitive condition, the crosstalk parameters corresponding to the photosensitive object can be determined, thereby determining the crosstalk parameters corresponding to the photosensitive modules located at the second position within each photosensitive unit. By constraining the positional relationship between the second mapped photosensitive object and the photosensitive modules to which it belongs, this calculation method can be used to obtain the crosstalk signal only when the positional relationship of these objects is satisfied, ensuring the accuracy of the obtained crosstalk signal.

[0041] In one possible implementation, based on the crosstalk information corresponding to each of the first-type photosensitive modules, the crosstalk information corresponding to the remaining photosensitive modules (excluding the first-type photosensitive modules) among the multiple photosensitive modules is determined, including:

[0042] For any first-type photosensitive module, determine the crosstalk information of multiple photosensitive modules located around the first-type photosensitive module, so as to obtain the crosstalk information corresponding to each of the remaining photosensitive modules.

[0043] In this implementation, by calculating the crosstalk information of the first type of photosensitive module, the crosstalk information of multiple photosensitive modules located around the first type of photosensitive module can be determined, thereby obtaining the crosstalk information corresponding to each of the remaining photosensitive modules. This method can determine the crosstalk information of all photosensitive modules in the image sensor, which can then be used to determine the correction signal of the image sensor.

[0044] In one possible implementation, the crosstalk information corresponding to each of the plurality of photosensitive modules is stored in the first storage space;

[0045] Obtain crosstalk information for each of the multiple photosensitive modules, including:

[0046] Obtain crosstalk information corresponding to each of the multiple photosensitive modules from the first storage space.

[0047] In this implementation, by storing the crosstalk information corresponding to each of the multiple photosensitive modules in the first storage space, it is convenient to extract the information when calculating the correction signal of the photosensitive object.

[0048] In one possible implementation, based on the crosstalk information corresponding to the photosensitive module, the detection signals corresponding to each of the multiple photosensitive objects in the photosensitive module are compensated to obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module, including:

[0049] For any photosensitive object in the photosensitive module, based on the crosstalk parameter corresponding to the photosensitive object, the crosstalk signal contained in the detection signal corresponding to the photosensitive object is removed to obtain the correction signal of the photosensitive object.

[0050] In this implementation, a correction signal for the photosensitive object can be obtained by removing crosstalk from the detection signal corresponding to the photosensitive object. Since crosstalk exists due to non-ideal factors, removing crosstalk from the detection signal allows the obtained correction signal to correct the image, thus avoiding the problem of grid noise in the final output image.

[0051] In one possible implementation, based on the crosstalk information corresponding to the photosensitive module, the detection signals corresponding to each of the multiple photosensitive objects in the photosensitive module are compensated to obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module, including:

[0052] The crosstalk information corresponding to the photosensitive module and the detection signal corresponding to each photosensitive object in the photosensitive module are input to the first model so that the first model outputs the correction signal corresponding to each of the multiple photosensitive objects in the photosensitive module.

[0053] The first model is used to compensate for the detection signal of the photosensitive object based on the crosstalk signal of the photosensitive object.

[0054] In this implementation, by training the first model, correction signals corresponding to multiple photosensitive objects can be obtained, which improves the efficiency of compensating for the detection signals of the photosensitive objects. At the same time, the first model also ensures the accuracy of the correction signals.

[0055] In one possible implementation, the method also includes:

[0056] For any given first-class photosensitive module, the detection signal of the non-photosensitive photosensitive object in the first-class photosensitive module is corrected to obtain the corrected detection signal corresponding to each non-photosensitive photosensitive object.

[0057] In this implementation, by correcting the detection signal of the non-photosensitive object in the first type of photosensitive module, it can be ensured that the image is corrected using the corrected detection signal of the photosensitive object, thereby reducing the error caused by the non-photosensitive object.

[0058] In one possible implementation, the detection signals of non-photosensitive objects in the first type of photosensitive module are corrected to obtain corrected detection signals corresponding to each non-photosensitive object, including:

[0059] For any non-photosensitive object, the detection signals of the surrounding photosensitive objects adjacent to the non-photosensitive object are fused to obtain the corrected detection signal corresponding to the non-photosensitive object.

[0060] In this implementation, the corrected detection signal corresponding to the non-photosensitive object can be determined by fusing the detection signals of the surrounding photosensitive objects adjacent to the non-photosensitive object. Since the non-photosensitive detection object cannot receive a signal, the detection signal of the detection object can be determined more accurately by fusing the detection signals of the surrounding photosensitive objects adjacent to it.

[0061] Secondly, embodiments of this application provide an image processing method apparatus. This image processing apparatus can be an electronic device, or a chip or chip system within an electronic device. The image processing apparatus may include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit may be a display screen. The display unit is used to perform display steps to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is an electronic device, the processing unit may be a processor. The image processing apparatus may further include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is a chip or chip system within an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip within the electronic device (e.g., a read-only memory, random access memory, etc.).

[0062] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory for storing code instructions, and the processor for running the code instructions to perform the methods described in the first aspect or any possible implementation of the first aspect.

[0063] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0064] Fifthly, embodiments of this application provide a computer program product including a computer program, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0065] Sixthly, this application provides a chip or chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to perform the methods described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, pins, or circuits, etc.

[0066] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0067] It should be understood that the second to sixth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0068] Figure 1 A schematic diagram illustrating the digital imaging process provided in an embodiment of this application;

[0069] Figure 2 A schematic diagram of the structure of the image sensor provided in the embodiments of this application;

[0070] Figure 3 A schematic diagram of the pixel structure of a back-illuminated image sensor provided in an embodiment of this application;

[0071] Figure 4 A schematic diagram of a pixel unit provided in an embodiment of this application;

[0072] Figure 5 A schematic diagram of the spot offset provided in an embodiment of this application;

[0073] Figure 6 A schematic diagram of an image containing lattice noise provided in an embodiment of this application;

[0074] Figure 7 This is a schematic diagram of the hardware structure of a terminal device provided in an embodiment of this application;

[0075] Figure 8 This is a schematic diagram of the software structure of a terminal device provided in an embodiment of this application;

[0076] Figure 9 A schematic diagram of the image sensor partitioning module provided in an embodiment of this application;

[0077] Figure 10 Schematic diagrams of two types of first-class photosensitive modules provided in embodiments of this application;

[0078] Figure 11 A schematic diagram of the photosensitive object type provided in the embodiments of this application;

[0079] Figure 12 This application provides an illustration of setting up a photosensitive object for detection. Figure 1 ;

[0080] Figure 13 This application provides an illustration of setting up a photosensitive object for detection. Figure 2 ;

[0081] Figure 14 This application provides an illustration of setting up a photosensitive object for detection. Figure 3 ;

[0082] Figure 15 This application provides an illustration of setting up a photosensitive object for detection. Figure 4 ;

[0083] Figure 16 This application provides an illustration of setting up a photosensitive object for detection. Figure 5 ;

[0084] Figure 17 This application provides an illustration of setting up a photosensitive object for detection. Figure 6 ;

[0085] Figure 18 This application provides an illustration of setting up a photosensitive object for detection. Figure 7 ;

[0086] Figure 19A A flowchart illustrating the process of obtaining crosstalk information provided in this application embodiment. Figure 1 ;

[0087] Figure 19B A flowchart illustrating the process of obtaining crosstalk information provided in this application embodiment. Figure 2 ;

[0088] Figure 20 A schematic diagram of crosstalk information calculation provided in the embodiments of this application. Figure 1 ;

[0089] Figure 21 A schematic diagram of determining crosstalk information of the remaining photosensitive modules provided in this application embodiment. Figure 1 ;

[0090] Figure 22 A schematic diagram of determining crosstalk information of the remaining photosensitive modules provided in this application embodiment. Figure 2 ;

[0091] Figure 23 A schematic diagram of crosstalk information calculation provided in the embodiments of this application. Figure 2 ;

[0092] Figure 24 A schematic diagram of the static correction process provided in the embodiments of this application;

[0093] Figure 25 A schematic diagram illustrating the principle of the first model provided in the embodiments of this application;

[0094] Figure 26 A schematic diagram of the dynamic correction process provided in the embodiments of this application;

[0095] Figure 27 A schematic diagram of the polarization layer process provided in an embodiment of this application;

[0096] Figure 28 A schematic diagram of an image sensor using a color separator structure provided in an embodiment of this application;

[0097] Figure 29 This is a schematic diagram illustrating the replacement of a normal filter according to an embodiment of this application;

[0098] Figure 30 A schematic flowchart of the image processing method provided in the embodiments of this application;

[0099] Figure 31 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0100] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0101] 1. Photosensitive pixels

[0102] A photosensitive pixel is a photosensitive unit in an image sensor; it's a physical hardware concept that converts light signals into electrical signals. A pixel is the basic unit of an image, the smallest element that makes up a digital image.

[0103] Photosensitive pixels are the foundation for generating pixels. The color and brightness information contained in the pixels of an image are derived from the capture and conversion of light signals by the photosensitive pixels.

[0104] 2. Optical image stabilization system

[0105] Optical image stabilization (OIS) is a technology used to reduce image blur, commonly found in cameras and video recording equipment. Its main function is to stabilize the image by compensating for camera or lens movement, thereby improving the sharpness and quality of the shot.

[0106] 3. OTP

[0107] One-Time Programmable Memory (OTP) in image sensors is a type of memory used to store permanent data. A key characteristic of OTP is that data written to it cannot be modified or deleted, making it ideal for storing data that needs to be preserved for a long time and does not require alteration.

[0108] 4. EEPROM

[0109] In image sensor modules, electrically erasable programmable read-only memory (EEPROM) is a commonly used memory type for storing various configuration information and calibration data. Unlike OTP, EEPROM can be erased and rewritten multiple times, providing flexibility and ease of updates.

[0110] 5. Crosstalk

[0111] Crosstalk refers to the phenomenon in electronic devices or circuit systems where a signal in one signal channel is interfered with by signals in other adjacent or related signal channels, thereby altering its originally normal signal state.

[0112] In the context of image sensors, crosstalk refers to the influence of surrounding pixels on the output signal of a pixel. This influence is not based on the light received by the pixel itself, but rather on the interference caused by surrounding pixels in the process of processing the light they receive and generating electrical signals.

[0113] 6. Other terms

[0114] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0115] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0116] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0117] 7. Electronic equipment

[0118] The electronic devices in this application embodiment may include handheld devices, vehicle-mounted devices, etc., with image acquisition and image processing functions. For example, some electronic devices include: mobile phones, tablets, PDAs, laptops, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, in-vehicle devices, wearable devices, terminal devices in 5G networks, or future evolution of public land mobile communication networks. Terminal devices in a network (PLMN), etc., are not limited to this in the embodiments of this application.

[0119] By way of example and not limitation, in this embodiment, the electronic device can also be a wearable device. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not merely hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are feature-rich, large in size, and can achieve complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific type of application function and require the use of other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0120] Furthermore, in this embodiment of the application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0121] The electronic devices in the embodiments of this application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0122] In this embodiment, the electronic device or various network devices include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0123] To better understand the technical solution of this application, the relevant technologies involved in this application will be further described in detail below.

[0124] Over the past few decades, image sensor technology has undergone rapid development, undergoing a significant transformation from traditional film imaging to digital imaging. As can be understood, digital imaging differs from film imaging; it does not rely on chemical processing but instead uses electronic devices to convert light signals into electrical signals. These electronic devices refer to the image sensor. The core function of an image sensor is to convert light signals into electrical signals, which is a fundamental step in digital imaging technology.

[0125] In digital imaging, image sensors convert incident light into electrical charges, and then convert these charges into electrical signals. Since the microlens array on an image sensor is typically designed to optimally focus light at its center, photoelectric conversion efficiency and signal uniformity are improved. Therefore, when light is accurately incident on the centrally symmetrical position of the image sensor, the sensor generates a uniform electrical signal. These uniform electrical signals, after further processing and optimization by the processing unit, can be converted into a high-quality digital image, presenting a clear and color-accurate visual effect.

[0126] In one implementation, assuming the terminal device uses a complementary metal-oxide-semiconductor (CMOS) sensor, the implementation of the digital imaging process can be understood by combining it with a CMOS image sensor. Figure 1 This is a schematic diagram of the digital imaging process provided in an embodiment of this application.

[0127] like Figure 1 As shown, assuming the terminal device uses a CMOS camera 103 to capture an object 101 and finally obtains an image 104, the CMOS camera 103 may include a lens 105, a CMOS image sensor 106, an image signal processor (ISP) chip 107, and an input / output (I / O) interface chip 108.

[0128] Specifically, when the incident light 102 of object 101 enters the lens 105, it passes through the image sensor 106. The image sensor 106 converts the received light signal into an electrical signal. After the electrical signal is processed by the ISP processing chip 107, it is converted into a digital signal. After being processed by the I / O interface chip 108, the final visible image 104 is obtained.

[0129] It should be noted that, in order to understand the digital imaging process, it is assumed that the camera used by the terminal device is a CMOS camera, but it can also be other cameras, which can be determined according to actual needs. This application does not limit this.

[0130] Based on the above, we can understand that the function of an image sensor is to convert light signals into electrical signals. Therefore, to explain how an image sensor converts light signals into electrical signals, the following section will... Figure 2 , Figure 3 , Figure 4 This section introduces information related to image sensors. Figure 2 This is a schematic diagram of the structure of the image sensor provided in an embodiment of this application. Figure 3This is a schematic diagram of the pixel structure of the back-illuminated image sensor provided in an embodiment of this application. Figure 4 This is a schematic diagram of a pixel unit provided in an embodiment of this application.

[0131] like Figure 2 As shown, assuming an image sensor 201 exists, the image sensor 201 may include a housing 202, a protective glass 203, a welding wire 204, and a sensor chip 205. The sensor chip 205 is composed of a large number of pixels arranged in a specific array. The function of each pixel is to convert the sensed light into an electrical signal, which is then converted into a digital signal by a readout circuit, thereby completing the process of digitizing the real-world scene.

[0132] like Figure 3 As shown, the basic structure of a pixel in a back-illuminated image sensor is illustrated. From top to bottom, it can include: an on-chip lens, a dielectric layer, a color filter, a photosensitive layer, and a metal circuit. The imaging diode is located within the photosensitive layer.

[0133] The on-chip lens can be understood as an array of microlenses covering the photosensitive element, used to focus light onto the opening of the pixel's photosensitive area. The dielectric layer separates the color filters and the on-chip lens, preventing direct contact and thus avoiding interference or damage. The color filters include red, green, and blue filters, which only allow light of their corresponding wavelengths to pass through; the presence of the filter structure ensures that each pixel can only sense one color. The photosensitive layer contains an imaging diode, which can also be understood as a photoelectric signal converter. The electrical signal converted by the diode is read out through a metal circuit. This metal circuit is used to read out the electrical signal converted by the imaging diode in the photosensitive area.

[0134] Therefore, it can be understood that when incident light enters the image sensor, it first reaches the on-chip lens, which collects the effective light. The light gathered by the on-chip lens passes through the dielectric layer, which acts as an insulator, allowing the light to reach the color filters. Then, each color filter can only transmit light of its corresponding color wavelength, thus enabling each pixel to sense only one color of light, achieving the filtering and separation of light colors. The filtered light reaches the imaging diode, which converts the received light signal into an electrical signal. Finally, the electrical signal converted by the imaging diode is read out through the metal circuit, thus completing the conversion process from light signal to electrical signal, and the converted electrical signal is transmitted to the subsequent processing circuit for processing.

[0135] It should be noted that the above describes the pixel structure of a back-illuminated image sensor. While back-illuminated image sensors are widely used in terminal devices, other pixel structures also exist, such as front-illuminated structures. In actual production, the pixel structure can be determined according to actual needs, and this application does not limit this aspect.

[0136] Based on the above, we can understand that color filters include red, green, and blue. This is because the human eye has cone cells that are sensitive to different wavelengths of light. These cone cells can be roughly divided into three categories: those sensitive to red light, green light, and blue light. Among them, the human eye is relatively more sensitive to green light. Based on the principle of cone cells in the human eye, the following will combine... Figure 4 This section explains how to configure pixels in an image sensor to make different pixels sensitive to different wavelengths of light.

[0137] like Figure 4 As shown, a pixel unit contains a red pixel 401, a green pixel 402, a green pixel 403, and a blue pixel 404. This layout can better capture the color information perceived by the human eye and improve the color reproduction of the image.

[0138] It can also be understood that a unit containing one red pixel, two green pixels, and one blue pixel can be considered a Bayer array unit. On the entire image sensor chip, these 2×2 Bayer array units are repeatedly arranged to form a large pixel matrix, thereby enabling the capture of color information for the entire shooting scene.

[0139] The above describes how, when light is accurately incident on the centrally symmetrical position of the image sensor, the sensor generates a uniform electrical signal. However, in certain scenarios, such as shooting backlit scenes, if camera shake occurs when the electronic shutter of the device is pressed, the OIS system of the device, in its image stabilization compensation by moving the lens, will cause the light incident on the image sensor to deviate from its original path and enter a position other than the centrally symmetrical position.

[0140] One intuitive manifestation of light deviating from its central symmetry is the shift in the light spot. The light spot is the bright area formed by light on an image sensor; normally, it should be in a relatively fixed and central position. However, when light deviates from the central symmetry of the image sensor, the position and size of the light spots falling on different colors will vary, resulting in uneven light distribution.

[0141] The following is combined with Figure 5 An example is provided to illustrate the situation where the light spot shifts. Figure 5 This is a schematic diagram of the spot offset provided in an embodiment of this application.

[0142] like Figure 5As shown, assuming the image sensor consists of 2×2 Bayer array units, each Bayer array unit can include a red pixel 501, a green pixel 502, a green pixel 503, and a blue pixel 504. Each color pixel is composed of 2×2 sub-pixels. This type of image sensor is used in high-resolution, high-precision image capture devices.

[0143] like Figure 5 As shown in (a), when light is incident on the centrally symmetrical position of the image sensor, light spots a1, a2, a3, and a4 are formed. Specifically, a1 is located at the center of the red pixel, a2 at the center of the green pixel, a3 at the center of the green pixel, and a4 at the center of the blue pixel. The area occupied by the light spot within a pixel is considered to represent the amount of light received by that pixel.

[0144] It's understandable that, taking red pixel 501 as an example, since spot a1 is located at the center of red pixel 501, spot a1 occupies the same area in the four sub-pixels, resulting in relatively uniform electrical signals output by the four sub-pixels. The same logic applies to green pixel 502, green pixel 503, and blue pixel 504, where the electrical signals output by the four sub-pixels corresponding to each color pixel are relatively uniform.

[0145] like Figure 5 As shown in (b), when the light rays are offset from the center symmetrical position of the image sensor, light spots b1, b2, b3, and b4 are formed. Since the incident light rays are offset from the center position of the image sensor, it can be observed that b1, b2, b3, and b4 are not located at the center position of each pixel, and the size of the light spots is also different. It can be understood that because the light spots are offset from the center position of each pixel, the amount of light received by the four sub-pixels corresponding to each color pixel is different, resulting in non-uniform electrical signals output by the four sub-pixels corresponding to each color pixel.

[0146] If image processing is performed based on these non-uniform electrical signals, the resulting image may contain grid noise. This noise manifests as irregular grids or stripes in the image, severely affecting image clarity and visual quality. Grid noise not only degrades the user experience but may also negatively impact subsequent image analysis and processing.

[0147] The following is combined with Figure 6 An example is provided to illustrate lattice noise. Figure 6 This is a schematic diagram of an image containing lattice noise, provided in an embodiment of this application.

[0148] like Figure 6As shown, assume there is a terminal device 601 to capture a backlit scene. In this backlit scene, the light source 602 is located behind the subject 603. When using the terminal device 601 to capture this scene, if camera shake occurs when the electronic shutter 604 is pressed, the terminal device 601 will activate the OIS system to move the lens to compensate for the shake. This lens movement will cause the light incident on the image sensor to deviate from the central symmetrical position, resulting in a shift in the light spot and uneven electrical signals. Consequently, the generated image 605 will exhibit grid noise.

[0149] It should be noted that the embodiments of this application exemplify images with grid noise caused by lens position shift due to camera shake during backlighting and the activation of OIS on the terminal device. In reality, besides the above scenarios, other non-ideal factors can also lead to poor image quality. These could include unavoidable factors during the image sensor manufacturing process, unavoidable factors introduced during camera module assembly, or errors caused by asymmetric pixel output in phase detection autofocus technology. This application does not limit the causes of poor image quality.

[0150] To address the problems described above, this application proposes a technical concept: in an image sensor, pixels that are normally photosensitive undergo special processing to make their photosensitive properties different from those of the surrounding pixels. This allows the sensor to collect and output information about non-ideal factors from the surrounding pixels during the photosensitive process. By extracting this information, image pixel correction is achieved, thereby avoiding the problem of grid noise.

[0151] The image processing method of this application embodiment can be executed by an electronic device equipped with a camera, or by a chip, chip system, or processor that supports the implementation of the image processing method by the electronic device, or by a logic module or software that can implement all or part of the functions of the electronic device. This application does not impose specific limitations in this regard. The image processing method of this application embodiment will be described in detail below using an electronic device as the execution subject as an example.

[0152] Electronic devices can be, for example, terminal devices. The following section will first combine... Figure 7 and Figure 8 A brief introduction to the terminal equipment.

[0153] For example, Figure 7 This is a schematic diagram of the hardware structure of a terminal device provided in an embodiment of this application.

[0154] Figure 7This is a schematic diagram of the terminal device provided in an embodiment of this application. The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a gyroscope sensor 180B, a proximity sensor 180F, a proximity light sensor 180G, an image sensor 180H, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0155] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device. In other embodiments of this application, the terminal device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0156] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. In this embodiment, processor 110 is used to receive raw image data and perform a series of processing operations. For example, denoising, depigmentation, color correction, and sharpening are performed on the acquired image.

[0157] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system. In this embodiment, an OTP memory and an EEPROM memory are used to store correction data for crosstalk information of acquired non-ideal factors.

[0158] The terminal device implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0159] Terminal devices can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0160] The Information Service Provider (ISP) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193. In this application embodiment, the ISP is used to process the image, processing the electrical signals output by the image sensor to obtain the final image.

[0161] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the terminal device may include one or N cameras 193, where N is a positive integer greater than 1. In this embodiment, the camera is equipped with an OIS system, which can compensate for shake by moving the lens. However, this also causes the light to deviate from the central symmetrical position of the image sensor, resulting in poor image quality. Therefore, the solution of this application can be used to solve the image quality problem.

[0162] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when a terminal device selects a frequency, a DSP can perform a Fourier transform on the frequency energy.

[0163] An NPU (Neural Processing Unit) is a neural network (NN) computing processor that, by borrowing from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding. In this embodiment, the AI ​​algorithm for correcting crosstalk is processed on the NPU, enabling efficient operation of the AI ​​algorithm and real-time image optimization.

[0164] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the terminal device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shaking of the terminal device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shaking of the terminal device 100 through reverse movement, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios. In this embodiment, shaking is detected by the gyroscope sensor. Shaking includes not only the clicking shaking that occurs when the shutter is pressed, but also involuntary biological shaking when the user holds the terminal device, which is not limited here.

[0165] The software system of a terminal device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc. This application uses the layered architecture Android system as an example to illustrate the software structure of the terminal device.

[0166] For example, Figure 8 This is a schematic diagram of the software structure of a terminal device provided in an embodiment of this application.

[0167] like Figure 8 As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the system may include an application layer, an application framework layer, an Android runtime and system libraries, a hardware abstraction layer (HAL), and a kernel layer. It should be noted that this application uses the Android system as an example; however, the solution can also be implemented in other operating systems (such as HarmonyOS, iOS, etc.) as long as the functions implemented by each module are similar to those in the embodiments of this application.

[0168] The application layer can include a series of application packages.

[0169] like Figure 8 As shown, the application package may include applications such as camera, gallery, phone, and map. Of course, the application layer may also include other application packages, such as third-party applications like payment apps, shopping apps, banking apps, and social apps; this application does not impose any limitations. In this embodiment, the camera is used to capture raw images and provide the image information required by this embodiment.

[0170] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0171] like Figure 8 As shown, the application framework layer can include a camera framework, window manager, content provider, view system, resource manager, etc.

[0172] The camera framework provides various services and interfaces for camera applications, enabling them to better interact with the underlying system. In this embodiment, the camera framework provides a camera service interface, allowing applications to access and control the camera hardware.

[0173] The Android runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0174] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0175] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0176] The system library can include multiple functional modules. For example: surface manager, memory manager, 3D graphics processing library (e.g., OpenGL ES), 2D graphics engine (e.g., SGL), etc.

[0177] The HAL layer is a wrapper around Linux kernel drivers, providing interfaces to the upper layers and shielding them from the implementation details of the lower-level hardware.

[0178] The HAL layer can include image processing interfaces, software code libraries, etc.

[0179] An image processing interface provides a standard interface for upper-layer software to interact with lower-layer hardware. In this embodiment, the image processing interface coordinates data transmission between the image sensor and the upper-layer software. For example, the image processing interface can pass raw data (such as RAW format data) acquired by the image sensor to the upper-layer software for further processing. It is understood that dynamic correction can be performed using the raw data.

[0180] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0181] The camera driver is part of the operating system kernel and is responsible for managing and controlling the camera hardware device. In this embodiment, the camera driver is responsible for direct communication with the image sensor and passing the data collected by the sensor to higher-level software components for processing. The technical solutions of this embodiment and how they solve the aforementioned technical problems are described in detail below with reference to the accompanying drawings and specific examples. These specific embodiments can be implemented independently or in combination with each other; similar or identical concepts or processes may not be repeated in some embodiments.

[0182] Based on the above description, the technical solution provided in this application will be described in detail below with reference to the specific accompanying drawings. Figure 9 This is a schematic diagram of the image sensor partitioning module provided in an embodiment of this application. Figure 10 The diagram shows two types of first-class photosensitive modules provided in the embodiments of this application. Figure 11 This is a schematic diagram of the photosensitive object type provided in the embodiments of this application. Figure 12 This application provides an illustration of setting up a photosensitive object for detection. Figure 1 , Figure 13 This application provides an illustration of setting up a photosensitive object for detection. Figure 2 , Figure 14 This application provides an illustration of setting up a photosensitive object for detection. Figure 3 , Figure 15 This application provides an illustration of setting up a photosensitive object for detection. Figure 4 , Figure 16 This application provides an illustration of setting up a photosensitive object for detection. Figure 5 , Figure 17 This application provides an illustration of setting up a photosensitive object for detection. Figure 6 , Figure 18 This application provides an illustration of setting up a photosensitive object for detection. Figure 7 .

[0183] In this embodiment, special processing is required for the normally photosensitive pixels in the image sensor. These pixels, which have undergone special processing, can be understood as detection photosensitive pixels. Crosstalk information can be extracted by setting the detection photosensitive object, and the image can be corrected based on this crosstalk information. The technical solution of this application is described below with several specific sub-implementations.

[0184] I. Division of Photosensitive Module

[0185] The technical solution of this application is based on the operation performed on the photosensitive object of the image sensor.

[0186] In one implementation, since the image sensor is based on a large number of photosensitive pixels, these pixels can be understood as photosensitive objects. A certain number of photosensitive pixels form a photosensitive unit, and a collection of such units can be considered as a photosensitive module.

[0187] When photosensitive pixels are used as the photosensitive object, a special process can be added to the location of each photosensitive pixel to obtain the photosensitive object to be detected. It can be understood that the photosensitive unit containing the photosensitive object can be considered a special photosensitive unit, and the photosensitive module containing the special photosensitive unit can be understood as a type I photosensitive module.

[0188] In another implementation, a certain number of photosensitive pixels form a photosensitive subunit, which can be understood as a photosensitive object. A certain number of photosensitive subunits can be combined into a photosensitive unit, and further, a collection of a certain number of photosensitive units can be considered as a photosensitive module.

[0189] When the photosensitive subunit is used as the photosensitive object, a special process can be added to the position of each photosensitive subunit to obtain the photosensitive object to be detected. Similarly, it can be understood that the photosensitive unit containing the photosensitive object can be understood as a special photosensitive unit, and the photosensitive module containing the special photosensitive unit can be understood as a first-type photosensitive module.

[0190] Therefore, for image sensors, photosensitive modules containing special photosensitive units can be classified as the first type of photosensitive modules, and photosensitive modules other than the first type of photosensitive modules can be understood as the remaining photosensitive modules.

[0191] Therefore, the photosensitive modules of the image sensor are divided into first-class photosensitive modules and remaining photosensitive modules. The first-class photosensitive modules can be used to calculate the crosstalk parameters of the remaining photosensitive modules, avoiding unnecessary waste of resources caused by adding special processes to all modules, and reducing manufacturing costs.

[0192] The following is combined with Figure 9 and Figure 10 Please understand the above content.

[0193] like Figure 9 As shown, suppose there exists an image sensor, which consists of several photosensitive modules. These photosensitive modules can be divided into two categories: one category is the first type of photosensitive module containing special photosensitive units, and the other category is the remaining photosensitive modules that do not contain special photosensitive units.

[0194] like Figure 10 As shown, there are two types of photosensitive modules for the first type: photosensitive module A and photosensitive module B.

[0195] For the first type of photosensitive module A, the case where photosensitive pixels are used as the photosensitive object can be considered. The first type of photosensitive module A consists of several photosensitive pixels, and a certain number of photosensitive pixels form a photosensitive unit. A special process can be added to each photosensitive pixel to obtain a photosensitive object for detection. The photosensitive unit containing the photosensitive object can be understood as a special photosensitive unit. Furthermore, a photosensitive module containing special photosensitive units can be understood as a first type of photosensitive module.

[0196] For the first type of photosensitive module B, there is the case where the photosensitive subunits are used as the photosensitive object. The first type of photosensitive module B consists of several photosensitive subunits, and a certain number of photosensitive subunits form a photosensitive unit. Special processes can be added to each photosensitive subunit to obtain a photosensitive object for detection. The photosensitive unit containing the photosensitive object can be understood as a special photosensitive unit. Furthermore, a photosensitive module containing special photosensitive units can be understood as a first type of photosensitive module.

[0197] It should be noted that, in one implementation, the position of the first type of photosensitive module can be set as follows: Figure 9 As shown in the figure, in actual production, the setting method of the first type of photosensitive module can be determined according to actual needs, and this application embodiment does not limit it.

[0198] II. Setting up the detection of photosensitive objects

[0199] Based on the above, it can be understood that there are two types of photosensitive objects: one is photosensitive pixels as photosensitive objects; the other is photosensitive subunits as photosensitive objects. Correspondingly, for these two types of photosensitive objects, there are two ways to configure the detection of photosensitive objects.

[0200] In this context, the photosensitive object can be understood as the photosensitive object that can be used to determine the crosstalk parameters. The crosstalk parameters can indicate the crosstalk signals of the surrounding photosensitive objects adjacent to the photosensitive object.

[0201] Therefore, depending on the different types of photosensitive objects, different sensors can be set to detect photosensitive objects, thereby improving the adaptability of the solution in this application.

[0202] The following is combined with Figure 11 To understand the two types of photosensitive objects.

[0203] like Figure 11 As shown in (a), assuming there exists a photosensitive unit, it may include: a red photosensitive subunit 1101, a green photosensitive subunit 1102, a green photosensitive subunit 1103, and a blue photosensitive subunit 1104. Each photosensitive subunit includes 4 photosensitive pixels. Therefore, it can be understood that the photosensitive unit includes 16 photosensitive pixels, namely: numbered 1, 2, ..., 15 and 16.

[0204] In one implementation, when the photosensitive object is a single photosensitive pixel, the photosensitive object can be understood, for example, as... Figure 11In the example (a), the photosensitive pixel numbered 1 can be detected as a photosensitive object after a special process has been applied to the photosensitive pixel numbered 1. It can also be understood that the photosensitive object can be a photosensitive object after a special process has been applied to photosensitive pixels numbered 2 through 16. Therefore, regarding... Figure 11 The photosensitive unit shown in (a) has 16 settings for detecting a single photosensitive pixel.

[0205] In another implementation, when the photosensitive object is a photosensitive subunit, the photosensitive object can be understood, for example, as... Figure 11 In the red photosensitive subunit 1101 shown in (b), the photosensitive object to be detected can be a photosensitive object after a special process has been added to the red photosensitive subunit 1101. It can also be understood that the photosensitive object to be detected can be a photosensitive object after a special process has been added to the photosensitive subunits 1102-1104. Therefore, for... Figure 11 The photosensitive unit shown in (b) has four settings for detecting the photosensitive object when the photosensitive object is a photosensitive subunit.

[0206] It should be noted that the above illustration is of a photosensitive unit of an image sensor. In actual operation, there are various types of image sensors. The specific structure or composition of the photosensitive unit can be determined according to the specific type of image sensor, and thus the setting method for detecting the photosensitive object can be determined. This application does not limit this.

[0207] The following sections describe how to set up the detection of two types of photosensitive objects.

[0208] Scenario 1: When the photosensitive object is a single pixel, the photosensitive object can be detected at the position of the single pixel.

[0209] When the photosensitive object is determined to be a single photosensitive pixel, for any first-type photosensitive module, it is necessary to set a detection photosensitive object at all possible positions in the photosensitive unit. The set detection photosensitive object can be used to extract the crosstalk parameters of the entire module.

[0210] The following is combined with Figures 12-16 This section introduces the different sensor settings for detecting light-sensitive objects.

[0211] like Figure 12 As shown, suppose there exists an image sensor of model A, whose photosensitive unit is as follows: Figure 12 As shown in (a) in the figure. For each photosensitive pixel in the photosensitive unit, a photosensitive object can be set at its position. The position of the photosensitive object can include 16 types, specifically positions numbered 1, 2...15 and 16.

[0212] like Figure 12 As shown in (b), assume there exists a photosensitive module containing 16×16 photosensitive pixels. For this photosensitive module, it can be... Figure 12 The 16 possible positions for detecting photosensitive objects in the photosensitive unit shown in (a) correspond to 16 different positions in the photosensitive module. Then, special processing is performed on these 16 corresponding positions to obtain a first type of photosensitive module with 16 different positions for detecting photosensitive objects.

[0213] After completing the settings for detecting photosensitive objects, you can target... Figure 12 Part 1201 in the middle, combined with Figure 12 (c) in the text is used to understand the process method for setting up an image sensor to detect a photosensitive object.

[0214] like Figure 12 As shown in (c), a side view of the image sensor after setting the photosensitive object is introduced. In this view, the photosensitive object 1203 is located below the filter layer 1202 and above the photosensitive layer 1204. The surface of the photosensitive layer is covered with metal to prevent the photosensitive object from being photosensitive.

[0215] like Figure 13 As shown, suppose there exists an image sensor of model B, whose photosensitive unit is as follows: Figure 13 As shown in (a) in the figure. For each photosensitive pixel in the photosensitive unit, a photosensitive object can be set at its position. The position of the photosensitive object can include 16 types, specifically positions numbered 1, 2...15 and 16.

[0216] like Figure 13 As shown in (b), assume there exists a photosensitive module containing 16×16 photosensitive pixels. For this photosensitive module, it can be... Figure 13 The 16 possible positions for detecting photosensitive objects in the photosensitive unit shown in (a) correspond to 16 different positions in the photosensitive module. Then, special processing is performed on these 16 corresponding positions to obtain a first type of photosensitive module with 16 different positions for detecting photosensitive objects.

[0217] After completing the settings for detecting photosensitive objects, you can target... Figure 13 Part 1301 in the middle, combined with Figure 13 (c) in the text is used to understand the process method for setting up an image sensor to detect a photosensitive object.

[0218] like Figure 13As shown in (c), a side view of the image sensor after setting the photosensitive object is introduced. In this view, the photosensitive object 1303 is located below the filter layer 1302 and above the photosensitive layer 1304. The surface of the photosensitive layer is covered with metal to prevent the photosensitive object from being photosensitive.

[0219] like Figure 14 As shown, suppose there exists an image sensor of model C, whose photosensitive unit is as follows: Figure 14 As shown in (a) in the figure. For each photosensitive pixel in the photosensitive unit, a photosensitive object can be set at its position. Therefore, the position of the photosensitive object can include 64 types, specifically positions numbered 1, 2...63 and 64.

[0220] like Figure 14 As shown in (b), assume there exists a photosensitive module containing 32×32 photosensitive pixels. For this photosensitive module, it can be... Figure 14 The 64 possible positions for detecting photosensitive objects in the photosensitive unit shown in (a) correspond to 64 different positions in the photosensitive module. Then, special processing is performed on these 64 corresponding positions to obtain a first type of photosensitive module with 64 different positions for detecting photosensitive objects.

[0221] After completing the settings for detecting photosensitive objects, you can target... Figure 14 Part 1401 in the middle, combined with Figure 14 (c) in the text is used to understand the process method for setting up an image sensor to detect a photosensitive object.

[0222] like Figure 14 As shown in (c), a side view of the image sensor after setting the photosensitive object is introduced. In this view, the photosensitive object 1403 is located below the filter layer 1402 and above the photosensitive layer 1404. The surface of the photosensitive layer is covered with metal to prevent the photosensitive object from being photosensitive.

[0223] like Figure 15 As shown, suppose there exists an image sensor of model D, whose photosensitive unit is as follows: Figure 15 As shown in (a) of the image. For each photosensitive pixel in the photosensitive unit, a photosensitive object can be set at its position. The positions of the photosensitive objects can include 32 types, specifically positions numbered 1, 2, ..., 31 and 32.

[0224] like Figure 15 As shown in (b), assume there exists a photosensitive module containing 32×20 photosensitive pixels. For this photosensitive module, it can be... Figure 15The 32 possible positions of the photosensitive unit shown in (a) correspond to 32 different positions in the photosensitive module. Then, special processing is performed on these 32 corresponding positions to obtain the first type of photosensitive module with 32 different positions for detecting photosensitive objects.

[0225] After completing the settings for detecting photosensitive objects, you can target... Figure 15 Part 1501 in the middle, combined with Figure 15 (c) in the text is used to understand the process method for setting up an image sensor to detect a photosensitive object.

[0226] like Figure 15 As shown in (c), a side view of the image sensor after setting the photosensitive object is introduced. In this view, the photosensitive object 1503 is located below the filter layer 1502 and above the photosensitive layer 1504. The surface of the photosensitive layer is covered with metal to prevent the photosensitive object from being photosensitive.

[0227] like Figure 16 As shown, suppose there exists an image sensor of model E, whose photosensitive unit is as follows: Figure 16 As shown in (a) of the image. For each photosensitive pixel in the photosensitive unit, a photosensitive object can be set at its location. The location of the photosensitive object can be one of four types: number 1, number 2, number 3, and number 4.

[0228] like Figure 16 As shown in (b), assume there exists a photosensitive module containing 8×8 photosensitive pixels. For this photosensitive module, it can be... Figure 16 The four possible settings for detecting the photosensitive object in the photosensitive unit shown in (a) correspond to four different positions in the photosensitive module. Then, special processing is performed on these four corresponding positions to obtain a first type of photosensitive module with four different settings for detecting the photosensitive object.

[0229] After completing the settings for detecting photosensitive objects, you can target... Figure 16 Part 1601 in the middle, combined with Figure 16 (c) in the text is used to understand the process method for setting up an image sensor to detect a photosensitive object.

[0230] like Figure 16 As shown in (c), a side view of the image sensor after setting the photosensitive object is introduced. In this view, the photosensitive object 1603 is located below the filter layer 1602 and above the photosensitive layer 1604. The surface of the photosensitive layer is covered with metal to prevent the photosensitive object from being photosensitive.

[0231] It should be noted that the above describes five types of image sensors. Among them, the structure of the first type of photosensitive module can be 16×16, 32×32, 32×20, or 8×8. The detection object is set according to the structure of the first type of photosensitive module. In practice, the composition structure of the first type of photosensitive module can be varied, and the structure of the first type of photosensitive module can be determined according to the specific type of image sensor. The embodiments of this application do not limit the structure of the first type of photosensitive module.

[0232] Scenario 2: When the photosensitive object is a photosensitive sub-unit, the photosensitive object can be detected based on the position of the photosensitive sub-unit.

[0233] The above describes the setting for detecting the photosensitive object when the photosensitive object is a photosensitive pixel. This setting applies to the image sensor's working mode, which outputs the information collected by each pixel one by one.

[0234] In fact, image sensors can also operate in a merged output mode. In this mode, the image sensor merges multiple adjacent pixels and then outputs a signal representing the overall situation of these merged pixels. For this purpose, the photosensitive sub-units composed of multiple pixels of the image sensor can serve as the photosensitive object, and the detection of the photosensitive object can be set according to the position of the photosensitive sub-units.

[0235] The following is combined with Figure 17 This section describes how sensors operating in merge mode are configured to detect light-sensitive objects.

[0236] like Figure 17 As shown, suppose there exists an image sensor of model F, whose photosensitive unit is as follows: Figure 17 As shown in (a) of the image. For each photosensitive subunit in the photosensitive unit, a photosensitive object can be set. The position of the photosensitive object can include four types, specifically positions numbered 1, 2, 3, and 4.

[0237] like Figure 17 As shown in (b), assume there exists a photosensitive module containing 16×16 photosensitive pixels. For this photosensitive module, it can be... Figure 17 The four possible settings for detecting the photosensitive object in the photosensitive unit shown in (a) correspond to four different positions in the photosensitive module. Then, special processing is performed on these four corresponding positions to obtain a first type of photosensitive module with four different settings for detecting the photosensitive object.

[0238] After completing the settings for detecting photosensitive objects, you can target... Figure 17 Part 1701 in the middle, combined with Figure 17(c) in the text is used to understand the process method for setting up an image sensor to detect a photosensitive object.

[0239] like Figure 17 As shown in (c), a side view of the image sensor after setting the photosensitive object is introduced. In this view, the photosensitive object 1703 is located below the filter layer 1702 and above the photosensitive layer 1704. The surface of the photosensitive layer is covered with metal to prevent the photosensitive object from being photosensitive.

[0240] exist Figures 12-17 In the described method of setting up the photosensitive object for detection, none of the photosensitive objects are photosensitive. This is understood because the photosensitive object is set up by covering the surface of the photosensitive layer with metal; therefore, the photosensitive object itself is not photosensitive, while the surrounding photosensitive objects adjacent to it are. It is also understood that because the photosensitive object is not photosensitive, it can only receive crosstalk signals from other surrounding photosensitive objects. The signals from the surrounding photosensitive objects do not include the crosstalk signal from the photosensitive object.

[0241] In another implementation, it is possible to detect the light sensitivity of a photosensitive object. This can be understood as follows: by placing photosensitive objects around a single photosensitive pixel, the photosensitive object does not generate crosstalk signals to the individual photosensitive pixel; the individual photosensitive pixel can only receive its own signal. Conversely, the photosensitive object cannot receive its own signal and can only receive crosstalk signals from surrounding photosensitive pixels. This method of setting up photosensitive objects allows for the detection of light sensitivity in photosensitive objects.

[0242] The following is combined with Figure 18 This section introduces a special method for setting up the detection of photosensitive objects.

[0243] like Figure 18 As shown, for image sensor model A, the photosensitive unit of this image sensor is as follows: Figure 18 As shown in (a) of the image. For each photosensitive pixel in the photosensitive unit, a photosensitive object can be set around the photosensitive pixel. Since there are 16 photosensitive pixels, a photosensitive object can be set for each of the 16 photosensitive pixels.

[0244] like Figure 18 As shown in (b), assume there exists a photosensitive module containing 20×20 photosensitive pixels. For this photosensitive module, it can be... Figure 18 In the photosensitive unit shown in (a), a photosensitive object is set around the 16 photosensitive pixels, which corresponds to the surrounding position of the 16 photosensitive pixels in the photosensitive module. Then, a special process is performed on the surrounding position of these 16 photosensitive pixels to obtain a first type of photosensitive module with the surrounding position of the 16 photosensitive pixels set.

[0245] After completing the settings for detecting photosensitive objects, you can target... Figure 18 The 1801 section, combined with Figure 18 (c) in the text is used to understand the process method for setting up an image sensor to detect a photosensitive object.

[0246] like Figure 18 As shown in (c), a side view of the image sensor after setting up the photosensitive object is introduced. In this view, the photosensitive object 1803 is located below the filter layer 1802 and above the photosensitive layer 1804. The surface of the photosensitive layer is covered with metal to prevent the photosensitive object from being photosensitive, while other photosensitive objects can be photosensitive normally.

[0247] It should be noted that the above describes a specific method for detecting light-sensitive objects using an image sensor of model A. In practice, other methods can also be used for... Figure 13 Image sensor of model B in Figure 14 Image sensor of model C in Figure 15 Image sensor of model D in Figure 16 The image sensor of model E and Figure 17 The image sensor of model F uses the same logic to set up the detection of photosensitive objects. This application does not limit the type of image sensor that can set up the detection of photosensitive objects in a special way.

[0248] Based on the above description, it is possible to set up photosensitive object detection when the photosensitive object is a photosensitive pixel, and also when the photosensitive object is a photosensitive subunit. By setting up photosensitive object detection, crosstalk information can be extracted using the photosensitive pixels.

[0249] In summary, this application first divides the image sensor into modules, defining the module containing the photosensitive object as the first type of photosensitive module, and then using the first type of module to calculate crosstalk parameters for the remaining photosensitive modules. Secondly, it introduces methods for setting the photosensitive object for different types of photosensitive objects. By setting the photosensitive object in the first type of photosensitive module, crosstalk information can be extracted. Setting the photosensitive object for different sensors according to the different types of photosensitive objects ensures the completeness and comprehensiveness of the solution in this application.

[0250] Based on the above, methods for setting detection targets for different types of sensors have been introduced. Building upon this, this application can obtain the crosstalk information of any photosensitive module based on the set detection target. Using the crosstalk information of this module, the crosstalk information of the remaining modules can be obtained, thereby achieving the extraction of all crosstalk information from the image sensor. The technical solution provided by this application will be described in detail below with reference to the accompanying drawings. Figure 19A A flowchart illustrating the process of obtaining crosstalk information provided in this application embodiment. Figure 1 , Figure 19B A flowchart illustrating the process of obtaining crosstalk information provided in this application embodiment. Figure 2 , Figure 20 A schematic diagram of crosstalk information calculation provided in the embodiments of this application. Figure 1 , Figure 21 A schematic diagram of determining crosstalk information of the remaining photosensitive modules provided in this application embodiment. Figure 1 , Figure 22 A schematic diagram of determining crosstalk information of the remaining photosensitive modules provided in this application embodiment. Figure 2 , Figure 23 A schematic diagram of crosstalk information calculation provided in the embodiments of this application. Figure 2 .

[0251] Based on the description of the section on setting up the photosensitive object, the photosensitive state of the photosensitive object can be divided into two types due to different methods of setting up the photosensitive object: In one implementation, surrounding photosensitive objects adjacent to the photosensitive object are photosensitive, while the photosensitive object itself is not photosensitive. This can be combined with... Figure 19A Understanding the process of calculating crosstalk information; in another implementation, surrounding photosensitive objects adjacent to the detected photosensitive object are not photosensitive, so the detected photosensitive object is photosensitive, which can be combined with... Figure 19B Understand the process of calculating crosstalk information.

[0252] By classifying the photosensitivity of the detected photosensitive objects, two methods can be set up to calculate the crosstalk information of any photosensitive module for the detected photosensitive objects with different photosensitivity, thereby achieving the integrity of the technical solution of this application.

[0253] Scenario 1: For example Figure 19A As shown, for the case where the photosensitive object is not photosensitive, the method may include:

[0254] S1901, Obtain the first detection signal of the first photosensitive object.

[0255] In this embodiment, the first photosensitive object can be understood as any photosensitive object adjacent to the detected photosensitive object; the first detection signal can be understood as the signal value detected by the first photosensitive object, including the signal value of the first photosensitive object itself and the signal values ​​of crosstalk between the first photosensitive object and other photosensitive objects.

[0256] The following is combined with Figure 20 The first photosensitive object and the first detection signal of the embodiments of this application are understood.

[0257] like Figure 20 As shown in (a), suppose there is a photosensitive unit that may include 16 photosensitive pixels, namely: numbered 1, numbered 2, ..., numbered 15 and numbered 16.

[0258] like Figure 20 As shown in (b) above, for Figure 20 The position of the photosensitive pixel numbered 1 shown in (a) can be mapped to the location of the photosensitive object after the photosensitive object is set. Figure 20 The photosensitive object SR1 is shown in (b) of the image.

[0259] like Figure 20 As shown in (b), taking the detected photosensitive object SR1 as an example, all photosensitive objects adjacent to SR1 can be understood as first photosensitive objects, which can be photosensitive objects PS1, PS2, PS3, PS4, PS5, PS6, PS7, and PS8. It can be understood that there can be 8 first photosensitive objects adjacent to the detected photosensitive object. It can also be understood that there can be 8 surrounding photosensitive objects adjacent to the detected photosensitive object.

[0260] Taking the first photosensitive object PS1 as an example, the detected signal value of PS1 includes PS1's own signal value and the crosstalk signal values ​​from the eight surrounding photosensitive objects adjacent to PS1. Since PS1 is adjacent to SR1, but SR1 is not photosensitive, SR1 does not generate crosstalk signal values ​​to PS1. Therefore, the signal value of PS1 includes PS1's own signal value and the crosstalk signal values ​​from the seven surrounding photosensitive objects adjacent to PS1. The signal value of PS1 can be represented as S. PS1 .

[0261] It should be noted that in this embodiment, PS1 is taken as the first photosensitive object. In practice, the first photosensitive object is not limited, but it is required to satisfy the requirement of surrounding photosensitive objects adjacent to the photosensitive object being detected.

[0262] S1902, Obtain the second detection signal of the second photosensitive object.

[0263] In this embodiment, the position of the second photosensitive object in the photosensitive unit is the same as the position of the first photosensitive object in the photosensitive unit, and there is no detection photosensitive object among the multiple photosensitive objects adjacent to the second photosensitive object.

[0264] Therefore, the second detection signal can be understood as the signal value detected by the second photosensitive object, including the signal value of the second photosensitive object itself and the crosstalk signal value of other photosensitive objects to the second photosensitive object.

[0265] Without considering the absence of a detection photosensitive object among the adjacent photosensitive objects of the second photosensitive object, since the position of the second photosensitive object in the photosensitive unit is the same as that of the first photosensitive object, the signal value detected by the second photosensitive object is the same as that detected by the first photosensitive object. However, if there is no detection photosensitive object among the adjacent photosensitive objects of the second photosensitive object, while there is one detection photosensitive object among the adjacent photosensitive objects of the first photosensitive object, then the second detection signal, compared to the first detection signal, contains an additional crosstalk signal from an adjacent photosensitive object whose position in the photosensitive unit is the same as that of the detection photosensitive object.

[0266] The following is combined with Figure 20 The second photosensitive object and the second detection signal of the embodiments of this application will be understood. In the present step, taking SR1 as the photosensitive object to be detected and PS1 as the first photosensitive object as an example, the second photosensitive object and the second detection signal will be explained.

[0267] like Figure 20 As shown in (a), in one implementation, the position of the photosensitive pixel numbered 16 can be... Figure 20 The position of the first photosensitive object PS1 shown in (b) corresponds to that in another implementation, the position of the photosensitive pixel numbered 16 can be... Figure 20 The position of PR1 shown in (b) corresponds to that in the diagram.

[0268] Since the positions of PR1 and PS1 are the same in the photosensitive unit, and there are no photosensitive objects around PR1, PR1 can be understood as the second photosensitive object.

[0269] It can also be understood that, taking the second photosensitive object PR1 as an example, the detected signal value of PR1 includes the signal value of PR1 itself and the crosstalk signal values ​​of the eight surrounding photosensitive objects adjacent to PR1. The signal value of PR1 can be expressed as S. PR1 .

[0270] It should be noted that in this embodiment of the application, PR1 is taken as the second photosensitive object. In practice, the second photosensitive object is not limited, but it needs to be in the same position in the photosensitive unit as the first photosensitive object, and there are no photosensitive objects to be detected in the adjacent surrounding photosensitive objects.

[0271] S1903. Determine the crosstalk signal between the first mapped photosensitive object and the second photosensitive object.

[0272] In this embodiment, the first mapped photosensitive object can be understood as a photosensitive object that is in the same position in the photosensitive unit as the detection photosensitive object, and the positional relationship between the first mapped photosensitive object and the second photosensitive object is the same as the positional relationship between the detection photosensitive object and the first photosensitive object.

[0273] The first detection signal includes the signal value of the first photosensitive object itself and the crosstalk signal values ​​of the seven surrounding photosensitive objects adjacent to the first photosensitive object, but does not include the crosstalk signal value of the detected photosensitive object to the first photosensitive object. The first detection signal includes the signal value of the second photosensitive object itself and the crosstalk signal values ​​of the eight surrounding photosensitive objects adjacent to the second photosensitive object, including the crosstalk signal value of the first mapped object to the second photosensitive object.

[0274] Meanwhile, the signal values ​​received by the first and second photosensitive objects are similar, the only difference being that the first photosensitive object is not affected by the detected photosensitive object. Therefore, the crosstalk signal between the first and second photosensitive objects can be the difference between the signal values ​​of the second and first photosensitive objects.

[0275] The following is combined with Figure 20 This application's embodiments will be used to understand the crosstalk signal between the first mapped photosensitive object and the second photosensitive object. In the current step, the detected photosensitive object is SR1, the first photosensitive object is PS1, and the first detection signal is S. PS1 The second photosensitive object is PR1 and the second detection signal is S. PR1 Let's take an example to illustrate.

[0276] like Figure 20 As shown in (a), SR1 corresponds to the position of photosensitive pixel number 1 in the photosensitive unit. The position of photosensitive pixel number 1 can also correspond to... Figure 20 The position of R1 is shown in (b) in the diagram, and since the positional relationship between R1 and PR1 is the same as that between SR1 and PS1, R1 can be understood as the first mapping object.

[0277] Since the first detection signal of PS1 is S PS1 Excluding the crosstalk signal from SR1 to PS1, and the second detection signal S of PR1PR1 Including the crosstalk signal from R1 to PR1, apart from the differences mentioned above, the crosstalk signal values ​​received from surrounding photosensitive objects are the same. Therefore, the crosstalk signal generated by R1 to PR1 can be determined according to S. PR1 With S PS1 The difference is used to determine the crosstalk signal generated by R1 to PR1, which can be expressed as CT. R1toPR1 S PR1 S PS1 and CT R1toPR1 The relationship can be shown in Formula 1:

[0278] CT R1toPR1 =S PR1 -S PS1 Formula 1

[0279] S1904. Determine the crosstalk signals of the photosensitive object located at the first position in each photosensitive unit to each of the adjacent surrounding photosensitive objects, as well as the crosstalk information corresponding to the first type of photosensitive module.

[0280] In the embodiments of this application, the first position can be understood as the position of the first mapped photosensitive object in the photosensitive unit.

[0281] According to Formula 1 of S1903, it can be determined that within each photosensitive unit, the crosstalk signal between the photosensitive object located at the first position and any adjacent photosensitive object can be the difference between the second detection signal and the first detection signal. Accordingly, multiple crosstalk signals can be obtained to indicate the crosstalk between the photosensitive object and the surrounding photosensitive objects adjacent to the photosensitive object.

[0282] The following is combined with Figure 20 The crosstalk signals of the photosensitive object located at the first position within each photosensitive unit to each of the adjacent surrounding photosensitive objects are understood.

[0283] like Figure 20 As shown in (a), assuming that a detection photosensitive object is set for the position of photosensitive pixel number 1, the detection photosensitive object can be obtained as SR1, the first mapped photosensitive object is R1, and the first position can be understood as the position of R1 in the photosensitive unit, that is, the position of photosensitive pixel number 1 in the photosensitive unit.

[0284] Assuming the first photosensitive object is PS1, and the first detection signal is S PS1 The second photosensitive object is PR1 and the second detection signal is S. PR1 Therefore, the crosstalk information of R1 to PR1 can be obtained through CT. R1toPR1 =S PR1 -S PS1 Calculated.

[0285] Assuming the first photosensitive object is PS2, and the first detection signal is S PS2 The second photosensitive object is PR2 and the second detection signal is S. PR2 Therefore, the crosstalk information of R1 to PR2 can be obtained through CT. R1toPR2 =S PR2 -S PS2 Calculated.

[0286] Assuming the first light-sensitive object is a PS3, and the first detection signal is S PS3 The second photosensitive object is PR3 and the second detection signal is S. PR3 Therefore, the crosstalk information of R1 to PR3 can be obtained through CT. R1toPR3 =S PR3 -S PS3 Calculated.

[0287] Assuming the first light-sensitive object is a PS4, and the first detection signal is S PS4 The second photosensitive object is PR4 and the second detection signal is S. PR4 Therefore, the crosstalk information of R1 to PR4 can be obtained through CT. R1toPR4 =S PR4 -S PS4 Calculated.

[0288] Assuming the first light-sensitive object is a PS5, and the first detection signal is S PS5 The second photosensitive object is PR5 and the second detection signal is S. PR5 Therefore, the crosstalk information of R1 to PR5 can be obtained through CT. R1toPR5 =S PR5 -S PS5 Calculated.

[0289] Assuming the first light-sensing object is PS6, and the first detection signal is S PS6 The second photosensitive object is PR6 and the second detection signal is S. PR6 Therefore, the crosstalk information of R1 to PR6 can be obtained through CT. R1toPR6 =S PR6 -S PS6 Calculated.

[0290] Assuming the first light-sensing object is a PS7, and the first detection signal is S PS7 The second photosensitive object is PR7 and the second detection signal is S. PR7 Therefore, the crosstalk information of R1 to PR7 can be obtained through CT. R1toPR7 =S PR7 -S PS7 Calculated.

[0291] Assuming the first light-sensing object is PS8, and the first detection signal is SPS8 The second photosensitive object is PR8 and the second detection signal is S. PR8 Therefore, the crosstalk information of R1 to PR8 can be obtained through CT. R1toPR8 =S PR8 -S PS8 Calculated.

[0292] Based on the above calculation formula, the first position of the first mapped object corresponding to the detected photosensitive object can be obtained. When it corresponds to the position of the photosensitive pixel numbered 1 in the photosensitive unit, the crosstalk information of the photosensitive object at the first position to the eight adjacent photosensitive objects can be obtained.

[0293] like Figure 20 As shown in (a), assuming that a detection photosensitive object is set for the position of photosensitive pixel number 2, the detection photosensitive object can be obtained as SR2, the first mapped photosensitive object is R2, and the first position can be understood as the position of R2 in the photosensitive unit, that is, the position of photosensitive pixel number 2 in the photosensitive unit.

[0294] Assuming the first photosensitive object is PY1 and the first detection signal is S PY1 The second photosensitive object is PL1 and the second detection signal is S. PL1 Therefore, the crosstalk information of R2 to PL1 can be obtained through CT. R2toPL1 =S PL1 -S PY1 Calculated.

[0295] Assuming the first photosensitive object is PY2 and the first detection signal is S PY2 The second photosensitive object is PL2 and the second detection signal is S. PL2 Therefore, the crosstalk information of R2 to PL2 can be obtained through CT. R2toPL2 =S PL2 -S PY2 Calculated.

[0296] Assuming the first photosensitive object is PY3 and the first detection signal is S PY3 The second photosensitive object is PL3 and the second detection signal is S. PL3 Therefore, the crosstalk information of R2 to PL3 can be obtained through CT. R2toPL3 =S PL3 -S PY3 Calculated.

[0297] Assuming the first photosensitive object is PY4 and the first detection signal is S PY4 The second photosensitive object is PL4 and the second detection signal is S. PL4 Therefore, the crosstalk information of R2 to PL4 can be obtained through CT. R2toPL4 =S PL4-S PY4 Calculated.

[0298] Assuming the first photosensitive object is PY5 and the first detection signal is S PY5 The second photosensitive object is PL5 and the second detection signal is S. PL5 Therefore, the crosstalk information of R2 to PL5 can be obtained through CT. R2toPL5 =S PL5 -S PY5 Calculated.

[0299] Assuming the first photosensitive object is PY6 and the first detection signal is S PY6 The second photosensitive object is PL6 and the second detection signal is S. PL6 Therefore, the crosstalk information of R2 to PL6 can be obtained through CT. R2toPL6 =S PL6 -S PY6 Calculated.

[0300] Assuming the first photosensitive object is PY7 and the first detection signal is S PY7 The second photosensitive object is PL7 and the second detection signal is S. PL7 Therefore, the crosstalk information of R2 to PL7 can be obtained through CT. R2toPL7 =S PL7 -S PY7 Calculated.

[0301] Assuming the first photosensitive object is PY8 and the first detection signal is S PY8 The second photosensitive object is PL8 and the second detection signal is S. PL8 Therefore, the crosstalk information of R2 to PL8 can be obtained through CT. R2toPL8 =S PL8 -S PY8 Calculated.

[0302] Based on the above calculation formula, the first position of the first mapped object corresponding to the detected photosensitive object can be obtained. When it corresponds to the position of the photosensitive pixel numbered 2 in the photosensitive unit, the crosstalk information of the photosensitive object at the first position to the eight adjacent photosensitive objects can be obtained.

[0303] So, assuming that a detection photosensitive object is set for the positions of photosensitive pixels numbered 3 to 16, the detection photosensitive object can be obtained, and the first mapped photosensitive object can be obtained based on the detection photosensitive object. Then, the crosstalk information of the photosensitive object located at the first position to the eight adjacent photosensitive objects can be calculated according to the same calculation logic as above.

[0304] It is understandable that after setting a photosensitive pixel at one location in a photosensitive unit, corresponding to the first location, the crosstalk parameters of the photosensitive object at the first location to the surrounding eight photosensitive objects can be obtained. The crosstalk signals received by the photosensitive object from the surrounding eight photosensitive objects can be summed to obtain the crosstalk parameters received by the photosensitive object. It is also understandable that a photosensitive unit can have 16 locations where photosensitive objects can be set, thus obtaining crosstalk parameters for 16 locations.

[0305] It can also be understood that, based on the crosstalk parameters of the 16 positions obtained, one of the 16 positions can be used to calculate the crosstalk parameters of the photosensitive object located at the first position to any surrounding photosensitive object for all photosensitive objects in the first type of photosensitive module that meet the first position.

[0306] It should be noted that the above refers to... Figure 12 Taking the A-type image sensor as an example, we calculate the crosstalk parameters of a photosensitive object located at the first position to its adjacent surrounding photosensitive objects. However, in actual operation... Figure 13 The B-type image sensor in the middle, Figure 14 The C-type image sensor in the middle, Figure 15 The D-type image sensor in the middle, Figure 16 The E-type image sensor and Figure 17 All F-type image sensors described above can calculate crosstalk parameters using the logic described above. This application does not limit the type of image sensor that can use the above calculation logic to calculate crosstalk parameters, but it must meet the constraints of detecting a photosensitive object, a first photosensitive object, a first detection signal, a second photosensitive object, a second detection signal, a first mapping object, and a first position.

[0307] It should also be noted that the first-type photosensitive module for image sensor model A can calculate crosstalk parameters at 16 locations, the first-type photosensitive module for image sensor model B can calculate crosstalk parameters at 16 locations, the first-type photosensitive module for image sensor model C can calculate crosstalk parameters at 64 locations, the first-type photosensitive module for image sensor model D can calculate crosstalk parameters at 32 locations, the first-type photosensitive module for image sensor model E can calculate crosstalk parameters at 4 locations, and the first-type photosensitive module for image sensor model F can calculate crosstalk parameters at 4 locations.

[0308] S1905. For the crosstalk information of any first-type photosensitive module, determine the crosstalk information of multiple photosensitive modules located around the first-type photosensitive module.

[0309] In the embodiments of this application, when the photosensitive object is detected to be non-photosensitive, the crosstalk signals of the photosensitive object located at the first position in each photosensitive unit to each of the adjacent surrounding photosensitive objects are used to obtain the crosstalk information of the first type of photosensitive module.

[0310] For example, the first-type photosensitive module of image sensor A can calculate crosstalk parameters at 16 locations, the first-type photosensitive module of image sensor B can calculate crosstalk parameters at 16 locations, the first-type photosensitive module of image sensor C can calculate crosstalk parameters at 64 locations, the first-type photosensitive module of image sensor D can calculate crosstalk parameters at 32 locations, the first-type photosensitive module of image sensor E can calculate crosstalk parameters at 4 locations, and the first-type photosensitive module of image sensor F can calculate crosstalk parameters at 4 locations.

[0311] Based on the above, crosstalk information for any single type-1 photosensitive module can be obtained. This crosstalk information can then be used to determine the crosstalk information of multiple photosensitive modules located around the single type-1 photosensitive module.

[0312] The following describes two methods for determining the crosstalk information of the remaining photosensitive modules. In one method, when the first type of photosensitive modules are uniformly distributed, a layer number threshold can be preset. Here, the layer number can be understood as the distance between the detected photosensitive object and surrounding photosensitive objects. When the layer number of the surrounding photosensitive modules adjacent to the first type of photosensitive module is not greater than this layer number threshold, the crosstalk parameter of the adjacent surrounding photosensitive modules can be the same as the crosstalk parameter of the first type of photosensitive module. In another method, when the first type of photosensitive modules are not uniformly distributed, the crosstalk parameter of the adjacent surrounding photosensitive modules can be calculated based on the calculation method of the equal distance difference and the crosstalk parameter of the first type of photosensitive module. Here, the equal distance difference can be understood as the difference in crosstalk parameter at each corresponding position of two adjacent photosensitive modules being the same.

[0313] The following section addresses the case of a uniformly distributed photosensitive module in the first category, combined with... Figure 21 This section introduces a method for determining the crosstalk parameters of the remaining photosensitive modules.

[0314] like Figure 21 As shown in (a), assuming there is an image sensor 2101, the image sensor 2101 is divided into a first type of photosensitive module and a remaining photosensitive module. The first type of photosensitive module can be understood as multiple 2103, that is, photosensitive modules that include the detection of photosensitive objects; the remaining photosensitive modules can be understood as multiple 2102, that is, photosensitive modules that do not include the detection of photosensitive objects.

[0315] Regarding the details of the first type of photosensitive module 2103, it can be combined with Figure 21 To understand it from (b) in the text. For example Figure 21 As shown in (b), suppose there is a first type of photosensitive module 2103 containing 10×10 photosensitive pixels, and the photosensitive objects detected in the first type of photosensitive module 2103 are sparsely distributed. For the four positions in a photosensitive unit that can be set to detect photosensitive objects, they can be mapped to the four positions in different photosensitive units of the first type of photosensitive module 2103 that can be set to detect photosensitive objects.

[0316] It should be noted that the above illustration shows one distribution of photosensitive objects. In practice, the distribution of photosensitive objects can be determined according to actual needs, and this application embodiment does not limit this.

[0317] The four locations of the photosensitive objects are set to correspond to different positions in the photosensitive unit. For example, the first photosensitive object 2104 is located in the upper left corner of the photosensitive unit, the second photosensitive object 2105 is located in the upper right corner of the photosensitive unit, the third photosensitive object 2106 is located in the lower left corner of the photosensitive unit, and the fourth photosensitive object 2107 is located in the lower right corner of the photosensitive unit.

[0318] Meanwhile, based on the method for calculating crosstalk parameters described above, assuming the crosstalk parameter at the position of the first photosensitive object 2104 is 'a', the crosstalk parameter at the position of the second photosensitive object 2105 is 'b', the crosstalk parameter at the position of the third photosensitive object 2106 is 'c', and the crosstalk parameter at the position of the fourth photosensitive object 2107 is 'd', then the crosstalk parameters for the remaining photosensitive objects can be obtained based on these four positions.

[0319] When determining the crosstalk parameters of the remaining photosensitive modules, such as Figure 21 As shown in (a), assuming the layer threshold is preset to 1, it can be understood that the crosstalk parameters of photosensitive modules 1 to 8 within a circle around the first type of photosensitive module 2103 are the same as the crosstalk parameters of the first type of photosensitive module.

[0320] For ease of understanding, regarding the crosstalk parameters at the four positions described above, let's assume that the value of 'a' is set to 1, the value of 'b' to 2, the value of 'c' to 3, and the value of 'd' to 4. Then, for any photosensitive unit in the first type of photosensitive module 2103, the crosstalk parameter of the photosensitive object located at the upper left corner of the photosensitive unit is 1, the crosstalk parameter of the photosensitive object located at the upper right corner of the photosensitive unit is 2, the crosstalk parameter of the photosensitive object located at the lower left corner of the photosensitive unit is 3, and the crosstalk parameter of the photosensitive object located at the lower right corner of the photosensitive unit is 4. Based on this, for the convenience of subsequent explanations, in this embodiment, the crosstalk parameters of each of the four photosensitive objects in a photosensitive unit can be represented as a sequence, which is (1, 2, 3, 4).

[0321] Since the crosstalk parameters of photosensitive modules 1 to 8 within a circle around the first type of photosensitive module 2103 are the same as those of the first type of photosensitive module, the crosstalk parameters corresponding to the photosensitive objects in each photosensitive unit of modules 1 to 8 can also be expressed as (1, 2, 3, 4).

[0322] It should be noted that the method of using a sequence to represent the crosstalk parameters of the four positions of the photosensitive module here can also be applied to the method of representing the crosstalk parameters of the photosensitive module when determining the crosstalk parameters of the remaining photosensitive modules using equal distance differences.

[0323] The above describes a method for determining the remaining photosensitive modules using a preset threshold when the first type of photosensitive modules are evenly distributed. In another embodiment, when the first type of photosensitive modules are unevenly distributed, the crosstalk parameters of the surrounding photosensitive modules adjacent to the first type of photosensitive modules can be determined using a difference method.

[0324] To understand the concept of difference, consider this example: given four numbers, the largest and smallest, to show a trend of difference between them, we can take the difference between the largest and smallest numbers. Then, we divide this difference by the number of intervals between the largest and smallest numbers to calculate the common difference. Based on this common difference, we can calculate the difference for each number.

[0325] For example, in a sequence (1, x, y, 4), to make the four numbers in the sequence show a trend of changing differences, we first need to calculate the common difference, which is calculated as (4-1) / 3 = 1. Based on the calculated common difference, we can determine the values ​​of x and y, calculated as x = 1 + 1 = 2, y = 2 + 1 = 3. Based on the calculated values ​​of x and y, we can determine that the sequence is (1, 2, 3, 4), and understand that the sequence shows a pattern of changing differences.

[0326] Based on the above logic, the following is combined with... Figure 22This paper introduces a method for determining the crosstalk parameters of the remaining photosensitive modules using equal distance differences.

[0327] like Figure 22 As shown, assuming there is an image sensor 2201, this image sensor 2201 is divided into a first type of photosensitive module and remaining photosensitive modules. The first type of photosensitive module can be understood as multiple 2203, i.e., photosensitive modules that include the detection of photosensitive objects; the remaining photosensitive modules can be understood as multiple 2202, i.e., photosensitive modules that do not include the detection of photosensitive objects.

[0328] Assuming that the first type of photosensitive modules are unevenly distributed in the image sensor, and the layer number threshold is preset to 1, it can be understood that the crosstalk parameters of photosensitive modules 1 to 8 within a circle around the first type of photosensitive module A are the same as the crosstalk parameters of the first type of photosensitive module A, and the crosstalk parameters of photosensitive modules 9 to 16 within a circle around the first type of photosensitive module B are the same as the crosstalk parameters of the first type of photosensitive module B.

[0329] For example, the first type of photosensitive module A has crosstalk parameters at 4 positions, which can be represented as (1, 2, 3, 4); the first type of photosensitive module B has crosstalk parameters at 4 positions, which can be represented as (4, 5, 6, 7).

[0330] Therefore, according to the preset layer threshold, the crosstalk parameters of photosensitive modules 1 to 8 are the same as those of the first type of photosensitive module A, which are (1, 2, 3, 4); the crosstalk parameters of photosensitive modules 9 to 16 are the same as those of the first type of photosensitive module B, which are (4, 5, 6, 7).

[0331] Since photosensitive module 17 belongs to the photosensitive modules within two rings surrounding photosensitive module A of the first type, and photosensitive module 17 belongs to the photosensitive modules within three rings surrounding photosensitive module B of the first type, both have a higher than the preset layer number threshold; similarly, photosensitive module 18 belongs to the photosensitive modules within two rings surrounding photosensitive module B of the first type, and photosensitive module 18 belongs to the photosensitive modules within three rings surrounding photosensitive module A of the first type, both have a higher than the preset layer number threshold. In one implementation, the crosstalk parameters of photosensitive modules 17 and 18 can be calculated using the crosstalk parameters of photosensitive modules 5 and 12.

[0332] In one possible implementation of the difference calculation, photosensitive module 17 and photosensitive module 18 are located between photosensitive module 5 and photosensitive module 12. The crosstalk parameters of photosensitive module 17 and photosensitive module 18 can be calculated by utilizing the difference law that the crosstalk parameters of each position of photosensitive module 5, photosensitive module 17, photosensitive module 18 and photosensitive module 12 conform to.

[0333] Based on the above, the crosstalk parameters of photosensitive module 5 are (1, 2, 3, 4), and the crosstalk parameters of photosensitive module 12 are (4, 5, 6, 7). Photosensitive module 12 is located within three concentric rings surrounding photosensitive module 5. Assuming equal distance differences, using the crosstalk parameter 1 at the first position of photosensitive module 5 and the crosstalk parameter 4 at the first position of photosensitive module 12, the tolerance of the crosstalk parameter at the first position can be calculated as (4-1) / 3 = 1. Following the same logic, the tolerance of the crosstalk parameter at the second position is (5-2) / 3 = 1, the tolerance of the crosstalk parameter at the third position is (6-3) / 3 = 1, and the difference in the crosstalk parameter at the fourth position is (7-4) / 3 = 1.

[0334] Based on the above, the crosstalk parameters of the photosensitive module 17, based on the photosensitive module 5, can be (1+1, 2+1, 3+1, 4+1), i.e. (2, 3, 4, 5); the crosstalk parameters of the photosensitive module 18, based on the photosensitive module 17, can be (2+1, 3+1, 4+1, 5+1), i.e. (3, 4, 5, 6).

[0335] By using the crosstalk parameters (1, 2, 3, 4) of photosensitive module 5, the crosstalk parameters (2, 3, 4, 5) of photosensitive module 17, the crosstalk parameters (3, 4, 5, 6) of photosensitive module 18, and the crosstalk parameters (4, 5, 6, 7) of photosensitive module 12, the crosstalk parameters can be gradually changed, reducing the error in calculating the crosstalk information of the remaining photosensitive modules using the crosstalk information of the first type of photosensitive module.

[0336] It should be noted that the above describes two methods for determining the crosstalk parameters of the remaining photosensitive module. Alternatively, the crosstalk parameters of the photosensitive module can be multiplied by a coefficient based on the distance between the photosensitive module and the first-type photosensitive module. In practice, the method for determining the crosstalk parameters of the remaining photosensitive module can be chosen according to specific needs, and this application does not limit this approach.

[0337] S1906. Determine the crosstalk information of all modules in the image sensor.

[0338] Based on the above, when the detected photosensitive object is not photosensitive, the crosstalk information of the first type of photosensitive module can be calculated according to the set photosensitive object. Based on the calculated crosstalk information of the first type of photosensitive module, the crosstalk information of the remaining photosensitive modules can be obtained according to a preset layer threshold or difference method. Therefore, the crosstalk information of all modules in the image sensor can be determined.

[0339] S1907. Store the crosstalk information corresponding to each of the multiple photosensitive modules in the first storage space.

[0340] In this embodiment, the first storage space can be understood as the OTP of the image sensor or the EEPROM within the module. Storing the crosstalk information corresponding to each of the multiple photosensitive modules calculated in the image sensor into the first storage space can improve the convenience of extracting crosstalk information when using correction algorithms later.

[0341] Scenario 2: For example Figure 19B As shown, for detecting the light sensitivity of a photosensitive object, the method may include:

[0342] S1908. Acquire the third detection signal of the photosensitive object.

[0343] When detecting light from a photosensitive object, the photosensitive objects can be positioned around a single photosensitive object. In this case, since the photosensitive objects do not receive signals, the single photosensitive object will not be affected by crosstalk from its neighboring photosensitive objects. Therefore, the signal value detected by the single photosensitive object is its own signal value, which can be understood as a third detection signal.

[0344] The following is combined with Figure 23 To understand the third detection signal, we will first explain the third detection signal in this step.

[0345] like Figure 23 As shown in (a), suppose there is a photosensitive unit that may include 16 photosensitive pixels, namely: numbered 1, numbered 2, ..., numbered 15 and numbered 16.

[0346] like Figure 23 As shown in (b) above, assume there exists a first-type photosensitive module 2301 of an image sensor with model A. Figure 23 As shown in (c), assume there is an image sensor of type A that does not contain a photosensitive module 2302 for detecting photosensitive objects. The photosensitive modules 2302 of the first type of photosensitive module 2301 are adjacent, and the positions of the photosensitive objects in the two modules correspond one-to-one. For example, the position of photosensitive object X1 in the first type of photosensitive module 2301 corresponds to the position of photosensitive object Y1 in the photosensitive module 2302, and the position of photosensitive object X2 in the first type of photosensitive module 2301 corresponds to the position of photosensitive object Y2 in the photosensitive module 2302.

[0347] Simultaneously, the position of photosensitive pixel number 1 within the photosensitive unit is the same as the position of photosensitive object X1 within the photosensitive unit. When photosensitive objects are placed around photosensitive object X1, since these surrounding photosensitive objects are not photosensitive, the signal detected by photosensitive object X1 can be understood as a third detection signal, which can be represented as S. X1 The signal value includes the signal value of the photosensitive object X1 itself.

[0348] S1909, Obtain the fourth detection signal of the second mapped photosensitive object.

[0349] In this embodiment, the photosensitive module to which the second mapped photosensitive object belongs is adjacent to the photosensitive module to which the detection photosensitive object belongs, and the position of the second mapped photosensitive object in its respective photosensitive module is the same as the position of the detection photosensitive object in its respective photosensitive module. The signal detected by the second mapped photosensitive object can be understood as the fourth detection signal.

[0350] The following is combined with Figure 23 To understand the second mapping object and the fourth detection signal, in this step, we will take setting up a detection photosensitive object around the photosensitive object X1 as an example to explain the second mapping object and the fourth detection signal.

[0351] Since the first photosensitive module 2301 and the photosensitive module 2302 are adjacent, and the position of photosensitive object X1 in the first photosensitive module 2301 is the same as the position of photosensitive object Y1 in the photosensitive module 2302, then photosensitive object Y1 can be understood as the second mapping object. It can be understood that the signal value detected by photosensitive object Y1 can be interpreted as the fourth detection signal, which can be represented as S. Y1 The signal value includes: the signal value of the photosensitive object Y1 itself and the crosstalk signal values ​​of the eight surrounding photosensitive objects adjacent to the photosensitive object Y1.

[0352] S1910. Determine the crosstalk parameters corresponding to the photosensitive object being detected.

[0353] The third detection signal value is the signal value of the photosensitive object itself, which is surrounded by the photosensitive object, and does not include the crosstalk signal value of the photosensitive object to the photosensitive object. The fourth detection signal includes the signal value of the second mapping photosensitive object itself and the crosstalk signal values ​​of the eight surrounding photosensitive objects to the second mapping photosensitive object.

[0354] Meanwhile, the signal value reception of the photosensitive object surrounded by the detected photosensitive object is similar to that of the second mapped photosensitive object. The only difference is that the photosensitive object surrounded by the detected photosensitive object is not affected by the detected photosensitive object.

[0355] Therefore, based on the difference between the fourth detection signal corresponding to the second mapped photosensitive object and the third detection signal corresponding to the photosensitive object surrounded by the detected photosensitive object, the crosstalk parameters of the eight surrounding photosensitive objects adjacent to the second mapped photosensitive object can be obtained.

[0356] The following is combined with Figure 23To understand the crosstalk parameters corresponding to the photosensitive object, in the current step, a photosensitive object is set up around the photosensitive object X1. The third detection signal is based on the signal value detected by the photosensitive object X1. The second mapping object is Y1. The fourth detection signal is based on the signal value detected by the second mapping object Y1.

[0357] Since the third detection signal of X1 is S X1 Excluding the crosstalk signal from the photosensitive object to X1, and the fourth detection signal S of Y1 Y1 This includes the crosstalk signals from the eight surrounding photosensitive objects adjacent to Y1 to Y1. Aside from the differences mentioned above, the received signal values ​​from these eight objects are identical. Therefore, the crosstalk signals from the eight surrounding photosensitive objects adjacent to Y1 to Y1 can be determined based on S... Y1 With S X1 The difference is used to determine the crosstalk signal of Y1 from the eight surrounding photosensitive objects adjacent to Y1, which can be expressed as CT. YtoY1 S Y1 S X1 and CT YtoY1 The relationship can be shown in Formula 2:

[0358] CT YtoY1 =S Y1 -S X1 Formula 2

[0359] S1911. Determine the crosstalk parameters corresponding to the photosensitive modules located in the second position within each photosensitive unit, as well as the crosstalk information corresponding to the first type of photosensitive modules.

[0360] In the embodiments of this application, the second position can be understood as the position of the photosensitive object surrounded by the photosensitive object in the photosensitive unit, and also corresponds to the position of the second mapped photosensitive object in the photosensitive unit.

[0361] According to Formula 2 of S1910, within each photosensitive unit, it can be determined that the crosstalk signal of the photosensitive object located at the second position to the surrounding eight photosensitive objects can be the difference between the fourth detection signal and the third detection signal. Based on this, the crosstalk signal of the surrounding photosensitive objects adjacent to the photosensitive object to the photosensitive object can be obtained.

[0362] The following is combined with Figure 23 The crosstalk signal of the photosensitive object located in the second position within each photosensitive unit to the photosensitive object located in the second position is understood.

[0363] like Figure 23As shown in (a), assuming that a detection photosensitive object is set around the position of photosensitive pixel number 1, the detection photosensitive object set around the photosensitive object X1 can be obtained. The second mapping photosensitive object is Y1. The second position can be understood as the position of X1 in the photosensitive unit, or it can be understood as the position of Y1 in the photosensitive unit, that is, the position of photosensitive pixel number 1 in the photosensitive unit.

[0364] Assume the third measured signal is S X1 The second photosensitive object is Y1 and the fourth detection signal is S. Y1 Therefore, the crosstalk information of the eight surrounding photosensitive objects adjacent to Y1 to Y1 can be obtained through CT. YtoY1 =S Y1 -S X1 Calculated.

[0365] like Figure 23 As shown in (a), assuming that a detection photosensitive object is set around the position of photosensitive pixel number 2, the detection photosensitive object set around the photosensitive object X2 can be obtained. The second mapping photosensitive object is Y2. The second position can be understood as the position of X2 in the photosensitive unit, or it can be understood as the position of Y2 in the photosensitive unit, that is, the position of photosensitive pixel number 2 in the photosensitive unit.

[0366] Assume the third measured signal is S X2 The second photosensitive object is Y2 and the fourth detection signal is S. Y2 Therefore, the crosstalk information of the eight surrounding photosensitive objects adjacent to Y2 to Y2 can be obtained through CT. YtoY2 =S Y2 -S X2 Calculated.

[0367] So, assuming that photosensitive objects are set around the positions of photosensitive pixels numbered 3 to 16, we can obtain photosensitive objects surrounding a single photosensitive object. Based on this single photosensitive object, we can obtain a second mapped photosensitive object. Then, using the same calculation logic as above, we can calculate the crosstalk information generated by the eight adjacent surrounding photosensitive objects on the photosensitive object located at the second position.

[0368] It can be understood that after setting up detection pixels around one position in a photosensitive unit, corresponding to a second position, the crosstalk parameters of the photosensitive object at the second position can be obtained, which are influenced by the crosstalk parameters of the eight adjacent photosensitive objects. Since a photosensitive unit can have 16 positions around which detection pixels can be set, 16 crosstalk parameters can be obtained. These crosstalk parameters are the sum of the crosstalk parameters generated by the eight surrounding photosensitive objects on the photosensitive object at the second position.

[0369] It can also be understood that, based on the obtained 16 crosstalk parameters, one of the 16 crosstalk parameters can be used to calculate the crosstalk parameters of the photosensitive object located at the second position to the eight adjacent photosensitive objects for all photosensitive objects in the first type of photosensitive module that meet the second position.

[0370] It should be noted that the above refers to... Figure 12 Taking the A-type image sensor as an example, we calculate the crosstalk parameters of a photosensitive object located at the first position to its adjacent surrounding photosensitive objects. However, in actual operation... Figure 13 The B-type image sensor in the middle, Figure 14 The C-type image sensor in the middle, Figure 15 The D-type image sensor in the middle, Figure 16 The E-type image sensor and Figure 17 All F-type image sensors described above can calculate crosstalk parameters using the logic described above. This application does not limit the type of image sensor that can use the above calculation logic to calculate crosstalk parameters, but it must meet the constraints of a special setting method for detecting the photosensitive object, a third detection signal, a second mapped photosensitive object, a fourth detection signal, and a second position.

[0371] It should also be noted that the first-type photosensitive module for image sensor model A can calculate crosstalk parameters at 16 locations, the first-type photosensitive module for image sensor model B can calculate crosstalk parameters at 16 locations, the first-type photosensitive module for image sensor model C can calculate crosstalk parameters at 64 locations, the first-type photosensitive module for image sensor model D can calculate crosstalk parameters at 32 locations, the first-type photosensitive module for image sensor model E can calculate crosstalk parameters at 4 locations, and the first-type photosensitive module for image sensor model F can calculate crosstalk parameters at 4 locations.

[0372] S1912. For the crosstalk information of any first-type photosensitive module, determine the crosstalk information of multiple photosensitive modules located around the first-type photosensitive module.

[0373] When a photosensitive object is detected to be photosensitive, the photosensitive object located in the second position within each photosensitive unit is subjected to crosstalk signals from the eight adjacent surrounding photosensitive objects, thereby obtaining the crosstalk information of the first type of photosensitive module.

[0374] For example, the first-type photosensitive module of image sensor A can calculate crosstalk parameters at 16 locations, the first-type photosensitive module of image sensor B can calculate crosstalk parameters at 16 locations, the first-type photosensitive module of image sensor C can calculate crosstalk parameters at 64 locations, the first-type photosensitive module of image sensor D can calculate crosstalk parameters at 32 locations, the first-type photosensitive module of image sensor E can calculate crosstalk parameters at 4 locations, and the first-type photosensitive module of image sensor F can calculate crosstalk parameters at 4 locations.

[0375] Based on the above, crosstalk information for any one of the first-type photosensitive modules can be obtained. This crosstalk information can then be used to determine the crosstalk information of multiple photosensitive modules located around the first-type photosensitive module. The specific implementation of this step can be found in S1905, and will not be elaborated upon here.

[0376] S1913. Determine the crosstalk information of all modules in the image sensor.

[0377] Based on the above, when detecting light from a photosensitive object, the crosstalk information of the first type of photosensitive module can be calculated according to the set photosensitive object. Based on the calculated crosstalk information of the first type of photosensitive module, the crosstalk information of the remaining photosensitive modules can be obtained according to a preset layer threshold or difference method. Therefore, the crosstalk information of all modules in the image sensor can be determined.

[0378] S1914. Store the crosstalk information corresponding to each of the multiple photosensitive modules in the first storage space.

[0379] In this embodiment, the first storage space can be understood as the OTP of the image sensor or the EEPROM within the module. Storing the crosstalk information corresponding to each of the multiple photosensitive modules calculated in the image sensor into the first storage space can improve the convenience of extracting crosstalk information when using correction algorithms later.

[0380] In summary, this application provides two methods for calculating crosstalk information of the first type of photosensitive modules by detecting whether a photosensitive object is photosensitive. Based on the calculated crosstalk information of the first type of photosensitive modules, the crosstalk information of the remaining photosensitive modules can be calculated using a preset threshold or difference method. Based on the obtained crosstalk information of the remaining photosensitive modules and the crosstalk information of the first type of photosensitive modules, the crosstalk information of all photosensitive modules in the image sensor can be obtained. Finally, storing the crosstalk information of all photosensitive modules in a first space facilitates subsequent use of correction algorithms to extract crosstalk parameters to achieve image correction.

[0381] In this embodiment, the detection signal of each photosensitive object can be compensated according to the crosstalk parameters of each photosensitive object in each photosensitive module to obtain a correction signal corresponding to each of the multiple photosensitive objects in the photosensitive module. Based on the correction signal, the captured image can be corrected to obtain a corrected target image with improved image quality.

[0382] The embodiments of this application can be divided into two correction scenarios: static correction and dynamic correction. In the static correction process, crosstalk information can be calculated for a specific image and then stored in an OTP or EEPROM. For subsequent images to be processed, the stored crosstalk information can be directly read from the OTP or EEPROM to correct the image. This method is suitable for situations where image accuracy requirements are low.

[0383] Furthermore, in the dynamic correction process, crosstalk information can be calculated in real time for the image to be processed, and then corrected based on this crosstalk information. Alternatively, crosstalk information can be calculated for each image separately, and corrections can be made based on the crosstalk information for each image individually. This approach is suitable for situations requiring high image accuracy.

[0384] The technical solution provided in this application will be described in detail below with reference to the specific accompanying drawings. Figure 24 This is a schematic diagram of the static correction process provided in an embodiment of this application. Figure 25 A schematic diagram illustrating the principle of the first model provided in this application embodiment. Figure 26 This is a schematic diagram of the dynamic correction process provided in the embodiments of this application.

[0385] The current section provides an illustrative description of the static calibration process, which may include:

[0386] S2401. Obtain crosstalk information corresponding to each of the multiple photosensitive modules from the first storage space.

[0387] Based on the above, the first storage space stores crosstalk information corresponding to each of the multiple photosensitive modules. In this embodiment, in order to correct images captured in a fixed scene, it is necessary to extract the crosstalk information corresponding to each of the multiple photosensitive modules from the first storage space to facilitate subsequent correction of the detection signal.

[0388] S2402, Extract the detection signal of the photosensitive object.

[0389] In this embodiment of the application, it is necessary to correct the detection signals obtained in the shooting scene. Therefore, it is necessary to extract the signals detected by each photosensitive object.

[0390] S2403. Obtain the corrected detection signal corresponding to the photosensitive object.

[0391] In this embodiment, crosstalk information is extracted based on the detected photosensitive object. However, since the non-photosensitive detected photosensitive object does not receive light signals, the detection signal of the detected photosensitive object is 0. In the image correction process, each photosensitive object corresponding to the image pixel needs to be corrected. Therefore, the detected photosensitive object needs to be corrected to obtain the detection signal corresponding to the non-photosensitive detected photosensitive object.

[0392] In one implementation, the detection signal of the non-photosensitive object can be obtained by fusing the surrounding photosensitive objects adjacent to it.

[0393] like Figure 20 As shown in (b), when the photosensitive object is SR1, the detection signal of SR1 can be obtained by using the average value of the detection signals of the eight photosensitive objects PS1 to PS8 adjacent to SR1.

[0394] For example, if the detection signals of PS1 to PS8 are all 1, then the average value of PS1 to PS8 is also 1, so the detection signal of SR1 can be obtained as 1.

[0395] It should be noted that the above provides a method for obtaining the signal value of a non-photosensitive object by using the average value of the detection signals of surrounding photosensitive objects. In practice, other methods can also be used to calculate the detection signal of a non-photosensitive object. The embodiments of this application do not limit the method for determining the detection signal of the photosensitive object.

[0396] S2404. Obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module.

[0397] In the embodiments of this application, there are two ways to determine the correction signal.

[0398] In one implementation, the crosstalk signals of each photosensitive object can be used to subtract the crosstalk signals from the detection signal to calculate the photosensitive object corrected signal.

[0399] Specifically, the correction signal for the photosensitive object can be obtained by subtracting the crosstalk signal from the detection signal of the photosensitive object. The crosstalk signal of the photosensitive object can be understood from the contents of S1904 and S1911, and will not be elaborated here.

[0400] It should be noted that the above method is a subtraction method to determine the correction signal. In practice, other calculation methods can also be used to determine the correction signal. This application does not limit the calculation method for removing crosstalk signals.

[0401] In another implementation, a model can be used to train and predict the corrected signal.

[0402] First, the crosstalk information corresponding to the photosensitive module, as well as the detection signals corresponding to each photosensitive object in the photosensitive module, are input into the first model. The first model can be understood as a machine learning model constructed from a neural network, which can be used to determine the correction signals for each photosensitive object.

[0403] Can be combined Figure 25 To understand the first model.

[0404] like Figure 25 As shown, crosstalk information from each photosensitive module and detection signals from each photosensitive object can be input into the first model, so that the first model outputs correction signals for the photosensitive objects used to correct image pixels. After determining the true correction signal and the predicted correction signal, the loss function value can be determined based on the true correction signal and the predicted correction signal.

[0405] In one possible implementation, a preset loss function can be used to process the true correction signal and the predicted correction signal to determine the loss function value. The specific implementation of the preset loss function can be selected and set according to actual needs, as long as the loss function value can reflect the difference between the true correction signal and the predicted correction signal.

[0406] After determining the loss function value, refer to Figure 25 This allows the model parameters of the first model to be updated based on the loss function value, thereby optimizing the first model and narrowing the gap between the predicted correction signal and the actual correction signal output by the first model, thus effectively ensuring the correctness of the predicted correction signal output by the first model.

[0407] It is understood that the model optimization objective in this embodiment is to make the predicted correction signals of each photosensitive object output by the first model as close as possible to the actual correction signals. More specifically, the optimization objective of the model parameters of the first model can be determined to be the minimum loss function value. Therefore, for example, the training of the first model can be considered complete when the loss function value is determined to be at its minimum, thus obtaining the trained first model.

[0408] The model training method for the correction signal provided in this embodiment determines the true correction signal based on the detection signal during the image capture process when no non-ideal factors are present. Since no non-ideal factors are present, the captured image will not exhibit noise or other problems, which can be understood as no crosstalk occurring.

[0409] After determining the true correction signal, the model parameters of the first model are updated based on the loss function value between the true correction signal and the predicted correction signal output by the first model. This effectively optimizes the first model, narrowing the gap between the true correction signal and the predicted correction signal, and ensuring that the predicted correction signal output by the trained first model is relatively close to the true correction signal.

[0410] The above describes the training process for the first model. After the first model is trained, the correction signal for each photosensitive object can be predicted based on the first model, and the image can be corrected accordingly.

[0411] Secondly, the first model outputs correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module.

[0412] Based on the above, once the first model is trained, correction signals for each photosensitive object can be output based on the first model. It can be understood that once the first model is trained, the output correction signal is close to the actual correction signal; therefore, the output correction signal can be used for subsequent operations.

[0413] Therefore, two different methods for determining the correction signal are provided, which can be flexibly selected according to different computational loads. When the crosstalk parameters contained in each photosensitive object are few, the subtraction method can be selected to determine the correction signal, thereby reducing the loss to the terminal equipment. When the crosstalk parameters contained in each photosensitive object are many, the computational load is large, and the model method can be selected to determine the correction signal, thereby improving computational efficiency.

[0414] S2405. Generate the target image based on the correction signal.

[0415] Based on the above, correction signals for each photosensitive object can be obtained. These correction signals are then used to correct the photosensitive objects corresponding to the pixels in the captured image, resulting in the target image. The target image can be understood as the final image presented to the user, which is free from grid noise.

[0416] The above describes the static correction process. The following illustrative description illustrates the dynamic correction process required for real-time image processing scenarios. This dynamic correction method may include:

[0417] S2601, Extract the detection signal of the photosensitive object.

[0418] For dynamic correction, image correction needs to be performed on each frame of the image during the shooting process. Therefore, it is necessary to extract the detection signal of the photosensitive object corresponding to the pixels of the captured image in each shooting process.

[0419] S2602, Set the photosensitive object to be detected.

[0420] Since dynamic correction is applied to images captured in complex and ever-changing environments, the crosstalk information stored in the first storage space cannot be used. Instead, crosstalk information is extracted and corrected for each frame of the image. Therefore, it is first necessary to set up a photosensitive object to extract crosstalk information. The process for setting up the photosensitive object can be found in the specific embodiments described above, and will not be repeated here.

[0421] S2603, Extract crosstalk information.

[0422] Based on the set photosensitive object, crosstalk information can be extracted. The method for calculating crosstalk information can be referred to in S1901 to S1907, or in S1908 to S1914, and will not be repeated here.

[0423] S2604. Obtain the corrected detection signal corresponding to the non-photosensitive photosensitive object.

[0424] When calibrating an image, it is necessary to calibrate the non-photosensitive detection objects. Refer to S2403 for details, which will not be repeated here.

[0425] S2605. Obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module.

[0426] When obtaining the correction signal, two methods can be used for calculation, which can be referred to in S2404 and will not be repeated here.

[0427] S2606. Generate the target image based on the correction signal.

[0428] Based on the correction signal, each frame of the image can be corrected to generate a target image with improved image quality.

[0429] In summary, static and dynamic corrections can be used to correct images captured in different scenarios. Static correction uses pre-stored crosstalk information in the first storage space to correct the image, reducing wear and tear on the terminal device. Dynamic correction calculates crosstalk information and performs correction for each frame, ensuring the quality of each frame. These methods avoid the problem of grid noise in the output image.

[0430] In the above process, crosstalk information is extracted by setting up a photosensitive object for detection. The detection signal of the photosensitive object is then compensated based on the crosstalk information to obtain a corrected signal. The image is then corrected based on the corrected signal to obtain the target image. The process design for the photosensitive object involves covering the photosensitive layer with metal to prevent signal reception, thus distinguishing it from other photosensitive objects. In practice, further processes can be added to any photosensitive object to achieve differentiation from other normal photosensitive objects. Photosensitive objects with added processes different from the photosensitive object for detection can be considered special photosensitive objects. Other processes are described below with reference to the accompanying drawings. Figure 27 This is a schematic diagram of the polarization layer process provided in an embodiment of this application. Figure 28 This is a schematic diagram of an image sensor using a color separator structure provided in an embodiment of this application. Figure 29 This is a schematic diagram of replacing a normal filter according to an embodiment of this application.

[0431] For specific photosensitive objects, one implementation incorporates nanostructures, such as titanium dioxide, between the filter layer and the photosensitive layer. This process ensures that the light incident on the photosensitive layer has a fixed polarization angle, such as 0 degrees, 45 degrees, 90 degrees, or 135 degrees. Combining this process with the techniques used to detect the photosensitive object not only extracts crosstalk information caused by non-ideal factors but also obtains more accurate polarization information, which helps achieve focusing and bokeh effects in the target image.

[0432] The following is combined with Figure 27 The design process for a special type of photosensitive object is introduced.

[0433] like Figure 27 As shown, suppose there exists an image sensor of model A, whose photosensitive unit is as follows: Figure 27 As shown in (a) in the figure. For each position in the photosensitive unit, a special photosensitive object can be set. Therefore, the positions for setting the special photosensitive object can include 16 types, specifically positions numbered 1, 2...15 and 16.

[0434] according to Figure 27 The photosensitive unit shown in (a) can be configured to hold the location of a special photosensitive object. All possible special photosensitive objects can be set in any photosensitive module, such as... Figure 27 As shown in (b) in the diagram. Figure 27 In (b) of the diagram, there exists a photosensitive module containing 16×16 photosensitive pixels. By setting 16 special photosensitive objects at different positions, more accurate polarization information can be obtained.

[0435] After completing the settings for the special photosensitive object, you can target... Figure 27Part 2701 in the middle, combined with Figure 27 (c) in the text can be used to understand the process of setting special photosensitive objects in an image sensor.

[0436] like Figure 27 As shown in (c), a side view of the image sensor with a special photosensitive object is presented. The special photosensitive object 2703 is a nanostructure incorporated below the filter layer 2702 and above the photosensitive layer 2704 to ensure that the light incident on the photosensitive layer has a fixed polarization angle.

[0437] In this application embodiment, the process used for detecting photosensitive objects is extended. In one implementation, the process used for detecting photosensitive objects can be applied to an image sensor using a dichroic structure. By setting the dichroic structure, the wavelength of the transmitted light can be different from the wavelength of the light transmitted by other normal structures. Using this dichroic structure, signals can still be detected, but the wavelengths of different signals are detected, thereby distinguishing them from other normal photosensitive objects.

[0438] The following is combined with Figure 28 An expanded description of the process used to detect photosensitive objects is provided.

[0439] like Figure 28 As shown, assuming there exists an image sensor using a color separator structure, the photosensitive unit of this image sensor is as follows: Figure 28 As shown in (a) in the figure. For each position in the photosensitive unit, a photosensitive object can be set. Therefore, the position for setting the photosensitive object can include 16 types, specifically positions numbered 1, 2...15 and 16.

[0440] according to Figure 28 The photosensitive unit shown in (a) can be configured to detect the location of the photosensitive object. All possible photosensitive objects can be configured in any type I photosensitive module, such as... Figure 28 As shown in (b) in the diagram. Figure 28 In (b) of the diagram, there exists a photosensitive module containing 16×16 photosensitive pixels. By setting 16 different locations for detecting photosensitive objects, it is possible to distinguish them from other normal photosensitive objects.

[0441] After completing the settings for detecting photosensitive objects, you can target... Figure 28 Part 2801 in the middle, combined with Figure 28 (c) in the text is used to understand the process method for setting up an image sensor to detect a photosensitive object.

[0442] like Figure 28As shown in (c), a side view of the image sensor after setting up the photosensitive object is presented. In this image sensor, a dichroic structure is used instead of an on-chip microlens structure; the dichroic structure can be understood as 2802. The special photosensitive object 2804 is located below the filter layer 2803 and above the photosensitive layer 2805.

[0443] like Figure 28 As shown in (d), a dichroic separator structure 2806 is used to replace the on-chip microlens and color filter structure. The dichroic separator can be understood as 2802. The special photosensitive object 2804 is located below the filter layer 2803 and above the photosensitive layer 2805.

[0444] It should be noted that the embodiments of this application illustrate a structure in which a dichroic separator replaces an on-chip microlens. In practice, a dichroic separator can also be used to replace a color filter, or a dichroic separator can be used to replace both a color filter and an on-chip microlens. In this process, the process used for detecting photosensitive objects is not employed; instead, it is achieved by introducing a new dichroic separator layer.

[0445] For specific photosensitive objects, filters can be replaced with narrowband or broadband filters. Narrowband filters allow these objects to absorb colors with a narrower spectrum, such as red, green, and blue, which have more concentrated wavelengths. Broadband filters allow them to absorb colors with a wider spectrum, such as cyan (the complementary color of red), magenta (the complementary color of green), and yellow (the complementary color of blue).

[0446] When the process used for this special photosensitive object is combined with the process used for detecting the photosensitive object, not only can the crosstalk information caused by the aforementioned non-ideal factors be extracted, but more accurate color information can also be obtained, which helps to achieve the color reproduction effect of the target image.

[0447] The following is combined with Figure 29 The process used for filters for special photosensitive objects is introduced.

[0448] like Figure 29 As shown, assuming there exists an image sensor of model A, the photosensitive unit of this image sensor is as follows: Figure 29 As shown in (a) in the figure. For each position in the photosensitive unit, a special photosensitive object can be set. Therefore, the positions for setting the special photosensitive object can include 16 types, specifically positions numbered 1, 2...15 and 16.

[0449] according to Figure 29 The photosensitive unit shown in (a) can be configured to hold the location of a special photosensitive object. All possible special photosensitive objects can be set in any photosensitive module, such as... Figure 29 As shown in (b) in the diagram. Figure 29 In (b) of the diagram, there exists a photosensitive module containing 16×16 photosensitive pixels. By setting 16 special photosensitive objects at different positions, it is possible to distinguish them from other normal photosensitive objects.

[0450] After completing the settings for the special photosensitive object, you can target... Figure 29 Part 2901 in the middle, combined with Figure 29 (c) in the text can be used to understand the process of setting special photosensitive objects in an image sensor.

[0451] like Figure 29 As shown in (c), a side view of the image sensor with a special photosensitive object is presented. In this view, the special photosensitive object 2902 uses a narrowband filter or a broadband filter instead of a standard color filter.

[0452] In summary, this application describes three processes: introducing nanostructures to form a polarization layer, applying the detected photosensitive object to an image sensor using a dichroic structure, and replacing normal filters with narrowband or broadband filters. Introducing nanostructures to form a polarization layer allows for the acquisition of more accurate polarization information, which helps achieve focusing and bokeh effects in the target image. Applying the detected photosensitive object to an image sensor using a dichroic structure allows for the detection of signals distinct from normal photosensitive objects. Combining the process of replacing normal filters with narrowband or broadband filters with the detected photosensitive object yields more accurate color information, contributing to better color reproduction of the target image.

[0453] Based on the above embodiments, the following is combined with... Figure 30 The image processing method provided in this application will be further explained. Figure 30 This is a schematic flowchart of the image processing method provided in an embodiment of this application.

[0454] S3001. Obtain crosstalk information corresponding to each of the multiple photosensitive modules. The crosstalk information includes crosstalk parameters corresponding to each of the multiple photosensitive objects in the photosensitive module. The crosstalk parameters are used to indicate the crosstalk signals of the surrounding photosensitive objects adjacent to the photosensitive object to the photosensitive object. The crosstalk parameters are determined according to the detected photosensitive object.

[0455] First, the embodiments of this application are applied to a terminal device, wherein the image sensor in the terminal device includes multiple photosensitive modules, and the photosensitive modules include multiple photosensitive objects; at least some of the multiple photosensitive modules are first type photosensitive modules, and the first type photosensitive modules include a detection photosensitive object, the detection photosensitive object having the opposite photosensitive status to the adjacent surrounding photosensitive objects, the photosensitive status being either photosensitive or not photosensitive.

[0456] For example, the image sensor, photosensitive module, first type of photosensitive module, and photosensitive object in the embodiments of this application can be, for example, the content described in the above embodiments, and will not be repeated here.

[0457] For any one of the multiple photosensitive modules, the photosensitive module includes N photosensitive units, the photosensitive unit includes M photosensitive sub-units, and the M photosensitive sub-units correspond to their respective photosensitive colors;

[0458] For example, such as Figure 12 As shown, for any photosensitive module, the photosensitive module includes N photosensitive units, where N is 16; each photosensitive unit includes M photosensitive subunits, where M is 4; the 4 photosensitive subunits correspond to the red, green, and blue photosensitive colors, respectively. The above can also be understood in a similar logical way by referring to the attached diagram showing the settings for detecting the photosensitive object.

[0459] The photosensitive subunit includes at least one photosensitive pixel, and the photosensitive unit includes a total of T photosensitive pixels, where N, M and T are all integers greater than or equal to 1.

[0460] For example, such as Figure 16 As shown, for any photosensitive module, it includes N photosensitive units, where N is 16; each photosensitive unit includes M photosensitive subunits, where M is 4; each photosensitive subunit contains one photosensitive pixel, and the photosensitive unit contains a total of T photosensitive pixels, where T is 4. Therefore, N, M, and T are all integers greater than or equal to 1. The above can also be understood using similar logic in conjunction with the attached diagram showing the settings for detecting the photosensitive object.

[0461] Wherein, the photosensitive object is either a photosensitive pixel or a photosensitive subunit. When the photosensitive object is a photosensitive pixel, the number of photosensitive objects detected in the first type of photosensitive module is T; or, when the photosensitive object is a photosensitive subunit, the number of photosensitive objects detected in the first type of photosensitive module is M.

[0462] For example, in the embodiments of this application, when the photosensitive object is a photosensitive pixel, the above content was analyzed in Case 1, and will not be repeated here. In the embodiments of this application, when the photosensitive object is a photosensitive sub-unit, the above content was analyzed in Case 2, and will not be repeated here.

[0463] Secondly, after setting the photosensitive object, the crosstalk information of each of the multiple modules is obtained based on the set photosensitive object. In one implementation, the crosstalk information of each of the multiple photosensitive modules can be obtained in the following way:

[0464] For any type I photosensitive module, obtain the crosstalk information corresponding to the type I photosensitive module.

[0465] Among the N photosensitive units of the first type of photosensitive module, at least some of the photosensitive units are special photosensitive units, and multiple photosensitive objects are distributed in each special photosensitive unit; and, in each special photosensitive unit, the photosensitive objects contained therein are located at different positions within the photosensitive unit.

[0466] For example, the special photosensitive unit can be understood based on the above content, and will not be repeated here.

[0467] This application implements the calculation of crosstalk information corresponding to the first type of photosensitive module in two cases.

[0468] When the detected photosensitive object is not photosensitive, for any surrounding photosensitive object adjacent to the detected photosensitive object, only one of the multiple photosensitive objects adjacent to the surrounding photosensitive object is the detected photosensitive object.

[0469] In one implementation, when the photosensitive object detected in the first type of photosensitive module is not photosensitive, the crosstalk information corresponding to the first type of photosensitive module is obtained, including:

[0470] For any first photosensitive object adjacent to the photosensitive object to be detected, a first detection signal of the first photosensitive object is acquired, and a second detection signal of the second photosensitive object is acquired. The position of the second photosensitive object in the photosensitive unit is the same as the position of the first photosensitive object in the photosensitive unit, and there is no photosensitive object to be detected among the multiple photosensitive objects adjacent to the second photosensitive object.

[0471] For example, the first photosensitive object and the first detection signal can be the content described in S1901, and the second photosensitive object and the second detection signal can be the content described in S1902, which will not be repeated here.

[0472] Based on the difference between the first detection signal and the second detection signal, the crosstalk signal of the first mapped photosensitive object to the second photosensitive object is determined. The positional relationship between the first mapped photosensitive object and the second photosensitive object is the same as the positional relationship between the detected photosensitive object and the first photosensitive object.

[0473] For example, the crosstalk signal between the first mapped photosensitive object and the second photosensitive object can be, for example, the content described in S1903, which will not be repeated here.

[0474] Based on the crosstalk signal of the first mapped photosensitive object to the second photosensitive object, the crosstalk signal of the photosensitive object located at the first position in each photosensitive unit to each of the adjacent surrounding photosensitive objects is determined, wherein the position of the first mapped photosensitive object in the photosensitive unit is equal to the first position.

[0475] For example, the crosstalk signal of the photosensitive object located at the first position in each photosensitive unit to each of the adjacent surrounding photosensitive objects is determined, for example, as described in S1904, which will not be repeated here.

[0476] When detecting light-sensitive objects in a first-type photosensitive module, the crosstalk information corresponding to the first-type photosensitive module is obtained, including:

[0477] In one implementation, for any photosensitive object to be detected, a third detection signal of the photosensitive object and a fourth detection signal of the second mapped photosensitive object are acquired. The photosensitive module to which the second mapped photosensitive object belongs is adjacent to the photosensitive module to which the photosensitive object belongs, and the position of the second mapped photosensitive object in its photosensitive module is the same as the position of the photosensitive object in its photosensitive module.

[0478] For example, the third detection signal may be the content described in S1908, and the fourth detection signal of the second mapped photosensitive object may be the content described in S1909, which will not be repeated here.

[0479] Based on the difference between the third and fourth detection signals, the crosstalk parameters corresponding to the detected photosensitive object are determined.

[0480] For example, the crosstalk parameters corresponding to the photosensitive object are determined, such as the content introduced in S1910, which will not be repeated here.

[0481] Based on the crosstalk parameters corresponding to the detected photosensitive object, the crosstalk parameters corresponding to each photosensitive module located at the second position in each photosensitive unit are determined to obtain the crosstalk information corresponding to the first type of photosensitive module. The second position is the position of the detected photosensitive object in its respective photosensitive unit.

[0482] For example, the crosstalk parameters corresponding to the photosensitive modules located in the second position within each photosensitive unit are determined. For example, the parameters described in S1911 can be used, which will not be repeated here.

[0483] Subsequently, based on the crosstalk signals of the photosensitive modules at multiple locations within each photosensitive unit to the adjacent surrounding photosensitive objects, the crosstalk parameters corresponding to each photosensitive object in the first type of photosensitive module are determined, so as to obtain the crosstalk information corresponding to the first type of photosensitive module.

[0484] Finally, based on the crosstalk information corresponding to each of the first-type photosensitive modules, the crosstalk information corresponding to the remaining photosensitive modules other than the first-type photosensitive modules is determined.

[0485] In one implementation, based on the crosstalk information corresponding to each of the first-type photosensitive modules, the crosstalk information corresponding to the remaining photosensitive modules (excluding the first-type photosensitive modules) among the multiple photosensitive modules is determined, including:

[0486] For any first-type photosensitive module, determine the crosstalk information of multiple photosensitive modules located around the first-type photosensitive module, so as to obtain the crosstalk information corresponding to each of the remaining photosensitive modules.

[0487] For example, the implementation process of the current step can be referred to the content described in S1905, and will not be repeated here.

[0488] The first storage space stores the crosstalk information corresponding to each of the multiple photosensitive modules;

[0489] For example, the implementation process of the current step can be referred to the content described in S1907, and will not be repeated here.

[0490] S3002. For any one of the multiple photosensitive modules, based on the crosstalk information corresponding to the photosensitive module, compensate the detection signals corresponding to each of the multiple photosensitive objects in the photosensitive module to obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module.

[0491] In one implementation, crosstalk information corresponding to each of the multiple photosensitive modules is obtained, including:

[0492] First, crosstalk information corresponding to each of the multiple photosensitive modules is obtained from the first storage space.

[0493] For example, the implementation process of the current step can be referred to the content described in S2401, and will not be repeated here.

[0494] Secondly, for any first-type photosensitive module, the detection signal of the non-photosensitive photosensitive object in the first-type photosensitive module is corrected to obtain the corrected detection signal corresponding to each non-photosensitive photosensitive object.

[0495] Specifically, the detection signals of non-photosensitive objects in the first type of photosensitive module are corrected to obtain corrected detection signals for each non-photosensitive object, including:

[0496] For any non-photosensitive object, the detection signals of the surrounding photosensitive objects adjacent to the non-photosensitive object are fused to obtain the corrected detection signal corresponding to the non-photosensitive object.

[0497] For example, the implementation process of the current step can be referred to the content described in S2403, and will not be repeated here.

[0498] Then, based on the crosstalk information corresponding to the photosensitive module, the detection signals corresponding to each of the multiple photosensitive objects in the photosensitive module are compensated to obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module, including:

[0499] For any photosensitive object in the photosensitive module, based on the crosstalk parameter corresponding to the photosensitive object, the crosstalk signal is removed from the detection signal corresponding to the photosensitive object to obtain the correction signal for the photosensitive object, including:

[0500] The crosstalk information corresponding to the photosensitive module and the detection signal corresponding to each photosensitive object in the photosensitive module are input to the first model so that the first model outputs the correction signal corresponding to each of the multiple photosensitive objects in the photosensitive module.

[0501] The first model is used to compensate for the detection signal of the photosensitive object based on the crosstalk signal of the photosensitive object.

[0502] For example, the implementation process of the current step can be referred to the content described in S2404, and will not be repeated here.

[0503] S3003. Generate a target image based on the correction signals corresponding to each of the multiple photosensitive objects.

[0504] For example, the implementation process of the current step can be referred to the content described in S2405, and will not be repeated here.

[0505] In summary, the technical solution proposed in this application can extract crosstalk information caused by non-ideal factors by setting a photosensitive object for detection, and correct the image based on the crosstalk information, thereby improving the image quality and avoiding the impact of grid noise and other problems on the image.

[0506] It should be noted that the module names involved in the embodiments of this application can all be defined as other names, as long as they can achieve the function of each module, and no specific restrictions are placed on the module names.

[0507] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0508] The image processing method of the present application embodiments has been described above. The apparatus for performing the above method provided in the present application embodiments is described below. Those skilled in the art will understand that the methods and apparatus can be combined with and referenced by each other, and the related apparatus provided in the present application embodiments can perform the steps in the above image processing method.

[0509] The image processing method provided in this application can be applied to electronic devices with image processing capabilities. Electronic devices include terminal devices, and the specific device form of the terminal device can be referred to the above-described related information, which will not be repeated here.

[0510] In one implementation, this application provides an electronic device. Figure 31 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0511] like Figure 31 As shown, the electronic device 310 includes: a processor 3101 and a memory 3102; the memory 3102 stores computer execution instructions; the processor 3101 executes the computer execution instructions stored in the memory 3102, causing the electronic device 310 to perform the above-described method.

[0512] When the memory 3102 is set up independently, the electronic device also includes a bus 3103 for connecting the memory 3102 and the processor 3101.

[0513] This application provides a chip. The chip includes a processor, which is used to call a computer program in memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those in the related embodiments described above, and will not be repeated here.

[0514] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0515] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0516] This application provides a computer program product, which includes a computer program that, when run, causes a computer to perform the above-described method.

[0517] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0518] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, The invention is applied to a terminal device, wherein the image sensor in the terminal device includes multiple photosensitive modules, and each photosensitive module includes multiple photosensitive objects; at least some of the multiple photosensitive modules are first-type photosensitive modules, and each first-type photosensitive module includes a detection photosensitive object, wherein the photosensitive nature of the detection photosensitive object is opposite to that of the adjacent surrounding photosensitive objects, and the photosensitive nature is either photosensitive or not photosensitive; The method includes: Obtain crosstalk information corresponding to each of the plurality of photosensitive modules. The crosstalk information includes crosstalk parameters corresponding to each of the plurality of photosensitive objects in the photosensitive module. The crosstalk parameters are used to indicate the crosstalk signals of the surrounding photosensitive objects adjacent to the photosensitive object to the photosensitive object. The crosstalk parameters are determined according to the detected photosensitive object. For any one of the plurality of photosensitive modules, the detection signals corresponding to each of the plurality of photosensitive objects in the photosensitive module are compensated according to the crosstalk information corresponding to the photosensitive module, so as to obtain the correction signals corresponding to each of the plurality of photosensitive objects in the photosensitive module. A target image is generated based on the correction signals corresponding to each of the multiple photosensitive objects.

2. The method according to claim 1, characterized in that, The step of obtaining the crosstalk information corresponding to each of the plurality of photosensitive modules includes: For any first-type photosensitive module, obtain the crosstalk information corresponding to the first-type photosensitive module; Based on the crosstalk information corresponding to each of the first type of photosensitive modules, the crosstalk information corresponding to the remaining photosensitive modules other than the first type of photosensitive modules is determined.

3. The method according to claim 2, characterized in that, For any one of the plurality of photosensitive modules, the photosensitive module includes N photosensitive units, the photosensitive unit includes M photosensitive sub-units, and the M photosensitive sub-units correspond to their respective photosensitive colors; The photosensitive subunit includes at least one photosensitive pixel, and the photosensitive unit includes a total of T photosensitive pixels, where N, M, and T are all integers greater than or equal to 1; Wherein, the photosensitive object is a photosensitive pixel, or the photosensitive object is a photosensitive subunit.

4. The method according to claim 3, characterized in that, When the photosensitive object is a photosensitive pixel, the number of photosensitive objects detected in the first type of photosensitive module is T; or, When the photosensitive object is a photosensitive subunit, the number of photosensitive objects detected in the first type of photosensitive module is M.

5. The method according to claim 4, characterized in that, In the N photosensitive units of the first type of photosensitive module, at least some of the photosensitive units are special photosensitive units, and the multiple photosensitive objects to be detected are distributed in each of the special photosensitive units; Furthermore, the location of the photosensitive object contained in each of the special photosensitive units is different within the photosensitive unit.

6. The method according to claim 5, characterized in that, When the detected photosensitive object is not photosensitive, for any surrounding photosensitive object adjacent to the detected photosensitive object, only one of the multiple photosensitive objects adjacent to the surrounding photosensitive object is the detected photosensitive object.

7. The method according to any one of claims 3-6, characterized in that, When the photosensitive object detected in the first type of photosensitive module is not photosensitive, the step of obtaining the crosstalk information corresponding to the first type of photosensitive module includes: For any of the detected photosensitive objects, based on the detection signals of each of the surrounding photosensitive objects adjacent to the detected photosensitive object, the crosstalk signals of the photosensitive module located at the first position in each photosensitive unit to each of the adjacent surrounding photosensitive objects are determined, where the first position is the position of the detected photosensitive object in the photosensitive unit. Based on the crosstalk signals of the photosensitive modules at multiple locations within each photosensitive unit to the adjacent surrounding photosensitive objects, the crosstalk parameters corresponding to each photosensitive object in the first type of photosensitive module are determined, so as to obtain the crosstalk information corresponding to the first type of photosensitive module.

8. The method according to claim 7, characterized in that, The step of determining the crosstalk signal of the photosensitive object located at the first position within each photosensitive unit to each of the adjacent surrounding photosensitive objects based on the detection signals of each of the surrounding photosensitive objects adjacent to the detected photosensitive object includes: For any first photosensitive object adjacent to the photosensitive object to be detected, a first detection signal of the first photosensitive object is acquired, and a second detection signal of the second photosensitive object is acquired. The position of the second photosensitive object in the photosensitive unit is the same as the position of the first photosensitive object in the photosensitive unit, and there is no photosensitive object among the multiple photosensitive objects adjacent to the second photosensitive object. Based on the difference between the first detection signal and the second detection signal, the crosstalk signal of the first mapped photosensitive object to the second photosensitive object is determined. The positional relationship between the first mapped photosensitive object and the second photosensitive object is the same as the positional relationship between the detected photosensitive object and the first photosensitive object. Based on the crosstalk signal of the first mapped photosensitive object to the second photosensitive object, the crosstalk signal of the photosensitive object located at the first position in each photosensitive unit to each of the adjacent surrounding photosensitive objects is determined, wherein the position of the first mapped photosensitive object in the photosensitive unit is equal to the first position.

9. The method according to any one of claims 3-6, characterized in that, When detecting light-sensitive objects in the first type of photosensitive module, the step of acquiring crosstalk information corresponding to the first type of photosensitive module includes: For any of the photosensitive objects to be detected, a third detection signal of the photosensitive object to be detected and a fourth detection signal of the second mapped photosensitive object to be detected are obtained. The photosensitive module to which the second mapped photosensitive object belongs is adjacent to the photosensitive module to which the photosensitive object to be detected belongs, and the position of the second mapped photosensitive object in its photosensitive module is the same as the position of the photosensitive object in its photosensitive module. Based on the difference between the third detection signal and the fourth detection signal, the crosstalk parameter corresponding to the detected photosensitive object is determined; Based on the crosstalk parameters corresponding to the detected photosensitive object, the crosstalk parameters corresponding to each photosensitive module located at the second position in each photosensitive unit are determined to obtain the crosstalk information corresponding to the first type of photosensitive module. The second position is the position of the detected photosensitive object in its respective photosensitive unit.

10. The method according to any one of claims 2-9, characterized in that, The step of determining the crosstalk information corresponding to each of the remaining photosensitive modules (excluding the first type of photosensitive modules) among the plurality of photosensitive modules based on the crosstalk information corresponding to each of the first type of photosensitive modules includes: For any one of the first type of photosensitive modules, determine the crosstalk information of multiple photosensitive modules located around the first type of photosensitive module, so as to obtain the crosstalk information corresponding to each of the remaining photosensitive modules.

11. The method according to any one of claims 2-10, characterized in that, The crosstalk information corresponding to each of the plurality of photosensitive modules is stored in the first storage space; The step of obtaining the crosstalk information corresponding to each of the plurality of photosensitive modules includes: Obtain the crosstalk information corresponding to each of the plurality of photosensitive modules from the first storage space.

12. The method according to any one of claims 1-10, characterized in that, The step of compensating the detection signals corresponding to each of the multiple photosensitive objects in the photosensitive module based on the crosstalk information corresponding to the photosensitive module, so as to obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module, includes: For any photosensitive object in the photosensitive module, the crosstalk signal contained in the detection signal corresponding to the photosensitive object is removed according to the crosstalk parameter corresponding to the photosensitive object to obtain the correction signal of the photosensitive object.

13. The method according to any one of claims 1-10, characterized in that, The step of compensating the detection signals corresponding to each of the multiple photosensitive objects in the photosensitive module based on the crosstalk information corresponding to the photosensitive module, so as to obtain the correction signals corresponding to each of the multiple photosensitive objects in the photosensitive module, includes: The crosstalk information corresponding to the photosensitive module and the detection signal corresponding to each photosensitive object in the photosensitive module are input to the first model so that the first model outputs the correction signal corresponding to each of the multiple photosensitive objects in the photosensitive module. The first model is used to compensate the detection signal of the photosensitive object based on the crosstalk signal of the photosensitive object.

14. The method according to claim 12 or 13, characterized in that, The method further includes: For any first-type photosensitive module, the detection signal of the non-photosensitive photosensitive object in the first-type photosensitive module is corrected to obtain the corrected detection signal corresponding to each of the non-photosensitive photosensitive objects.

15. The method according to claim 14, characterized in that, The step of correcting the detection signals of non-photosensitive objects in the first type of photosensitive module to obtain corrected detection signals for each of the non-photosensitive objects includes: For any non-photosensitive photosensitive object, the detection signals of the surrounding photosensitive objects adjacent to the non-photosensitive photosensitive object are fused to obtain the corrected detection signal corresponding to the non-photosensitive photosensitive object.

16. An electronic device, characterized in that, The electronic device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 15.

17. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 15.

19. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 15.

Images