A multispectral processing method and multispectral detection device

Abstract

The embodiment of the application provides a multispectral processing method and a multispectral detection device, relates to the chip technical field, and can support small-pixel, low-noise high-quality multispectral imaging in multispectral processing, realize high-resolution signal acquisition, and improve dark-light performance. The specific scheme is that in a plurality of spectral arrays, each pixel unit includes a PD, a voltage detection circuit and a current detection circuit, the voltage detection circuit includes a first switching device, and the current detection circuit includes a second switching device; a logic control circuit is used for controlling the first switching device and the second switching device in each pixel unit to perform imaging or ambient light detection in different working modes; and the logic control circuit is also used for controlling a row-column control circuit to adopt different signal merging modes for a plurality of pixel units in the plurality of spectral arrays, and perform imaging enhancement or ambient light detection enhancement. The embodiment of the application is used for multispectral imaging and multispectral ambient light detection.

Description

Technical Field

[0001] This application relates to the field of chip technology, and in particular to a multispectral processing method and a multispectral detection device. Background Technology

[0002] Multispectral technology is increasingly being applied to ambient light detection and material identification, providing richer information for accurate color reproduction. Currently, most multispectral detectors utilize standard imaging photon detector (IPD) structures for photoelectric conversion to generate current, which is then quantized by an analog-to-digital converter (ADC) to obtain spectral information. In this approach, the Si surface of each IPD is coated with an interference filter, and the number and thickness of the film layers are designed and fabricated to suit the desired bandpass spectrum, forming the final multispectral detector.

[0003] However, for the chip housing a multispectral sensor, the total chip area is fixed, and each pixel, including the IPD structure, requires a significant area. Therefore, the number of pixels is limited, resulting in lower image resolution and making it impossible to achieve high-resolution signal acquisition under the target surface acceptable to chips like those used in mobile phones. Furthermore, the high gain of the ADC is designed for low-light environments. In low-light conditions, the ADC that acquires the current and the aforementioned photoelectric conversion process lack noise suppression methods (such as correlated double sampling), leading to poor noise levels due to factors like dark current. This results in poor low-light performance, making it impossible to achieve effective brightness and spectral information detection and high-quality imaging in ultra-night environments. Summary of the Invention

[0004] This application provides a multispectral processing method and a multispectral detection device, which can support high-quality multispectral imaging with small pixels and low noise in multispectral processing, realize high-resolution signal acquisition and improve low-light performance.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions.

[0006] In a first aspect, a multispectral detection device is provided, comprising a logic control circuit, multiple spectrum arrays, a row and column control circuit, multiple analog-to-digital converters (ADCs) for acquiring current (integrating ADC2) and multiple ADCs for acquiring voltage (step ADC1); each spectrum array includes multiple single-channel spectral units, and each single-channel spectral unit includes multiple pixel units; for each pixel unit, each pixel unit is coupled to the row and column control circuit, one of the current-acquiring ADCs and one of the voltage-acquiring ADCs; each pixel unit includes a photodetector (PD), a voltage detection circuit and a current detection circuit, the voltage detection circuit including a first switching device and the current detection circuit including a second switching device; the logic control circuit is used to control the first and second switching devices in each pixel unit to perform imaging or ambient light detection in different operating modes; the logic control circuit is also used to control the row and column control circuit to use different signal combining methods for the multiple pixel units in the multiple spectrum arrays to perform image enhancement or ambient light detection enhancement.

[0007] Therefore, in this application, a newly designed circuit structure for a pixel unit allows the operating mode of the pixel unit to be switched as needed. Specifically, the operating mode of each pixel unit for imaging or ambient light detection can be determined by the on / off state of the first and second switching devices. Based on this, this application combines pixel units in multiple frequency arrays using row and column control circuits, achieving ambient light detection enhancement or imaging enhancement under various combining methods. Thus, by reading out the new pixel units through multiple combining methods in multiple frequency arrays, high-resolution, high-quality imaging and high-dynamic ambient light detection can be achieved. This supports high-quality multispectral imaging with small pixels and low noise in multispectral processing, enabling high-resolution signal acquisition and improved low-light performance.

[0008] In one possible design, a logic control circuit is used to control the first switching device in each pixel unit to be turned on and the second switching device to be turned off, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted to the ADC of the acquisition voltage through the first switching device for imaging; or, the logic control circuit is used to control the first switching device in each pixel unit to be turned off and the second switching device to be turned on, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted to the ADC of the acquisition current through the second switching device for ambient light detection. That is to say, when performing multispectral imaging, the photoelectric converted current signal can be combined and input into the ADC of the acquisition voltage by turning on the first switching device in the voltage detection circuit; when performing multispectral ambient light detection, the photoelectric converted current signal can be combined and input into the ADC of the acquisition current by turning on the second switching device in the current detection circuit, so as to achieve high-quality imaging of multiple spectra and high dynamic ambient light detection.

[0009] In one possible design, the current sensing circuit in each pixel unit further includes a current mirror, and the voltage sensing circuit in each pixel unit further includes a reset transistor, a source follower transistor, and a row select transistor. For each pixel unit, the output terminal of the PD is coupled to the first terminal of the first switching device and the first terminal of the second switching device. The second terminal of the first switching device is coupled to the first terminal of the reset transistor and the first terminal of the source follower transistor. The second terminal of the source follower transistor is coupled to the first terminal of the row select transistor. The second terminal of the row select transistor is coupled to the first input terminal of the row and column control circuit. The first output terminal of the row and column control circuit is coupled to the ADC of the acquired voltage, and the second output terminal of the row and column control circuit is coupled to the ADC of the acquired current. The second terminal of the second switching device is coupled to the first terminal of the current mirror, and the first output terminal of the current mirror is coupled to the second input terminal of the row and column control circuit. For each pixel unit, by switching between the current sensing circuit and the voltage sensing circuit, multiple purposes such as multispectral imaging or ambient light detection can be achieved.

[0010] In one possible design, the logic control circuit is used to: turn off the first switching device and turn on the second switching device in each pixel unit when ambient light intensity detection is determined; and control the row and column control circuit to combine all single-channel spectral units in each spectral array as a group, with the combined signal of one group used to indicate the ambient light intensity of a spectral array region. Thus, with the ambient light intensity detection method provided in this application, for each pixel unit, the part of the circuit coupled to the first switching device (including capacitor C) is not working, avoiding saturation problems caused by signal clamping, making it suitable for high dynamic range light sensing detection. Furthermore, this application can also be used to detect ambient light of different intensities by using different current summing methods.

[0011] In one possible design, the logic control circuit is used to: turn off a first switching device and turn on a second switching device in each pixel unit when ambient light intensity detection is determined; and control the row and column control circuit to divide multiple spectral arrays into regions, combining the single-channel spectral units in all spectral arrays of each region as a group, with the combined signal of one group used to indicate the ambient light intensity of a spectral array region. Thus, by further dividing and combining multiple spectral arrays, noise reduction or increased light intensity detection sensitivity can be achieved.

[0012] In one possible design, the logic control circuit is used to: control the first switching device in each pixel unit to turn off and the second switching device to turn on when ambient light type detection is determined; control the row and column control circuit to divide multiple spectral arrays into regions, and combine the same single-channel spectral units in all spectral arrays in each region as a group to obtain a combined signal for each type of single-channel spectral unit in each region. The combined signal of one group in each region is used to indicate an ambient light type. Therefore, the ambient light type detection method for ultra-high dynamic range scenes provided in this application, by partitioning multiple spectral arrays and utilizing flexible signal combining methods, can be applied to different ambient light brightness scenes, achieving a balance between detection accuracy and resolution for ambient light types, without causing saturation problems due to signal clamping, and is suitable for high dynamic range light sensing detection. This multi-region high dynamic multi-channel spectral information can be used to assist other cameras in performing regional ambient light type detection and regional AWB parameter adaptation, assisting in more accurate color reproduction for single or mixed color temperature scenes.

[0013] In one possible design, the logic control circuit is used to: control a first switching device in each pixel unit to be turned on and a second switching device to be turned off when multispectral imaging is determined; control the row and column control circuit to combine multiple pixel units of each single-channel spectral unit of each spectrum array as a group to obtain a combined signal of each single-channel spectral unit of each spectrum array, and the combined signal of each single-channel spectral unit of each spectrum array is used for multispectral imaging.

[0014] Therefore, the multispectral imaging method provided in this application can be used for hyperspectral imaging. By utilizing high-resolution, high-dynamic-range multispectral information, it can achieve color reproduction and image quality with a wider color gamut. Compared to existing solutions, which cannot effectively and flexibly balance multispectral and high-dynamic-range effects, this application can achieve high-resolution hyperspectral imaging, balancing resolution and color accuracy, achieving low noise, and is suitable for high-quality image reproduction and night scene enhancement.

[0015] In one possible design, the logic control circuit is used to: control the first switching device in each pixel unit to be turned on and the second switching device to be turned off when multispectral imaging is determined; control the row and column control circuit to group multiple single-channel spectral units in each of the multiple spectral arrays, with each group including multiple adjacent single-channel spectral units; control the row and column control circuit to merge the signals of each group in each spectral array to obtain a merged signal for each group in each spectral array, and the merged signal for each group in each spectral array is used for multispectral imaging. This method of merging adjacent spectral channels, although sacrificing some color gamut space, can achieve improvements in color and dynamic range during multispectral imaging.

[0016] In one possible design, signal combining includes: charge combining, analog domain combining, or digital domain combining. However, it is not limited to these three combining methods.

[0017] In one possible design, the logic control circuit is further configured to: reduce the exposure and / or gain of each pixel unit when acquiring the signal of each pixel unit in the next frame image if the ambient light intensity of the previous frame image is greater than or equal to a preset threshold; and increase the exposure and / or gain of each pixel unit when acquiring the signal of each pixel unit in the next frame image if the ambient light intensity of the previous frame image is less than the preset threshold. In this way, the readout method of each pixel unit at the lowest level can dynamically adapt to low or high gain based on the brightness detection information of the previous frame, further meeting the requirements of the dynamic range of light intensity detection.

[0018] Secondly, a multispectral processing method is provided, applied to a multispectral detection device. The multispectral detection device includes a logic control circuit, multiple spectrum arrays, row and column control circuits, multiple analog-to-digital converters (ADCs) for acquiring current, and multiple ADCs for acquiring voltage. Each spectrum array includes multiple single-channel spectral units, and each single-channel spectral unit includes multiple pixel units. For each pixel unit, each pixel unit is coupled to the row and column control circuit, one of the current-acquiring ADCs, and one of the voltage-acquiring ADCs. Each pixel unit includes a photodetector (PD), a voltage detection circuit, and a current detection circuit. The voltage detection circuit includes a first switching device, and the current detection circuit includes a second switching device. The method includes: the multispectral detection device control logic control circuit using the first and second switching devices in each pixel unit to perform imaging or ambient light detection in different operating modes; and the multispectral detection device control logic control circuit using different signal combining methods for the multiple pixel units in the multiple spectrum arrays to perform image enhancement or ambient light detection enhancement.

[0019] In one possible design, the control logic circuit for the multispectral detection device, used for imaging or ambient light detection of the first and second switching devices in each pixel unit under different operating modes, includes: the control logic circuit for the multispectral detection device, where the first switching device in each pixel unit is turned on and the second switching device is turned off, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted through the first switching device to the ADC of the acquisition voltage for imaging; or, the control logic circuit for the multispectral detection device, where the first switching device in each pixel unit is turned off and the second switching device is turned on, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted through the second switching device to the ADC of the acquisition current for ambient light detection.

[0020] In one possible design, the current sensing circuit in each pixel unit further includes a current mirror, and the voltage sensing circuit in each pixel unit further includes a reset transistor, a source follower transistor, and a row select transistor. For each pixel unit, the output terminal of the PD is coupled to the first terminal of the first switching device and the first terminal of the second switching device. The second terminal of the first switching device is coupled to the first terminal of the reset transistor and the first terminal of the source follower transistor. The second terminal of the source follower transistor is coupled to the first terminal of the row select transistor. The second terminal of the row select transistor is coupled to the first input terminal of the row and column control circuit. The first output terminal of the row and column control circuit is coupled to the ADC of the acquired voltage, and the second output terminal of the row and column control circuit is coupled to the ADC of the acquired current. The second terminal of the second switching device is coupled to the first terminal of the current mirror, and the first output terminal of the current mirror is coupled to the second input terminal of the row and column control circuit.

[0021] In one possible design, the control logic circuit of the multispectral detection device uses different signal combining methods for multiple pixel units in multiple spectrum arrays to perform imaging enhancement or ambient light detection enhancement. This includes: when determining to perform ambient light intensity detection, controlling the first switching device in each pixel unit to turn off and the second switching device to turn on; and controlling the row and column control circuit to combine all single-channel spectral units in each spectrum array as a group for signal combining, with the signal after combining a group used to indicate the ambient light intensity of a spectrum array region.

[0022] In one possible design, the multispectral detection device controls the logic control circuit to use different signal combining methods for multiple pixel units in multiple spectrum arrays to perform imaging enhancement or ambient light detection enhancement. This includes: when determining to perform ambient light intensity detection, controlling the first switching device in each pixel unit to turn off and the second switching device to turn on; controlling the row and column control circuit to divide the multiple spectrum arrays into regions, and combining the single-channel spectral units in all spectrum arrays in each region as a group. The signal after combining the signals of a group is used to indicate the ambient light intensity of a spectrum array region.

[0023] In one possible design, the control logic circuit of the multispectral detection device uses different signal combining methods for multiple pixel units in multiple spectrum arrays to perform imaging enhancement or ambient light detection enhancement. This includes: when determining that ambient light type detection is to be performed, controlling the first switching device in each pixel unit to turn off and the second switching device to turn on; controlling the row and column control circuit to divide the multiple spectrum arrays into regions, and combining the same single-channel spectral units in all spectrum arrays in each region as a group to obtain the combined signal of each single-channel spectral unit in each region. The signal after combining the signals of one group in each region is used to indicate an ambient light type.

[0024] In one possible design, the control logic circuit of the multispectral detection device uses different signal combining methods for multiple pixel units in multiple spectrum arrays to perform image enhancement or ambient light detection enhancement. This includes: when multispectral imaging is determined to be performed, controlling the first switching device in each pixel unit to be turned on and the second switching device to be turned off; controlling the row and column control circuit to combine the multiple pixel units of each single-channel spectral unit of each spectrum array as a group to obtain the combined signal of each single-channel spectral unit of each spectrum array, and the combined signal of each single-channel spectral unit of each spectrum array is used for multispectral imaging.

[0025] In one possible design, the control logic circuit of the multispectral detection device uses different signal combining methods for multiple pixel units in multiple spectrum arrays to perform image enhancement or ambient light detection enhancement. This includes: when multispectral imaging is determined to be performed, controlling the first switching device in each pixel unit to be turned on and the second switching device to be turned off; controlling the row and column control circuit to group multiple single-channel spectral units in each spectrum array, with each group including multiple adjacent single-channel spectral units; controlling the row and column control circuit to combine the signals of each group in each spectrum array to obtain a combined signal of each group in each spectrum array, and the combined signal of each group in each spectrum array is used for multispectral imaging.

[0026] In one possible design, signal combining includes: charge combining, or analog domain combining, or digital domain combining.

[0027] The method further includes: when the ambient light intensity of the previous frame image is greater than or equal to a preset threshold, reducing the exposure and / or gain of each pixel unit when acquiring the signal of each pixel unit in the next frame image; and when the ambient light intensity of the previous frame image is less than a preset threshold, increasing the exposure and / or gain of each pixel unit when acquiring the signal of each pixel unit in the next frame image.

[0028] Thirdly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the multispectral processing method described in the second aspect and any possible implementation thereof.

[0029] Fourthly, embodiments of this application provide a computer program product that, when run on a computer or processor, causes the computer or processor to execute the multispectral processing method in the second aspect and any possible implementation thereof.

[0030] Fifthly, a chip is provided, including a multispectral detection device as described in the first aspect and any possible implementation thereof.

[0031] It is understood that any of the multispectral detection devices, chips, computer-readable storage media or computer program products provided above can be applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0032] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0033] Figure 1 This application provides a schematic diagram of a system architecture for capturing images.

[0034] Figure 2 This application provides a schematic diagram of the circuit structure of a multispectral detector portion in an image sensor.

[0035] Figure 3 This is a schematic diagram of a partial circuit structure of a multispectral detector provided in an embodiment of this application;

[0036] Figure 4 This is a schematic diagram of the structure of a multispectral detection device provided in an embodiment of this application;

[0037] Figure 5This is a schematic diagram of the structure of a multispectral detection device provided in an embodiment of this application;

[0038] Figure 6 This is a schematic diagram of the structure of a multispectral detection device provided in an embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the structure of each pixel unit in a multispectral detection device provided in an embodiment of this application;

[0040] Figure 8 This is a schematic flowchart of an ambient light intensity detection method provided in an embodiment of this application;

[0041] Figure 9 This application provides a schematic diagram illustrating the process of merging single-channel spectral units in each spectral array into an all-pass channel.

[0042] Figure 10 This is a schematic flowchart of an ambient light type detection method provided in an embodiment of this application;

[0043] Figure 11 This application provides a schematic diagram of merging single-channel spectral units in each spectral array.

[0044] Figure 12 This application provides a schematic diagram of the merging of identical single-channel spectral units in an embodiment.

[0045] Figure 13 This is a schematic diagram of a multispectral imaging method provided in an embodiment of this application;

[0046] Figure 14 This is a schematic diagram illustrating the merging of pixel units in a spectrum array, provided as an embodiment of this application.

[0047] Figure 15 A schematic diagram illustrating signal combining of different single-channel spectral units in a spectral array, provided as an embodiment of this application;

[0048] Figure 16 This is a schematic diagram of a multispectral detection device architecture provided in an embodiment of this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0050] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0051] When taking pictures, such as Figure 1 As shown, the scene can be projected onto the surface of the image sensor through the optical image generated by the lens device (Lens), and then converted into an electrical signal. After being amplified by the pre-amplifier circuit, the digital image signal converted by the automatic chroma control (ACC) and analog to digital (A / D) converter is transmitted to the digital signal processing chip for processing. Then, it is transmitted to the central processing unit (CPU) for processing through the input / output (I / O) interface, and finally the image is output to the display screen.

[0052] During image capture, white balance is required. White balance refers to a camera's ability to reproduce white objects under different lighting conditions. The physical image captured by the camera should accurately reflect the color of the object as perceived by the human eye under the same lighting conditions. Therefore, ambient light detection is necessary during white balance to ensure accurate color reproduction in the image.

[0053] Currently, multispectral technology is increasingly being applied to ambient light detection. Furthermore, it can be used in processes such as material identification, providing richer information for accurate color reproduction. For example, multispectral technology is gradually becoming a mainstream solution in fields such as scientific material detection equipment, skin texture detection equipment in the beauty industry, spectral testing equipment in industry, and mobile phone cameras.

[0054] For example, in some multispectral technologies, ambient light detection can be achieved based on IPD and ADC of acquisition current, with each photosensitive channel using interferometric filtering to achieve specific spectrum selection. Figure 2 The diagram shows a portion of the circuitry for an image sensor. This portion can be understood as the core circuitry of a multispectral detector. It may also include peripheral logic control circuitry. The PD in the diagram responds to the received transmitted light signal, essentially projecting the selected spectrum onto the IPD. The IPD responds to the light signal, performing photoelectric conversion to obtain a current signal, which is then transmitted to this part of the circuit. When the current signal reaches the ADC (Analog and Diode Array) for current acquisition, the ADC converts the current signal into a digital signal, which is then transmitted to a digital signal processing chip for processing. In an image sensor, each pixel position corresponds to one IPD. Figure 2 The circuit shown, excluding the IPD, can be considered as an ADC circuit for acquiring current. The multispectral detector includes multiple IPDs and multiple ADC circuits for acquiring current. Each IPD is coupled to an ADC circuit for acquiring current, and an ADC circuit for acquiring current can be coupled to multiple IPDs.

[0055] In this process, the silicon (Si) surface of each IPD can be coated with an interference filter. The number and thickness of the film system are designed and manufactured according to the designed bandpass spectrum to form the final multispectral detector.

[0056] However, for a single pixel, the smallest photosensitive unit, it requires a large area, and in low-light scenes, the high gain noise level of the ADC is poor. Therefore, it cannot achieve high-resolution signal acquisition under the target surface acceptable to architectures such as mobile phones. At the same time, its low-light performance is poor, and it cannot achieve effective brightness and spectral information detection in ultra-night environments.

[0057] In other multispectral techniques, such as Figure 3 As shown, multispectral detectors can achieve high-resolution, low-noise photoelectric conversion and signal quantization based on pin-photodiodes, 4T readout structures, and ramp-type ADCs. However, Figure 3 The dynamic range of the multispectral detector shown is limited by the FD capacitor. Even with a multi-capacitor switching design, it is impossible to achieve unsaturated high dynamic capability under sunlight within a limited area.

[0058] Dynamic range can be understood as the largest tonal range between the darkest and brightest areas that a camera can capture, representing the camera's ability to record grayscale levels. The larger the dynamic range, the richer the levels captured.

[0059] Therefore, this application proposes a multispectral detection device. The circuit structure of a single pixel and the circuit structure around the pixel are different from those of the existing ones. The direction of the current signal after photoelectric conversion can be realized by switching control. It supports high-quality multispectral imaging with small pixels and low noise, as well as detection of ambient light source type and ambient light intensity under unsaturated ultra-high dynamic range under sunlight. At the same time, it expands the detection capability of dark states.

[0060] Secondly, this application provides a higher resolution multispectral device that is also multifunctional, capable of imaging, light source type detection, and light source intensity detection. Specifically, this is achieved through a novel pixel structure combined with a novel readout method. In other words, the novel pixel combination readout method proposed in this application can achieve multiple functions, such as high-resolution multispectral high dynamic range imaging, enabling better color effects, safety, and health detection during imaging; high dynamic range multi-region ambient light intensity detection, which can be used to assist in upgrading the effects of high dynamic range (HDR) images; and high dynamic range multi-region ambient light type detection, which is used to assist in improving the accuracy and effect of zoned color reproduction. This allows for an upgrade of existing multispectral devices, serving as an independent multifunctional camera, bringing benefits to system-level high dynamic range and accurate color, while maintaining controllable costs.

[0061] The multispectral detection device provided in this application can be applied to high-resolution, multi-functional hyperspectral detectors, specifically in various application scenarios. For example, when applied to multispectral cameras in mobile phone applications, it can achieve high-resolution multispectral imaging and improve color accuracy; it can flexibly configure multi-zone ambient light type detection according to the scene, assisting in achieving fine-grained automatic white balance (AWB) and accurate color reproduction in mixed color temperature scenes; it can also flexibly configure multi-zone ambient light intensity detection according to the scene, assisting in achieving more accurate automatic exposure (AE) control and more accurate high dynamic range (HDR) configuration in high dynamic range scenes. Furthermore, it can also be used on mobile phones for multispectral-based skin texture detection and facial anti-spoofing detection.

[0062] For example, it can also be applied to multispectral cameras used in mobile phones, watches, and accessories to achieve health monitoring such as substance composition detection, blood glucose, blood oxygen, and blood pressure.

[0063] For example, it can also be applied to smart homes and vehicles to achieve liveness detection and provide the information needed for security scenarios.

[0064] The multispectral detection device provided in this application can be applied to photosensitive sensor devices such as image sensors.

[0065] The multispectral detection device provided in this application is described below.

[0066] like Figure 4 The image shows a multispectral detection device 40 provided in this application. The multispectral detection device includes a logic control circuit, multiple spectrum arrays, a row and column control circuit (SWAP SEL), multiple ADCs for acquiring current, and multiple ADCs for acquiring voltage.

[0067] The ADC for acquiring current in this application can be, for example, an integrating ADC or other current-acquiring ADCs; the ADC for acquiring voltage in this application can be, for example, a step ADC or other voltage-acquiring ADCs.

[0068] Each spectral array comprises multiple single-channel spectral units, and each single-channel spectral unit comprises multiple pixel units.

[0069] In this application, each spectral array can be understood as the smallest unit for detecting ambient light type and the smallest unit for detecting ambient light intensity.

[0070] like Figure 5 As shown, C 11 C 12 C 13 ..., C mn The array can be understood as a spectrum array, in which a single C 11 Or C 12 ... or C mn Each C represents a single-channel spectral unit, or a spectral channel, where m and n are integers greater than or equal to 1. 11 Or C 12 ... or C mn Pixel units including multiple identical spectral channels: (1,1), (1,2), ..., (h,k), where (h,k) represents a minimum pixel unit and indicates the position information of the pixel unit.

[0071] ADC group 1 represents multiple ADCs that acquire voltages, and ADC group 2 represents multiple digital-to-analog converters (ADCs) that acquire currents.

[0072] like Figure 6 As shown, for each of the multiple pixel units, each pixel unit is coupled to a row and column control circuit, a current-acquiring ADC (ADC2) among the multiple current-acquiring ADCs, and a voltage-acquiring ADC (ADC1) among the multiple voltage-acquiring ADCs.

[0073] Each pixel unit includes a photodetector PD, a voltage detection circuit, and a current detection circuit. The voltage detection circuit includes a first switching device TX1, and the current detection circuit includes a second switching device TX2.

[0074] A logic control circuit is used to control the first switching device TX1 and the second switching device TX2 in each pixel unit to perform imaging or ambient light detection in different working modes.

[0075] The logic control circuit is also used to control the row and column control circuit SWAP SEL to use different signal combining methods for multiple pixel units in multiple spectrum arrays to perform image enhancement or ambient light detection enhancement.

[0076] Therefore, in this application, a new circuit structure for a pixel unit is designed so that the operating mode of the pixel unit can be switched as needed. Specifically, the operating mode of each pixel unit for imaging or ambient light detection can be determined by turning on and off the first switching device TX1 and the second switching device TX2.

[0077] Building upon this foundation, this application combines pixel units in multiple frequency arrays using row and column control circuits to enhance ambient light detection or imaging under various combining methods. Thus, by reading out new pixel units through multiple combining methods in multiple frequency arrays, high-resolution, high-quality imaging and high-dynamic ambient light detection can be achieved. Specifically, it supports high-quality multispectral imaging with small pixels and low noise in multispectral processing, enabling high-resolution signal acquisition and improved low-light performance.

[0078] In some embodiments, the logic control circuit is used to control the first switching device TX1 in each pixel unit to be turned on and the second switching device TX2 to be turned off, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted to the ADC (ADC1) of the acquisition voltage through the first switching device TX1 for imaging; or, the logic control circuit is used to control the first switching device TX1 in each pixel unit to be turned off and the second switching device TX2 to be turned on, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted to the ADC (ADC2) of the acquisition current through the second switching device TX2 for ambient light detection.

[0079] In other words, this application can improve the high-resolution signal acquisition under the target surface of mobile phones and other architectures by arranging multiple spectrum arrays. Moreover, the circuit structure of the single pixel unit provided in this application is different from the existing single pixel circuit structure. The current signal after photoelectric conversion of the PD in the single pixel unit provided in this application can be transmitted to ADC1 through the first switching device TX1 or to ADC2 through the second switching device TX2, unlike the prior art where the current signal of the circuit structure of a single pixel unit can only be transmitted through one type of ADC.

[0080] Furthermore, this application can combine signals from single-channel spectral units in multiple spectral arrays according to a preset grouping type. In other words, by combining the novel circuit structure of the single pixel unit provided in this application, multiple grouping types based on single-channel spectral units can be obtained. This enables the detection of ambient light type or intensity under multiple grouping types, and also enables the detection of ambient light with ultra-high dynamic range. This solves the existing problems of not being able to perform high-resolution imaging and the problem of detecting ambient light with ultra-high dynamic range such as sunlight.

[0081] In some embodiments, Figure 6 Based on the circuit structure of the pixel unit shown, in this application, as Figure 7 As shown, the current sensing circuit in each pixel unit also includes a current mirror M, and the voltage sensing circuit in each pixel unit also includes a reset transistor RST, a source follower transistor SF, a capacitor C, and a row select transistor SEL1.

[0082] For each pixel unit, the output terminal a of PD is coupled to the first terminal b of the first switching device TX1 and the first terminal c of the second switching device TX2. The second terminal d of the first switching device TX1 is coupled to the first terminal e of the reset transistor RST and the first terminal f of the source follower transistor SF. The second terminal g of the source follower transistor SF is coupled to the first terminal h of the row select transistor SEL1. The second terminal i of the row select transistor SEL1 is coupled to the first input terminal j of the row and column control circuit (SWAP SEL). The first output terminal k of the row and column control circuit (SWAP SEL) is coupled to the ADC (ADC1) for acquiring voltage. The second output terminal l of the row and column control circuit (SWAP SEL) is coupled to the ADC (ADC2) for acquiring current. The second terminal m of the second switching device TX2 is coupled to the first terminal n of the current mirror M. The first output terminal o of the current mirror M is coupled to the second input terminal p of the row and column control circuit (SWAP SEL). A capacitor C is coupled between the input terminal of PD and the second terminal d of the first switching device TX1.

[0083] It should be noted that, Figure 7The two row and column control circuits shown are actually the same SWAP SEL, that is, the same SWAP SEL is coupled to at least one ADC1 and to at least one ADC2. Figure 7 In this context, bitLine refers to the column signal line connected to the second terminal g of the source follower transistor SF, and iLine refers to the column signal line connected to the o terminal of the current mirror M.

[0084] In other words, according to Figure 7 The circuit structure for each pixel unit includes, in addition to PD, first switching device TX1 and second switching device TX2, a current mirror M, a reset transistor RST, a source follower transistor SF, a capacitor C, and a row select transistor SEL1. The number of row and column control circuits (SWAP SELs) can also be at least one, with each pixel unit coupled to one of the at least one row and column control circuits (SWAP SELs). Each of the at least one row and column control circuits (SWAP SELs) is coupled to one of a plurality of ADC1 and to one of a plurality of ADC2.

[0085] For a single pixel unit, the logic control circuit can control the current signal of the PD in the pixel unit after photoelectric conversion to flow through the first switching device TX1 or the second switching device TX2. When the current signal flows through the first switching device TX1, the current signal can be transmitted to the source follower transistor SF through TX1. The source follower transistor SF can output the received current signal to the row selection transistor SEL1 without loss. The row selection transistor SEL1 is used to determine the pixel unit of a certain row in the spectrum array according to the instruction of the logic control circuit. When selected, the current signal of the pixel unit in that row can be output to the row and column control circuit (SWAP SEL). The row and column control circuit (SWAP SEL) is used to perform signal combining of single-channel spectrum units in the spectrum array according to the instruction of the logic control circuit and according to a preset combination type.

[0086] When the current signal flows through the second switching device TX2, the current signal can be transmitted to the row and column control circuit (SWAP SEL) through the current mirror M, so that the row and column control circuit (SWAP SEL) can combine the signals of the single-channel spectrum unit according to the preset combination type according to the instruction of the logic control circuit.

[0087] Based on the multispectral detection device provided in this application, the following sections describe its applications in three scenarios: ambient light intensity detection in ultra-high dynamic range scenarios, ambient light type detection in ultra-high dynamic range scenarios, and high-resolution high-dynamic range multispectral imaging. Of course, the multispectral detection device provided in this application is not limited to these three scenarios; it can also be applied to other scenarios, and this application makes no limitation on its application.

[0088] This application provides an embodiment of an ambient light intensity detection method for ultra-high dynamic scenes, such as... Figure 8 As shown, the method includes:

[0089] 801. When the multispectral detection device determines to perform ambient light intensity detection, it combines the single-channel spectral units in each of the multiple spectrum arrays into a single all-pass channel to obtain the combined signal of the multiple all-pass channels and read it out.

[0090] In some embodiments, when the logic control circuit in the multispectral detection device receives an ambient light intensity detection indication, since the digital-to-analog converter (ADC) that collects the current has a higher dynamic range detection capability, the logic control circuit can control the first switching device TX1 of each pixel unit in multiple spectrum arrays to turn off and the second switching device TX2 to turn on, and instruct the row and column control circuit SWAP SEL to control the row and column control circuit to combine all single-channel spectral units in each spectrum array as a group for signal merging, and the signal after the group is merged is used to indicate the ambient light intensity of a frequency array region.

[0091] For example, a schematic diagram of merging single-channel spectral units in each spectral array into an all-pass channel is shown below. Figure 9 As shown, it can be understood that a single-channel spectral unit C in a spectral array... 11 C 12 C 13 ..., C mn Merged into a single all-through channel, resulting in Figure 9 The full-pass channel (1,1) in the array will connect the single-channel spectral unit C in another spectral array. 11 C 12 C 13 ..., C mn Merged into a single all-through channel, resulting in Figure 9 The full-pass channel (1,2) in the array allows for the execution of single-channel spectral units C of multiple spectral arrays. 11 C 12 C 13 ..., C mn After merging, the resulting fully connected channels include (1,1), (1,2), ..., (x,y).

[0092] In some embodiments, the merging method is not limited to charge merging, analog domain merging, and digital domain merging.

[0093] In a spectral array, charge coalescence can be understood as C 11 C 12 C 13 ..., C mnThe charge merging of these multiple single-channel spectral units, specifically single-channel spectral unit C 11 The charge of pixel units (1,1), (1,2), ..., (h,k) and the single-channel spectral unit C 12 The charge of pixel units (1,1), (1,2), ..., (h,k) and the single-channel spectral unit C 13 The charges of pixel units (1,1), (1,2), ..., (h,k), ..., single-channel spectral unit C mn The charges of pixel units (1,1), (1,2), ..., (h,k) in the array are combined. The combination of charges of two pixel units can be understood as simultaneously turning on the second switching device TX2 coupled to both pixel units, so that the current signals of the two pixel units after photoelectric conversion are transmitted to the same current path to the row and column control circuit SWAP SEL. Similarly, by combining the current signals of the pixel units involved in each spectrum array, the current signal of the full-pass channel of each single-channel spectral unit in the same spectrum array after charge combination can be obtained.

[0094] Taking a spectral array as an example, analog domain merging can be understood as C 11 C 12 C 13 ..., C mn The simulation domains of these multiple single-channel spectral units are merged, specifically into single-channel spectral unit C. 11 The analog domain signal of pixel units (1,1), (1,2), ..., (h,k) and single-channel spectral unit C 12 The analog domain signal of pixel units (1,1), (1,2), ..., (h,k) and single-channel spectral unit C 13 The analog domain signals of pixel units (1,1), (1,2), ..., (h,k), ..., single-channel spectral unit C mn The analog domain signals of pixel units (1,1), (1,2), ..., (h,k) in the array are combined. The analog domain combination of two pixel units can be understood as follows: the current signal after photoelectric conversion of each pixel unit can be transmitted on its own current path, but analog signals are combined when transmitted to ADC2. Similarly, by combining the analog signals of the pixel units involved in each spectrum array at ADC2, the analog signal of each single-channel spectrum unit in the same spectrum array after analog signal combination is input to at least one ADC2.

[0095] Taking a spectrum array as an example, digital domain merging can be understood as C 11 C 12 C 13 ..., Cmn The digital domains of these multiple single-channel spectral units are combined, specifically into single-channel spectral unit C. 11 The digital domain signals of pixel units (1,1), (1,2), ..., (h,k) and single-channel spectral unit C 12 The digital domain signals of pixel units (1,1), (1,2), ..., (h,k) and single-channel spectral unit C 13 The digital domain signals of pixel units (1,1), (1,2), ..., (h,k), ..., single-channel spectral unit C mn The digital domain signals of pixel units (1,1), (1,2), ..., (h,k) in the array are combined. The digital domain combination of two pixel units can be understood as the current signal after photoelectric conversion of each pixel unit being transmitted to ADC2 along its respective current path, and then the digital signals after analog-to-digital conversion by ADC2 are combined (e.g., the result after weighted averaging). Similarly, by accumulating (combining) the digital signals of the pixel units involved in each spectrum array after analog-to-digital conversion in ADC2, the fully-pass digital signal of each single-channel spectrum unit in the same spectrum array can be obtained.

[0096] 802. When the multispectral detection device determines that the combined signal read from each full-pass channel is less than or equal to a preset threshold, it continues to divide the multiple full-pass channels into equal intervals to obtain multiple grouped and combined signals and read them out.

[0097] If the logic control circuit in the multispectral detection device determines that the combined signal read from each all-pass channel is less than a preset threshold, then C in one all-pass channel can be considered as... 11 C 12 ..., C mn The ambient light intensity determined by the combined signal is still very weak, and it can still be used to... Figure 9 The multiple all-pass channels (1,1), (1,2), ..., (x,y) obtained in the process are further divided into multiple groups at equal intervals. The division result can be represented as (x / k1,y / k2), (k1,k2=1,2,3...) groups, or (x / k1,y / k2) pixel regions, in order to achieve noise reduction or increase sensitivity. The merging method is not limited to charge merging, analog domain merging, and digital domain merging.

[0098] In this way, the signal obtained after dividing the (x / k1, y / k2) groups (pixel regions) and merging them can be quantized by at least one ADC2 in ADC group 2 to obtain the ambient light intensity of different groups or pixel regions, or ambient light brightness.

[0099] Step 802 is equivalent to, in the logic control circuit, when determining to perform ambient light intensity detection, controlling the first switching device in each pixel unit to turn off and the second switching device to turn on. The row and column control circuit divides multiple spectrum arrays into regions, and combines the single-channel spectral units in all spectrum arrays of each region as a group. The signal after combining the signals of a group is used to indicate the ambient light intensity of a spectrum array region.

[0100] In some embodiments, the logic control circuit of this application can dynamically adapt to low gain or high gain according to the ambient light intensity of the previous frame image when performing signal readout for each pixel unit at the bottom layer, so as to further meet the requirements of dynamic range of light intensity detection.

[0101] That is, when the logic control circuit determines that the ambient light intensity of the previous frame image is greater than or equal to a preset threshold, it reduces the exposure and / or gain of each pixel unit when acquiring the signal of each pixel unit in the next frame image.

[0102] When the ambient light intensity of the previous frame image is determined to be less than a preset threshold, the exposure and / or gain of each pixel unit is increased when acquiring the signal of each pixel unit in the next frame image.

[0103] In other words, the gains of multiple pixel units in multiple spectrum arrays are not entirely the same. When the logic control circuit determines that the ambient light intensity of the previous frame image is strong, the gain of the pixel unit can be appropriately reduced before obtaining the next frame image, i.e., when acquiring the light signal of the pixel unit required for the next frame image; conversely, when the ambient light intensity of the previous frame image is weak, the gain of the pixel unit can be appropriately increased. This multi-gain reading method of the underlying pixel unit can further enhance the dynamic range capability of ambient light detection.

[0104] Therefore, with the ambient light intensity detection method provided in this application, for each pixel unit, the part of the circuit coupled to the first switching device TX1 (including capacitor C) in that pixel unit is not working, and there will be no saturation problem caused by signal clamping, making it suitable for high dynamic range light sensing detection. Moreover, this application can also be used to detect ambient light of different intensities by using different current summing methods (step 801 or step 802).

[0105] Compared to existing ambient light intensity detection solutions, which can only detect ambient light intensity at a single point (a single pixel unit) and cannot effectively detect the actual dynamic range of the environment, this application cannot effectively assist video post-production software (After Effects, AE) in accurately and quickly configuring different High Dynamic Range (HDR) ratios. This application, however, can partition (group) multiple pixel units and, through flexible signal merging methods, is applicable to different ambient light brightness scenarios, thus achieving a balance between ambient light detection accuracy and resolution.

[0106] The ambient light intensity detection scheme provided in this application uses multi-regional high dynamic ambient light intensity information to assist other cameras in making accurate and rapid judgments on the afterimage (AE) or to accurately configure the AE for different HDR ratios in HDR modes. It can also assist HDR fusion algorithms to adapt to more suitable compression curves.

[0107] This application provides a method for detecting ambient light type in ultra-high dynamic scenes, such as... Figure 10 As shown, the method includes:

[0108] 1001. When the multispectral detection device determines that ambient light type detection is to be performed, it controls the first switching device in each pixel unit to turn off and the second switching device to turn on.

[0109] When the logic control unit determines to perform ambient light type detection, it can control the first switching device TX1 in each pixel unit to turn off and the second switching device TX2 to turn on, so that the current signal after photoelectric conversion of the PD in each pixel unit flows to the row and column control circuit through TX2 and the current mirror M.

[0110] 1002. The multispectral detection device divides multiple spectrum arrays into regions, and combines the same single-channel spectral units in all spectrum arrays in each region as a group to obtain the combined signal of each single-channel spectral unit in each region. The signal after combining the signals of one group in each region is used to indicate a type of ambient light.

[0111] In some embodiments, the logic control circuit may instruct the row and column control circuit to divide the multiple spectrum arrays into regions, where the division can be understood as... Figure 9 The (1,1), (1,2), ..., (x,y) spectrum arrays shown are read out by merging the arrays according to (x / k1,y / k2) (k1,k2=1,2,3...). Unlike step 801, the row and column control circuit does not need to merge multiple single-channel spectral units in each spectrum array, but instead merges the signals of the same single-channel spectral units in each region.

[0112] like Figure 11 As shown, assuming x=y=4, k1=k2=2, there are a total of 16 spectrum arrays, which are divided into 4 partitions, or 4 groups: ①, ②, ③, and ④. Among them, partition ① includes 4 spectrum arrays (1,1), (1,2), (2,1), and (2,2); partition ② includes 4 spectrum arrays (1,3), (1,4), (2,3), and (2,4); partition ③ includes 4 spectrum arrays (3,1), (3,2), (4,1), and (4,2); and partition ④ includes 4 spectrum arrays (3,3), (3,4), (4,3), and (4,4).

[0113] like Figure 12 As shown, taking the four spectrum arrays in partition ① as an example, the identical single-channel spectral units in these four spectrum arrays can be grouped together for signal merging. That is, the four single-channel spectral units C in these four spectrum arrays... 11 The pixel units in the array are combined to combine the signals of the four single-channel spectral units C in the four spectral arrays. 21 The pixel units in the array are combined to combine the signals of the four single-channel spectral units C in the four spectral arrays. 31 The pixel units in the array are combined into signals, ..., the four single-channel spectral units C in these four spectral arrays are combined. mn The pixel units in the partition are then combined to obtain the combined signal for each single-channel spectral unit in partition ①. A similar signal combining process can be performed for the other three partitions.

[0114] Among them, the merging methods of the same single-channel spectral units or the same spectral channels in each partition are not limited to the above-mentioned charge merging, analog domain merging and digital domain merging.

[0115] In some embodiments, the logic control circuit can adapt the partitioning and merging method according to the requirements of dynamic range and ambient light type detection accuracy, that is, appropriately determine the values ​​of k1 and k2. For example, when the ambient light intensity was determined to be strong during the imaging of the previous frame, the values ​​of k1 and k2 can be appropriately increased, that is, the number of spectrum arrays in each partition is smaller. When the ambient light intensity was determined to be weak during the imaging of the previous frame, the values ​​of k1 and k2 can be appropriately decreased, that is, the number of spectrum arrays in each partition is larger, so as to improve the ambient light type detection accuracy.

[0116] Understandably, the merged signal, after being quantized by ADC2, can yield spectral response information for different partitions.

[0117] In some embodiments, in low-light scenarios, scenarios with low dynamic range, or scenarios with complex light sources, the logic control unit can also control the first switching device TX1 to turn on and the second switching device TX2 to turn off. Thus, the signal merged in step 1002 is quantized by ADC1. This is because ADC1 has a strong quantization capability for low light conditions.

[0118] In some embodiments, the signal readout method of each pixel unit at the bottom layer can be dynamically adapted to low or high gain based on the ambient light intensity detection information of the previous frame image, further meeting the requirements of the dynamic range of light intensity detection. The specific implementation is similar to the gain adaptation process in step 802.

[0119] Therefore, the ambient light type detection method for ultra-high dynamic range scenes provided in this application can be applied to different ambient light brightness scenes by partitioning multiple spectral arrays and utilizing flexible signal merging methods. This achieves a balance between detection accuracy and resolution for different ambient light types, without the saturation problem caused by signal clamping, making it suitable for high dynamic range light sensing. This multi-regional high dynamic range multi-channel spectral information can be used to assist other cameras in performing regional ambient light type detection and regional AWB parameter adaptation, facilitating more accurate color reproduction in single or mixed color temperature scenes.

[0120] This application provides a multispectral imaging method, such as... Figure 13 As shown, the method includes:

[0121] 1301. When the multispectral detection device determines to perform multispectral imaging, it controls the first switching device in each pixel unit to be turned on and the second switching device to be turned off.

[0122] Since the ADC for acquiring voltage, namely ADC1 in this application, can be used for correlation double sampling quantization of voltage signals, when the logic control circuit determines that multispectral imaging is to be performed, the current signal after photoelectric conversion of the pixel unit in each single-channel spectral unit in the spectrum array can be directed to the ADC group 1 for acquiring voltage.

[0123] 1302. The row and column control circuit combines multiple pixel units of each single-channel spectral unit of each spectrum array as a group to obtain the combined signal of each single-channel spectral unit of each spectrum array. The combined signal of each single-channel spectral unit of each spectrum array is used for multispectral imaging.

[0124] In some embodiments, the logic control circuit can control each pixel unit of each single-channel spectral unit in each spectral array to individually input the current signal after photoelectric conversion to the ADC1 for reading out, thus obtaining a digital signal. That is, the current signal of the bottom pixel unit of the same spectral channel after photoelectric conversion can be individually output to the ADC1 and quantized to improve image resolution in high-brightness scenes.

[0125] In some embodiments, to achieve higher color gamut color reproduction and image effects, the logic control circuit can control the row and column control circuit to group or partition multiple pixel units (pixel unit arrays) in each single-channel spectral unit of each of the multiple spectral arrays, and perform signal merging on the grouping or partitioning. For example... Figure 14 As shown, the row and column control circuit can combine signals from multiple pixels in each single-channel spectral unit into groups of four adjacent pixel units, such as... Figure 14 The middle pixel units (1,1), (1,2), (2,1) and (2,2) are grouped together for signal merging, and the pixel units (1,k-1), (1,k), (2,k-1) and (2,k) are grouped together for signal merging. The merging method is not limited to charge merging, analog domain merging or digital domain merging.

[0126] In some embodiments, in order to achieve higher color gamut color reproduction and image effects, the logic control circuit can control the row and column control circuit to group multiple single-channel spectral units of each spectral array in multiple spectral arrays, with each group including multiple adjacent single-channel spectral units.

[0127] The row and column control circuit combines the signals of each group of each spectrum array to obtain the combined signal of each group of each spectrum array. The combined signal of each group of each spectrum array is used for multispectral imaging.

[0128] In other words, the row and column control circuit can combine signals from different single-channel spectral units in each spectral array. For example... Figure 15 As shown, taking a spectrum array as an example, the row and column control circuit can control four adjacent single-channel spectral units C in a spectrum array. 11 C 12 C 22 and C 22 For signal merging in a group, single-channel spectral unit C 13 C 14 C 23 and C 24 Signals are combined into a single group. While this sacrifices some color gamut space, it can improve both the color accuracy and dynamic range of the image.

[0129] In some embodiments, the signal readout method of each pixel unit at the bottom layer can be dynamically adapted to low gain or high gain, or a fusion of high and low gain, based on the brightness information of the previous frame image.

[0130] In some embodiments, the underlying pixel units of the same single-channel spectral unit can also be configured with different exposures or gains and subjected to high dynamic range synthesis compression.

[0131] In some embodiments, Bayer image remosaic of multiple single-channel spectral units can be performed on-chip or on the platform side.

[0132] Therefore, the multispectral imaging method provided in this application can be used for hyperspectral imaging. By utilizing high-resolution, high-dynamic-range multispectral information, it can achieve color reproduction and image quality with a wider color gamut. Compared to existing solutions, which cannot effectively and flexibly balance multispectral and high-dynamic-range effects, this application can achieve high-resolution hyperspectral imaging, balancing resolution and color accuracy, achieving low noise, and is suitable for high-quality image reproduction and night scene enhancement.

[0133] It is understood that, in order to achieve the above functions, the multispectral detection device includes hardware and / or software modules that perform the respective functions. Based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is performed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0134] This embodiment can divide the electronic device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0135] When dividing each function into modules according to its corresponding function. Figure 16 A schematic diagram of a possible composition of the multispectral detection device 160 involved in the above embodiments is shown, such as... Figure 16 As shown, the multispectral detection device 160 may include a switch control unit 1601 and a signal merging unit 1602.

[0136] The signal combining unit 1602 can be used to support the multispectral detection device 160 in performing the above-described steps 801, 802, 1002 and 1302, and / or other processes used in the technology described herein.

[0137] The switch control unit 1601 can be used to support the multispectral detection device 160 in performing the above-described steps 1001 and 1301, and / or other processes for the techniques described herein.

[0138] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0139] The multispectral detection device 160 provided in this embodiment is used to perform the above-described multispectral processing method, and therefore can achieve the same effect as the above-described implementation method.

[0140] When using an integrated unit, the multispectral detection device 160 may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the operation of the multispectral detection device 160, for example, to support the multispectral detection device 160 in executing the steps performed by the switch control unit 1601 and the signal merging unit 1602. The storage module can be used to store program code and data in the multispectral detection device 160. The communication module can be used to support communication between the multispectral detection device 160 and other devices.

[0141] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0142] Embodiments of this application also provide a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the antenna gain adjustment method in the above embodiments.

[0143] Embodiments of this application also provide a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the antenna gain adjustment method performed by the electronic device in the above embodiments.

[0144] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the multispectral processing method executed by the electronic device in the above method embodiments.

[0145] In this embodiment, the multispectral detection device, computer storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0146] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0148] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0150] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0151] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multispectral detection device, characterized in that, The multispectral detection device includes a logic control circuit, multiple spectrum arrays, row and column control circuits, multiple analog-to-digital converters (ADCs) for acquiring current, and multiple ADCs for acquiring voltage. Each of the spectral arrays includes multiple single-channel spectral units, and each single-channel spectral unit includes multiple pixel units. For each of the plurality of pixel units, each pixel unit is coupled to the row and column control circuit, one of the plurality of current-acquiring ADCs and one of the plurality of voltage-acquiring ADCs. Each pixel unit includes a photodetector (PD), a voltage detection circuit, and a current detection circuit. The voltage detection circuit includes a first switching device, and the current detection circuit includes a second switching device. The logic control circuit is used to control the first and second switching devices in each pixel unit to perform imaging or ambient light detection in different working modes. The logic control circuit is also used to control the row and column control circuit to use different signal combining methods for multiple pixel units in the multiple spectrum arrays to perform image enhancement or ambient light detection enhancement.

2. The multispectral detection device according to claim 1, characterized in that, The logic control circuit is used to control the first switching device in each pixel unit to be turned on and the second switching device to be turned off, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted to the ADC of the acquisition voltage for imaging through the first switching device; or, the logic control circuit is used to control the first switching device in each pixel unit to be turned off and the second switching device to be turned on, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted to the ADC of the acquisition current for ambient light detection through the second switching device.

3. The multispectral detection device according to claim 2, characterized in that, The current sensing circuit in each pixel unit further includes a current mirror, and the voltage sensing circuit in each pixel unit further includes a reset transistor, a source follower transistor, and a row select transistor; For each pixel unit, the output terminal of the PD is coupled to the first terminal of the first switching device and the first terminal of the second switching device; the second terminal of the first switching device is coupled to the first terminal of the reset transistor and the first terminal of the source follower transistor; the second terminal of the source follower transistor is coupled to the first terminal of the row select transistor; the second terminal of the row select transistor is coupled to the first input terminal of the row and column control circuit; the first output terminal of the row and column control circuit is coupled to the ADC of the acquired voltage; and the second output terminal of the row and column control circuit is coupled to the ADC of the acquired current. The second terminal of the second switching device is coupled to the first terminal of the current mirror; and the first output terminal of the current mirror is coupled to the second input terminal of the row and column control circuit.

4. The multispectral detection device according to any one of claims 1-3, characterized in that, The logic control circuit is used for: When it is determined that ambient light intensity detection is to be performed, the first switching device in each pixel unit is turned off and the second switching device is turned on. Using a single spectrum array as a unit, the row and column control circuit controls the signal merging of all single-channel spectral units in each spectrum array as a group. The signal after merging a group is used to indicate the ambient light intensity of a spectrum array region.

5. The multispectral detection device according to any one of claims 1-3, characterized in that, The logic control circuit is used for: When it is determined that ambient light intensity detection is to be performed, the first switching device in each pixel unit is turned off and the second switching device is turned on. The row and column control circuit is used to divide the multiple spectrum arrays into regions. The single-channel spectral units in all spectrum arrays in each region are grouped together and the signal of the grouped signal is used to indicate the ambient light intensity of a spectrum array region.

6. The multispectral detection device according to any one of claims 1-3, characterized in that, The logic control circuit is used for: When determining the ambient light type detection, the first switching device in each pixel unit is turned off and the second switching device is turned on. The row and column control circuit is used to divide the multiple spectrum arrays into regions. The same single-channel spectral units in all spectrum arrays in each region are grouped together and their signals are merged to obtain the merged signal of each single-channel spectral unit in each region. The signal after merging one group in each region is used to indicate an ambient light type.

7. The multispectral detection device according to any one of claims 1-3, characterized in that, The logic control circuit is used for: When multispectral imaging is to be performed, the first switching device in each pixel unit is turned on and the second switching device is turned off. The row and column control circuit controls the multiple pixel units of each single-channel spectral unit in each of the multiple spectral arrays to merge signals as a group, thereby obtaining the merged signal of each single-channel spectral unit of each spectral array. The merged signal of each single-channel spectral unit of each spectral array is used for multispectral imaging.

8. The multispectral detection device according to claim 7, characterized in that, The logic control circuit is used for: When multispectral imaging is to be performed, the first switching device in each pixel unit is turned on and the second switching device is turned off. The row and column control circuit controls the grouping of multiple single-channel spectral units in each of the multiple spectral arrays, with each group including multiple adjacent single-channel spectral units; The row and column control circuits are controlled to combine the signals of each group of each spectrum array to obtain a combined signal of each group of each spectrum array. The combined signal of each group of each spectrum array is used for multispectral imaging.

9. The multispectral detection device according to claim 4, characterized in that, The signal merging includes: Charge merging, or analog domain merging, or digital domain merging.

10. The multispectral detection device according to claim 4, characterized in that, The logic control circuit is also used for: When the ambient light intensity of the previous frame image is determined to be greater than or equal to a preset threshold, the exposure and / or gain of each pixel unit is reduced when acquiring the signal of each pixel unit in the next frame image. When the ambient light intensity of the previous frame image is determined to be less than the preset threshold, the exposure and / or gain of each pixel unit is increased when acquiring the signal of each pixel unit in the next frame image.

11. A multispectral processing method, characterized in that, It is applied to a multispectral detection device, which includes a logic control circuit, multiple spectrum arrays, row and column control circuits, multiple analog-to-digital converters (ADCs) for acquiring current, and multiple ADCs for acquiring voltage. Each of the spectral arrays includes multiple single-channel spectral units, and each single-channel spectral unit includes multiple pixel units. For each of the plurality of pixel units, each pixel unit is coupled to the row and column control circuit, one of the plurality of current-acquiring ADCs and one of the plurality of voltage-acquiring ADCs. Each pixel unit includes a photodetector (PD), a voltage detection circuit, and a current detection circuit. The voltage detection circuit includes a first switching device, and the current detection circuit includes a second switching device. The method includes: The multispectral detection device controls the logic control circuit to perform imaging or ambient light detection in different working modes for the first and second switching devices in each pixel unit. The multispectral detection device controls the logic control circuit to use different signal combining methods for multiple pixel units in the multiple spectrum arrays to perform imaging enhancement or ambient light detection enhancement.

12. The method according to claim 11, characterized in that, The multispectral detection device controls the logic control circuit to perform imaging or ambient light detection for the first and second switching devices in each pixel unit in different operating modes, including: The multispectral detection device controls the logic control circuit to turn on the first switching device and turn off the second switching device in each pixel unit, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted to the ADC of the acquisition voltage for imaging through the first switching device; Alternatively, the multispectral detection device controls the logic control circuit to turn off the first switching device and turn on the second switching device in each pixel unit, so that the current signal after photoelectric conversion of the PD in each pixel unit is transmitted to the ADC of the current acquisition for ambient light detection through the second switching device.

13. The method according to claim 11, characterized in that, The current sensing circuit in each pixel unit further includes a current mirror, and the voltage sensing circuit in each pixel unit further includes a reset transistor, a source follower transistor, and a row select transistor; For each pixel unit, the output terminal of the PD is coupled to the first terminal of the first switching device and the first terminal of the second switching device; the second terminal of the first switching device is coupled to the first terminal of the reset transistor and the first terminal of the source follower transistor; the second terminal of the source follower transistor is coupled to the first terminal of the row select transistor; the second terminal of the row select transistor is coupled to the first input terminal of the row and column control circuit; the first output terminal of the row and column control circuit is coupled to the ADC of the acquired voltage; and the second output terminal of the row and column control circuit is coupled to the ADC of the acquired current. The second terminal of the second switching device is coupled to the first terminal of the current mirror; and the first output terminal of the current mirror is coupled to the second input terminal of the row and column control circuit.

14. The method according to claim 11 or 12, characterized in that, The multispectral detection device controls the logic control circuit to use different signal combining methods for multiple pixel units in the multiple spectrum arrays to perform image enhancement or ambient light detection enhancement, including: When it is determined that ambient light intensity detection is to be performed, the first switching device in each pixel unit is turned off and the second switching device is turned on. Using a single spectrum array as a unit, the row and column control circuit controls the signal merging of all single-channel spectral units in each spectrum array as a group. The signal after merging a group is used to indicate the ambient light intensity of a spectrum array region.

15. The method according to claim 11 or 12, characterized in that, The multispectral detection device controls the logic control circuit to use different signal combining methods for multiple pixel units in the multiple spectrum arrays to perform image enhancement or ambient light detection enhancement, including: When it is determined that ambient light intensity detection is to be performed, the first switching device in each pixel unit is turned off and the second switching device is turned on. The row and column control circuit is used to divide the multiple spectrum arrays into regions. The single-channel spectral units in all spectrum arrays in each region are grouped together and the signal of the grouped signal is used to indicate the ambient light intensity of a spectrum array region.

16. The method according to claim 11 or 12, characterized in that, The multispectral detection device controls the logic control circuit to use different signal combining methods for multiple pixel units in the multiple spectrum arrays to perform image enhancement or ambient light detection enhancement, including: When determining the ambient light type detection, the first switching device in each pixel unit is turned off and the second switching device is turned on. The row and column control circuit is used to divide the multiple spectrum arrays into regions. The same single-channel spectral units in all spectrum arrays in each region are grouped together and their signals are merged to obtain the merged signal of each single-channel spectral unit in each region. The signal after merging one group in each region is used to indicate an ambient light type.

17. The method according to claim 11 or 12, characterized in that, The multispectral detection device controls the logic control circuit to use different signal combining methods for multiple pixel units in the multiple spectrum arrays to perform image enhancement or ambient light detection enhancement, including: When multispectral imaging is to be performed, the first switching device in each pixel unit is turned on and the second switching device is turned off. The row and column control circuit controls the multiple pixel units of each single-channel spectral unit in each of the multiple spectral arrays to merge signals as a group, thereby obtaining the merged signal of each single-channel spectral unit of each spectral array. The merged signal of each single-channel spectral unit of each spectral array is used for multispectral imaging.

18. The method according to claim 17, characterized in that, The multispectral detection device controls the logic control circuit to use different signal combining methods for multiple pixel units in the multiple spectrum arrays to perform image enhancement or ambient light detection enhancement, including: When multispectral imaging is to be performed, the first switching device in each pixel unit is turned on and the second switching device is turned off. The row and column control circuit controls the grouping of multiple single-channel spectral units in each of the multiple spectral arrays, with each group including multiple adjacent single-channel spectral units; The row and column control circuits are controlled to combine the signals of each group of each spectrum array to obtain a combined signal of each group of each spectrum array. The combined signal of each group of each spectrum array is used for multispectral imaging.

19. The method according to claim 14, characterized in that, The signal merging includes: Charge merging, or analog domain merging, or digital domain merging; The method further includes: when the ambient light intensity of the previous frame image is greater than or equal to a preset threshold, when acquiring the signal of each pixel unit of the next frame image, reducing the exposure and / or gain of each pixel unit; When the ambient light intensity of the previous frame image is determined to be less than the preset threshold, the exposure and / or gain of each pixel unit is increased when acquiring the signal of each pixel unit in the next frame image.

20. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in any one of claims 11-19.

Images