Sensing device, sensing method and sensing operating assembly

Abstract

The present application provides a sensing device, a sensing method and a sensing operation assembly for sensing an object. The sensing device can include a sensor including a light detecting unit for absorbing (i) a first incident light having a first wavelength to generate a first detection signal and (ii) a second incident light having a second wavelength to generate a second detection signal. The sensing device can further include a computing circuit coupled to the sensor. The computing circuit can output a computing result according to the first detection signal and the second detection signal. The sensing device can further include an adjusting circuit coupled to the computing circuit. The adjusting circuit can perform an adjustment on one or more functions associated with the sensing device according to the computing result.

Description

Technical Field

[0001] This application relates primarily to sensor devices. More specifically, this application relates to a sensing device capable of absorbing multiple wavelengths of light and adjusting one or more of their functions. Background Technology

[0002] A sensor is a component, module, machine, or subsystem that can be used to detect events or changes in its environment and transmit information to other electronic devices. In distance measurement, time-of-flight (ToF) sensors are one solution for determining the distance between a sensor and a three-dimensional target at various points in an image, by measuring the round-trip time of a laser or LED light signal. With technological advancements, ToF sensors can be used in many mobile / wearable / portable telecommunications or entertainment electronic devices, such as image sensors, data communication or medical components, and many other suitable applications. Summary of the Invention

[0003] The various forms and advantages of the embodiments of this application will be known and understood through the following description or by implementing the embodiments of this application.

[0004] One exemplary embodiment of this application is a sensing device. The sensing device may include a sensor comprising a photodetector unit configured to absorb (i) a first incident light having a first wavelength to generate a first detection signal and (ii) a second incident light having a second wavelength to generate a second detection signal. The sensing device may further include a computing circuit coupled to the sensor. The computing circuit may output a calculation result based on the first and second detection signals. The sensing device may further include an adjustment circuit coupled to the computing circuit. The adjustment circuit may adjust one or more functions associated with the sensing device based on the calculation result.

[0005] In some embodiments, the sensor may further include a first light source configured to emit a first transmitted light having a first wavelength; and a second light source configured to emit a second transmitted light having a second wavelength. The first incident light is a portion of the first transmitted light reflected by an object; and the second incident light is a portion of the second transmitted light reflected by an object.

[0006] In some implementations, the first transmitted light is modulated at a first modulation frequency in a first period and at a second modulation frequency in a second period.

[0007] In some implementations, the first detection signal has a first value and the second detection signal has a second value, wherein the first value and the second value are different.

[0008] In some embodiments, the computing circuit may include a pre-computation unit configured to perform an integral or differential calculation based on a first detection signal and a second detection signal to output a pre-computation result. The computing circuit may further include an AI modeling unit coupled to the pre-computation unit, configured to perform image data analysis and output a modeling signal. The computing circuit may further include a prediction unit coupled to the AI ​​modeling unit, configured to predict a state as the calculation result based on the modeling signal.

[0009] In some embodiments, the photodetector unit includes a first photodetector and a second photodetector, wherein the first photodetector has a first light-absorbing material and the second photodetector has a second light-absorbing material, and wherein the first light-absorbing material and the second light-absorbing material are different.

[0010] In some embodiments, the first light-absorbing material comprises silicon, and the second light-absorbing material comprises germanium or a group III-V material.

[0011] In some implementations, the sensing device further includes a two-dimensional light detection unit configured to detect an object and generate a two-dimensional image.

[0012] In some implementations, the computing circuit is configured to output a calculation result based on a first detection signal, a second detection signal, and a two-dimensional image.

[0013] In some implementations, the sensor is configured to output depth information, correction information, error correction code information, or numerical information based on a first detection signal and a second detection signal.

[0014] In some embodiments, the sensor may further include a light-emitting unit configured to emit a first transmitted light having a first wavelength and a second transmitted light having a second wavelength. The sensor may further include a controller configured to output a sensor output signal based on the first detection signal and the second detection signal. The sensor is implemented on one or more chips.

[0015] In some implementations, the light-emitting unit is implemented on at least one first chip, the light-detecting unit is implemented on at least one second chip, and the controller is implemented on a third chip.

[0016] In some implementations, at least one of the first chip and the second chip is wire bonded to the third chip.

[0017] In some implementations, at least one of the first chip and the second chip is stacked on the third chip.

[0018] In some implementations, at least one of the first chip and the second chip is flip-chip bonded to the third chip.

[0019] In some implementations, at least one of the first chip and the second chip is wafer-bonded to the third chip.

[0020] In some implementations, the light-emitting unit, the light-detecting unit, and the controller are implemented on a shared chip.

[0021] In some implementations, the controller and one of the light-emitting unit and the light-detecting unit are implemented on a first chip; and the other of the light-emitting unit and the light-detecting unit is implemented on a second chip.

[0022] In some embodiments, the light-emitting unit may include a first chip, which includes a first light source and is configured to emit first transmitted light. The light-emitting unit may also include a second chip, which includes a second light source and is configured to emit second transmitted light.

[0023] In some embodiments, the photodetector unit may include a first chip containing a first photodetector configured to absorb first incident light. The photodetector unit may also include a second chip containing a second photodetector configured to absorb second incident light.

[0024] Another example of this application is a sensing method. The sensing method may include a sensor comprising a three-dimensional light detection unit absorbing first incident light having a first wavelength to generate a first detection signal, and absorbing second incident light having a second wavelength to generate a second detection signal. The sensing method may further include a calculation circuit determining a calculation result based on the first and second detection signals. The sensing method may further include an adjustment circuit determining an output based on the calculation result to perform adjustments on one or more functions associated with a sensing device.

[0025] In some embodiments, the sensing method may further include emitting a first transmitted light having a first wavelength from a first light source. The sensing method may further include emitting a second transmitted light having a second wavelength from a second light source. The first incident light is a portion of the first transmitted light reflected by an object; and the second incident light is a portion of the second transmitted light reflected by an object.

[0026] In some implementations, the first transmitted light is modulated at a first modulation frequency in a first period and at a second modulation frequency in a second period.

[0027] In some implementations, the first detection signal has a first value and the second detection signal has a second value, wherein the first value and the second value are different.

[0028] In some implementations, the calculation result is determined based on a procedure that includes a pre-computation unit performing an integral or differential calculation based on a first and a second detection signal to output a pre-computation result. The procedure may further include an AI modeling unit performing image data analysis to output a modeling signal. The procedure may further include a prediction unit predicting a state as the calculation result based on the modeling signal.

[0029] In some embodiments, the photodetector unit includes a first photodetector and a second photodetector, wherein the first photodetector has a first light-absorbing material and the second photodetector has a second light-absorbing material, and wherein the first light-absorbing material and the second light-absorbing material are different.

[0030] In some embodiments, the first light-absorbing material comprises silicon, and the second light-absorbing material comprises germanium or a group III-V material.

[0031] Another example of this application is a sensing operation component. The sensing operation component may include a sensor comprising a photodetector unit configured to absorb (i) a first incident light having a first wavelength to generate a first detection signal; and (ii) a second incident light having a second wavelength to generate a second detection signal. The sensing operation component may further include one or more processors and a memory, the memory including one or more tangible non-transient computer-readable media. The media may store computer-readable instructions that, when executed by one or more processors, cause one or more processors to perform operations. The operations may include determining a calculation result based at least in part on the first and second detection signals. The operations may further include determining adjustments to one or more functions of the sensing operation component based at least in part on the calculation result. The operations may further include implementing adjustments to the sensing operation component. The sensor may include a near-infrared or short-wave infrared sensor.

[0032] In some implementations, one or more of the first wavelength and the second wavelength have a wavelength greater than 1050 nm.

[0033] In some implementations, the calculation result is determined, at least in part, based on the first detection signal and the second detection signal, to include the determination of the material of the object associated with the first detection signal and the second detection signal.

[0034] In some implementations, determining the material of an object associated with the first detection signal and the second detection signal includes inputting the first detection signal and the second detection signal into a machine learning model, and receiving an output indication representing the material of the object as an output of the machine learning model.

[0035] Another example of this application is a sensor. The sensor may include a light-emitting unit configured to emit a first transmitted light having a first wavelength and a second transmitted light having a second wavelength. The sensor may further include a photodetector unit configured to absorb (i) the first incident light having the first wavelength to generate a first detection signal; and (ii) the second incident light having the second wavelength to generate a second detection signal. The sensor may further include a controller configured to output a sensor output signal based on the first detection signal and the second detection signal. The sensor may be implemented on one or more chips.

[0036] In some implementations, the light-emitting unit is implemented on at least one first chip, the light-detecting unit is implemented on at least one second chip, and the controller is implemented on a third chip.

[0037] In some implementations, at least one of the first chip and the second chip is wire bonded to the third chip.

[0038] In some implementations, at least one of the first chip and the second chip is stacked on the third chip.

[0039] In some implementations, at least one of the first chip and the second chip is flip-chip bonded to the third chip.

[0040] In some implementations, at least one of the first chip and the second chip is wafer-bonded to the third chip.

[0041] In some implementations, the light-emitting unit, the light-detecting unit, and the controller are implemented on a shared chip.

[0042] In some implementations, the controller, along with one of the light-emitting unit and the light-detecting unit, is implemented on a first chip, while the other of the light-emitting unit and the light-detecting unit is implemented on a second chip.

[0043] In some embodiments, the light-emitting unit may further include a first chip containing a first light source configured to emit first transmitted light. The light-emitting unit may further include a second chip containing a second light source configured to emit second transmitted light.

[0044] In some embodiments, the photodetector unit may further include a first chip containing a first photodetector configured to absorb the first incident light. The photodetector unit may further include a second chip containing a second photodetector configured to absorb the second incident light.

[0045] Another example of this application is an optical device for optical spectral analysis. The optical device may include a substrate made of at least silicon. The optical device may further include a plurality of sensors formed of at least germanium, wherein the plurality of sensors are supported by the substrate. The optical device may further include a plurality of wavelength filters disposed between the plurality of sensors and a target object, wherein the plurality of wavelength filters are configured to receive reflected light from the target object and filter the reflected light into a plurality of beams with different wavelength ranges, and wherein each of the plurality of sensors is configured to receive an individual beam of a specific wavelength range from the plurality of beams.

[0046] In some implementations, the optical device further includes a circuit system coupled to multiple sensors, the circuit system being configured to read out multiple electrical signals generated by the multiple sensors.

[0047] In some implementations, the optical device further includes one or more processors configured to determine a target object based on electrical signals.

[0048] In some implementations, the characteristics of the target object include its water content, carbohydrate content, sugar content, protein content, or fat content.

[0049] In some implementations, one or more processors are configured to determine whether a target object is a forgery based on the characteristics of the target object.

[0050] In some implementations, multiple sensors are formed on a first die and a circuit system is formed on a second die, wherein the multiple sensors are coupled to the circuit system via a die-level bonding between the first die and the second die.

[0051] In some embodiments, multiple sensors are formed on a first wafer and a circuit system is formed on a second wafer, wherein the multiple sensors are coupled to the circuit system via wafer-level bonding between the first wafer and the second wafer.

[0052] In some implementations, multiple sensors and circuit systems are formed on a substrate.

[0053] In some implementations, multiple sensors are configured as a one-dimensional array.

[0054] In some implementations, multiple sensors are configured as a one- or two-dimensional array.

[0055] In some implementations, the multiple wavelength filters comprise one or more absorptive materials, multilayer coatings, periodic gratings, non-periodic gratings, or combinations thereof.

[0056] In some implementations, the reflected light ranges from 700 nm to 1700 nm.

[0057] In some implementations, multiple sensors are positioned between multiple wavelength filters and the substrate.

[0058] In some implementations, the substrate is positioned between multiple wavelength filters and multiple sensors.

[0059] In some implementations, the optical device further includes a microlens array positioned between multiple wavelength filters and the target object.

[0060] In some embodiments, the optical device further includes a separator formed between the microlens array and the multiple sensors. Multiple wavelength filters may be formed within the separator.

[0061] Another example of this application is a method for operating an optical device to perform optical spectral analysis. The method may include receiving light reflected from a target object using multiple wavelength filters. The method may further include filtering the received light into multiple beams with different wavelength ranges using multiple wavelength filters. The method may further include receiving each of the multiple beams having a specific wavelength range using one or more individual sensors among a plurality of sensors. The method may further include generating electrical signals using multiple sensors based on the multiple beams. The method may further include providing electrical signals to one or more processors by a readout circuit system. The method may further include determining the characteristics of the target object by one or more processors based on the electrical signals.

[0062] Other examples of this application include systems, methods, apparatus, sensors, operating components, tangible non-transitory computer-readable media, and memory components.

[0063] The above and other features, features, and advantages of various embodiments will be described below with reference to the claims. The accompanying drawings form part of this specification and are intended to illustrate embodiments of the present application and to explain the related principles in conjunction with the description. Attached Figure Description

[0064] The following detailed description, together with the accompanying drawings, will enable the reader to more easily grasp and understand the above-mentioned features and the many advantages of this application, as shown in the drawings:

[0065] Figure 1 This is a block diagram of a sensing device example structure based on the example state of this application.

[0066] Figure 2 This is a block diagram of a sensing device example structure based on the example state of this application.

[0067] Figure 3 This is a block diagram of a sensing device example structure based on the example state of this application.

[0068] Figure 4 This is a block diagram of a sensing device example structure based on the example state of this application.

[0069] Figure 5 This is a block diagram of a sensing device example structure based on the example state of this application.

[0070] Figure 6 This is a block diagram of a sensing device example structure based on the example state of this application.

[0071] Figure 7 This is a block diagram of a sensing device with computer vision technology according to the example structure of this application.

[0072] Figure 8 Based on the example of this application, an implementation of a broadband application example of a sensing device is described.

[0073] Figure 9 Based on the example of this application, an implementation of a broadband application example of a sensing device is described.

[0074] Figure 10 This is a block diagram of an example broadband sensor based on the example configuration of this application.

[0075] Figure 11 This is a cross-sectional view of an example photodetector based on the example state of this application.

[0076] Figure 12 This is a flowchart illustrating an exemplary method based on the example state of this application.

[0077] Figure 13 This is a sample system diagram based on the example state of this application.

[0078] Figure 14 This is a sample system diagram based on the example state of this application.

[0079] Figure 15 This is a block diagram of an example sensor based on the example state of this application.

[0080] Figure 16 This is a block diagram of an example sensor based on the example state of this application.

[0081] Figure 17 This is a block diagram of an example sensor based on the example state of this application.

[0082] Figure 18 This is a block diagram of an example sensor based on the example state of this application.

[0083] Figure 19 This is a block diagram of an example sensor based on the example state of this application.

[0084] Figure 20 This is a block diagram of an example sensor based on the example state of this application.

[0085] Figure 21 This is a block diagram of an example sensor based on the example state of this application.

[0086] Figure 22 This is a block diagram of an example sensor based on the example state of this application.

[0087] Figure 23 This is a block diagram of an example sensor based on the example state of this application.

[0088] Figure 24 This is a block diagram of an example sensor based on the example state of this application.

[0089] Figure 25 This is a block diagram of an example sensor based on the example state of this application.

[0090] Figure 26 This is a block diagram of an example sensor based on the example state of this application.

[0091] Figure 27 This is a block diagram of an example optical detection circuit based on the example state of this application.

[0092] Figure 28 This is a block diagram of an example light source based on the example state of this application.

[0093] Figure 29A This is a block diagram of an example photodetector array based on the example state of this application.

[0094] Figure 29B This is a block diagram of an example photodetector array based on the example state of this application.

[0095] Figure 30 This is an example of a photodetector array described in accordance with the example of this application.

[0096] Figure 31 This is an example of a photodetector array described in accordance with the example of this application.

[0097] Figure 32 This is an example of a photodetector array described in accordance with the example of this application.

[0098] Figure 33 This is an example of a photodetector array described in accordance with the example of this application.

[0099] Explanation of symbols in the diagram

[0100] 100: Sensing device; 102: Broadband sensor, sensor

[0101] 104: Calculation Circuit 106: Adjustment Circuit

[0102] 108: Light detector, network 110: Light detector

[0103] 112: First light source, light source 114: Second light source, light source

[0104] 140: Object; 160: Light-emitting unit

[0105] 180: Light detection unit; 190: Second light detection unit, light detection unit

[0106] 200: Sensing device; 202: Light detection circuit, light detector

[0107] 203: Controller; 300: Sensing device

[0108] 400: Sensing device; 401: First chip, chip

[0109] 402: Second chip, chip 403: Third chip, chip

[0110] 404: Fourth chip, chip; 405: Fifth chip, chip.

[0111] 460: Guide hole; 500: Sensing device

[0112] 502: Sun; 600: Sensing device

[0113] 601: Guide hole; 602: Guide hole

[0114] 603: Pilot hole; 604: Pilot hole

[0115] 605: Pilot hole; 606: Pilot hole

[0116] 700: Sensing device; 800: Sensing device, component

[0117] 900: Sensing device; 1000: Broadband sensor

[0118] 1005: Voltage regulator; 1010: Temperature sensor

[0119] 1015: I2C controller; 1020: PPL

[0120] 1025: Timing generator; 1030: Modulation

[0121] 1035: Programmable delay; 1040: Pixel array

[0122] 1041: Pre-computation unit; 1042: First AI modeling unit

[0123] 1043: Second AI Modeling Unit; 1044: Prediction Unit

[0124] 1045: Read 1050: ADC

[0125] 1055: Signal Processor; 1060: Memory

[0126] 1065: Output interface 1200: Method

[0127] 1202: Step 1204: Step

[0128] 1206: Step 1208: Step

[0129] 1210: Step 1212: Step

[0130] 1214: Step 1216: Step

[0131] 1218: Step 1220: Step

[0132] 1300: Operating system, example system; 1302: Light-emitting unit

[0133] 1304: Light detection unit; 1306: Computing unit

[0134] 1308: Pre-computation unit; 1310: AI modeling unit

[0135] 1312: Prediction Unit; 1314: Image Analysis Unit

[0136] 1316: Material Judgment Unit; 1318: Adjustment Unit

[0137] 1320: Implementation Unit 1400: Example System, Example Operating System

[0138] 1402: Operating System; 1404: Operating Components

[0139] 1406: Processor; 1408: Memory

[0140] 1410: Instruction 1412: Data

[0141] 1414: Communication Interface; 1416: Machine Learning Model, Model

[0142] 1420: Networks 1430: Machine Learning Operating Systems

[0143] 1432: Processor; 1434: Memory

[0144] 1436: Data 1438: Instructions

[0145] 1440: Communication Interface; 1442: Machine Learning Model, Model

[0146] 1444: Model trainer; 1446: Training data, training dataset

[0147] 2401: Chip; 2501: First chip, chip

[0148] 2502: Second chip; 2601: First chip; chip

[0149] 2602: Second chip; 2701: Shared substrate, silicon substrate

[0150] 2801: Silicon substrate; 2802: N+III-V material contact layer

[0151] 2803: N-III-V material coating layer; 2804: Quantum dot (QD) material layer

[0152] 2805: P-III-V material coating layer; 2806: P+III-V material contact layer

[0153] 2900A: One-dimensional photodetector array; 2900B: Two-dimensional photodetector array

[0154] 2902: Substrate; 2904: Photodetector

[0155] 2904A: Sensor; 2904N: Sensor

[0156] 3000: FSI photodetector array; 3002: substrate

[0157] 3004: Light Detector; 3004A: Sensor

[0158] 3004B: Sensor; 3000C: Sensor

[0159] 3004N: Sensor; 3006: Optical Filter

[0160] 3006A: Bandpass filter; 3006B: Bandpass filter

[0161] 3006C: Bandpass filter; 3006N: Bandpass filter

[0162] 3100: FSI photodetector array; 3104: photodetector

[0163] 3104A: Sensor 3104B: Sensor

[0164] 3104C: Sensor; 3104N: Sensor

[0165] 3106: Filter 3106A: Filter

[0166] 3106B: Filter 3106C: Filter

[0167] 3106N: Filter; 3108: Microlens array

[0168] 3108A: Microlens array; 3108B: Microlens array

[0169] 3108C: Microlens array; 3108N: Microlens array

[0170] 3110: Separator; 3200: BSI Photodetector Array

[0171] 3202: Substrate; 3204: Photodetector

[0172] 3204A: Sensor 3204B: Sensor

[0173] 3204C: Sensor; 3204N: Sensor

[0174] 3206: Optical filter; Filter 3206A: Filter

[0175] 3206B: Filter 3206C: Filter

[0176] 3206N: Filter; 3208: Microlens array

[0177] 3210: Separator; 3300: BSI Photodetector Array

[0178] 3304A: Sensor 3304B: Sensor

[0179] 3304C: Sensor; 3304N: Sensor

[0180] 3306A: Filter 3306B: Filter

[0181] 3306C: Filter; 3306N: Filter

[0182] 3308A: Microlens array; 3308B: Microlens array

[0183] 3308C: Microlens array; 3308N: Microlens array

[0184] IL1: First incident light; IL2: Second incident light

[0185] SO: Sensor output signal, Sensor output ED: Calculation result

[0186] TL1: First teleportation beam; TL2: Second teleportation beam

[0187] P01: Pre-calculation result; S01: First modeling signal

[0188] S02: Second modeling signal; EN1: First numerical value

[0189] EN2: Second value; CS1: First control signal

[0190] CS2: Second control signal; DS1: Detection signal

[0191] DS2: Detection signal W1: Connection line

[0192] W2: Joint line W3: Joint line

[0193] W4: Joint line B1: Solder block

[0194] B2: Welding block B3: Welding block

[0195] B4: Solder block λ1: Wavelength

[0196] λ2: wavelength; λ3: wavelength

[0197] λN: Wavelength FD1: Readout area

[0198] FD2: Readout area; DEMOD1: Demodulation area

[0199] DEMOD2: Demodulation area; CLKP: Clock signal

[0200] CLKN: Clock signal; SH: Control signal

[0201] Msh1: Control transistor Msh2: Control transistor

[0202] RST: Control signal; Mrt1: Reset transistor

[0203] Mrt2: Reset transistor; BS: Control signal

[0204] Msf1: Source follower Msf2: Source follower

[0205] Mbt1: Column select transistor Mbt2: Column select transistor

[0206] C1: Capacitor C2: Capacitor

[0207] MTL: Metal VIA: Perforated Detailed Implementation

[0208] This application claims the benefit of three U.S. Provisional Patent Applications: No. 63 / 035,859, filed June 8, 2020; No. 63 / 079,452, filed September 16, 2020; and No. 63 / 173,488, filed April 11, 2021. The entire contents of these applications are incorporated herein by reference.

[0209] The components, systems, and methods of this application, as exemplary embodiments, use broadband optical wavelengths to sense objects and / or the operating environment, and adjust one or more functions accordingly. For example, the sensing device may include a photodetector unit configured to absorb first incident light at a first wavelength to generate a first detection signal, and absorb second incident light at a second wavelength to generate a second detection signal. In some embodiments, the photodetector unit may include a near-infrared or short-wave infrared sensor. In some embodiments, the photodetector unit may include germanium. The sensing device may be further configured to determine a calculation result based on the first and second detection signals. The sensing device may be further configured to perform adjustments to one or more functions associated with the sensing device based on the calculation result. For example, when skin (e.g., body tissue of a hand holding the sensing device) is detected, the sensing device may adjust from a low-power operating mode to a normal-power operating mode.

[0210] The components, systems, and methods of this application can provide a variety of technical effects and benefits. For example, to develop more broadband sensing applications, this application provides a sensing device with a broadband sensor to achieve novel environmental / contextual awareness (e.g., biometric identification, 3D reconstruction, gesture / skeleton detection, target feature analysis, or user context definition) and provide more practical applications (e.g., unlocking / payment confirmation, navigation, user interface (UI) / gesture monitoring, and wireless radio frequency signal / power optimization).

[0211] Furthermore, the components, systems, and methods of this application utilize reflectance data at various wavelengths to perform material analysis, thereby helping to improve the accuracy of object classification. For example, in some embodiments, reflectance data at various wavelengths can be used as input to a machine learning model to determine the material type of an object (e.g., wood, skin, etc.) and / or one or more material-related properties (e.g., blood sugar levels, etc.). In some embodiments, additional inputs, such as image data, point cloud data, or other data, can also be provided to the machine learning model to further improve object classification analysis.

[0212] Furthermore, the components, systems, and methods of this application can measure the reflectance data of target objects at various wavelengths, thereby performing optical spectral analysis to achieve functions similar to near-infrared spectral analysis, thus determining one or more characteristics of the target object, including its moisture content, carbohydrate content, sugar content, protein content, or fat content. Optical spectral analysis (e.g., near-infrared spectral analysis) can be used in the food industry to monitor and evaluate food items. For example, such optical spectral analysis can be used to determine whether a target object (e.g., a piece of fruit) is counterfeit. Low-cost image sensor-based near-infrared spectral analysis can be achieved holistically using a GeSi image sensor array with a monolithic bandpass filter.

[0213] The system and method of this application can also achieve more immersive augmented reality (AR) and virtual reality (VR) experiences. For example, after identifying the material type of an object, the interaction between the user and the object can be tailored to the material type in the AR / VR experience. This enables more realistic (e.g., based on physical laws) and more engaging interactions (e.g., based on software-defined rules rather than physical laws).

[0214] Furthermore, the system and method of this application contribute to improving the safety of sensing devices. For example, by improving the accuracy of object classification, downstream analysis, such as the performance of automated components (e.g., vehicles, robots, etc.), can be improved, thereby enhancing bystander safety (e.g., nearby pedestrians). Additionally, when implemented using a germanium-based broadband sensor according to the exemplary embodiments of this application, it can detect light wavelengths in the near-infrared (NIR, e.g., wavelengths from 780 nm to 1400 nm, or any similar wavelength range as defined for a particular application) and short-wave infrared (SWIR, e.g., wavelengths from 1400 nm to 3000 nm, or any similar wavelength range as defined for a particular application) spectra. This allows for the protection of bystander vision within the visible spectrum by limiting maximum permissible exposure (MPE) when using higher-power optical devices (e.g., lasers).

[0215] The following will describe in detail the exemplary embodiments of this application with reference to the accompanying drawings. Figure 1A sensing device 100 is illustrated, comprising a broadband sensor 102, a computing circuit 104, and an adjustment circuit 106. The broadband sensor 102 can absorb first incident light IL1 having a first wavelength to generate a first detection signal, and absorb second incident light IL2 having a second wavelength to generate a second detection signal. The first and second detection signals can be electrical signals generated by a photodetector within the broadband sensor 102. An exemplary portion of the broadband sensor 102 and its exemplary form according to this application will be referred to... Figure 10 and 11 Detailed explanation.

[0216] Furthermore, the broadband sensor 102 can output a sensor output signal SO based on the first detection signal and the second detection signal. In some embodiments, the sensor output signal SO can be a digital data stream, which may include information about the first detection signal and the second detection signal, such as depth information, calibration information, error correction code (ECC) information, numerical information, etc.

[0217] The calculation circuit 104 is coupled to the broadband sensor 102 and outputs a calculation result ED based on the sensor output signal SO. The adjustment circuit 106 is coupled to the calculation circuit 104 and performs adjustments based on the calculation result ED.

[0218] In some embodiments, the first wavelength and the second wavelength may be different, and may be 850nm / 905nm / 940nm / 1050nm / 1064nm / 1310nm / 1350nm / 1400nm / 1550nm and / or any other wavelength suitable for performing sensing.

[0219] In some embodiments, the broadband sensor 102 may be implemented using direct ToF, indirect ToF, and / or structured photosensitive technologies. For example, a direct ToF sensor may transmit pulsed light (e.g., from a laser, diode, etc.) to an object and detect the arrival time of the reflected pulses. For example, a photodetector may detect the reflected pulses, and auxiliary electronics may determine the distance to the object (or a portion thereof) based on the arrival time of the reflected pulses. In some embodiments, multiple light pulses may be transmitted, and multiple corresponding reflected pulses may be detected by photodetectors(s). In some embodiments, auxiliary electronics may perform analysis (e.g., histogram analysis) on the reflected pulses to determine a three-dimensional representation of the object (e.g., a point cloud).

[0220] In another example, an indirect ToF sensor can be used to emit modulated light pulses with a specific period. The period of the light wave can be used to determine the distance required for the emitted light wave to complete its entire period. For example, a sinusoidal or square wave can be emitted, and the distance to an object can be determined based on the phase difference (e.g., phase shift) between the output signal (e.g., emitted light) and the incoming signal (e.g., reflected light detected by a photodetector).

[0221] In some embodiments, the computing circuitry 104 may be implemented using an application-specific integrated circuit (ASIC), digital signal processing (DSP) processor, or general-purpose processor capable of performing artificial intelligence (AI) or deep learning computations (e.g., one or more machine learning models).

[0222] In some embodiments, the adjustment circuit 106 may be implemented in hardware, firmware, or software to adjust one or more functions of the sensing device 100 (e.g., turning the wireless radio frequency / power / display unit on / off). In other embodiments, the adjustment circuit 106 may be used to generate one or more output signals to another circuitry of the sensing device 100 to adjust one or more functions of the sensing device 100. In other embodiments, the adjustment circuit 106 may generate one or more output signals to another component to adjust one or more functions of the component.

[0223] Figure 2 A sensing device 200 is illustrated, including a broadband sensor 102, a computing circuit 104, and an adjustment circuit 106. The broadband sensor 102 can emit a first transmit light TL1 having a first wavelength and a second transmit light TL2 having a second wavelength. The broadband sensor 102 includes a light-emitting unit 160 (also referred to as a light source circuit) and a light-detecting unit 180. The light-emitting unit 160 may include multiple light sources (e.g., light sources 112 and 114), and the light-detecting unit 180 may include one or more light detectors (e.g., light detectors 108 and 110). In some embodiments, the one or more light detectors of the light-detecting unit 180 may be light detectors for three-dimensional (3D) depth sensing (e.g., i-TOF or d-TOF light detectors), proximity sensing, optical spectral analysis, two-dimensional (2D) sensing (e.g., two-dimensional IR imaging), or combinations thereof. Light detectors 108 and 110 may each be implemented using a single light detector or an array of light detector pixels (e.g., Figure 29A and 29B (One-dimensional or two-dimensional optical detector array).

[0224] In some embodiments, the first transmitting light TL1 and the second transmitting light TL2 are emitted by light source 112 and light source 114, respectively, wherein light source 112 and light source 114 may be lasers (e.g., VCSELs) or light-emitting diodes (LEDs).

[0225] In some embodiments, photodetector 108 may absorb first incident light IL1 having a first wavelength (e.g., 940 nm), and photodetector 110 may absorb second incident light IL2 having a second wavelength (e.g., 1350 nm). The sensor output signal SO may be a digital data stream, which may include depth information, calibration information, error correction code (ECC) information, numerical information, etc. The first incident light IL1 may be a portion of a first transmitted light TL1 reflected by object 140. The second incident light IL2 may be a portion of a second transmitted light TL2 reflected by object 140. In some embodiments, the light-absorbing material used in photodetector 108 may be silicon, and the light-absorbing material used in photodetector 110 may be germanium or a group III-V material. In other embodiments, photodetector 108 and photodetector 110 may use the same light-absorbing material (e.g., germanium or a group III-V material).

[0226] Since the reflectivity of the first wavelength and the reflectivity of the second wavelength depend on object 140, the value of the first detection signal detected by photodetector 108 may differ from the value of the second detection signal detected by photodetector 110. In some embodiments, calculation circuit 104 may calculate the ratio of the first value to the second value as a calculation result ED. In some embodiments, calculation circuit 104 may calculate the difference between the first value and the second value. The calculation result ED calculated by calculation circuit 104 can be provided to adjustment circuit 106 to perform further adjustments.

[0227] Figure 3 An embodiment of the sensing device 300 is illustrated. It utilizes a single photodetector 108 to absorb a first incident light IL1 and a second incident light IL2 at different time intervals. For example, in a first period t1-t2, the photodetector 108 can absorb the first incident light IL1 having a first wavelength (e.g., 940 nm), and in a second period t3-t4, the photodetector 108 can absorb the second incident light IL2 having a second wavelength (e.g., 1350 nm). Thus, the single photodetector 108 can be configured to simultaneously detect the first incident light IL1 and the second incident light IL2.

[0228] Figure 4An embodiment of the sensing device 400 includes a second photodetector unit 190 (e.g., a silicon-based RGB camera) to generate a two-dimensional image for improving object 140 recognition. The second photodetector unit 190 may include one or more two-dimensional photodetectors. Based on this architecture, the computing circuit 104 can obtain information from both the photodetector unit 180 and the photodetector unit 190. Therefore, the sensing device 400 can process more information to improve object 140 recognition. For example, the two-dimensional image detected by the photodetector unit 190 and the information from the first incident light IL1 and the second incident light IL2 detected by the photodetector unit 180 can be output as an SO signal and provided to the computing circuit 104. In some embodiments, the computing circuit 104 may use one or more machine learning models (e.g., AI) to detect and classify the object 140.

[0229] In some embodiments, when the photodetector unit 180 uses only a single photodetector 108, the photodetector 108 may absorb the first incident light IL1 and the second incident light IL2 at different times.

[0230] Figure 5 An embodiment of the sensing device 500 is illustrated. In this embodiment, sunlight emitted by the sun 502 is used as the transmitted light. Figure 5 As shown, sunlight includes a first transmitted light TL1 having a first wavelength (e.g., 940 nm) and a second transmitted light TL2 having a second wavelength (e.g., 1050 nm), which can be reflected by object 140 and then absorbed by light detection unit 180. Compared to the above embodiment, when sunlight is used as transmitted light TL1 and TL2, the light-emitting unit 160 can be selectively omitted. Similarly, light detection unit 180 may include one or more light detectors to detect the first transmitted light TL1 and the second transmitted light TL2. In some embodiments, sensing device 500 may also use ambient light as transmitted light, which is also reflected by object 140 to become the first incident light IL1 and the second incident light IL2. In embodiments where the first incident light IL1 and the second incident light IL2 are not modulated signals, the photodetector unit 180 can be used as a near-infrared (NIR) sensor or a short-wave infrared (SWIR) sensor to sense the values ​​of the first incident light IL1 and the second incident light IL2, so that the sensing device 500 can determine the material of the object 140 or continue to perform other applications.

[0231] Figure 6An embodiment of the sensing device 600 is illustrated. In this embodiment, a photodetector unit 190 may be added to generate a two-dimensional image, thereby improving the recognition effect. The photodetector unit 190 may include one or more two-dimensional photodetectors. Based on this architecture, the computing circuit 104 can obtain information from the photodetector unit 180 and from the photodetector unit 190. Therefore, the sensing device 600 can process more information, thereby improving the recognition of the object 140. In implementations where the first incident light IL1 and the second incident light IL2 are not modulated signals, the photodetector unit 180 can be used as a near-infrared (NIR) sensor or a short-wave infrared (SWIR) sensor to sense the values ​​of the first incident light IL1 and the second incident light IL2, enabling the sensing device 600 to determine the material of the object 140 or to continue with other applications.

[0232] With respect to any of the embodiments described herein, in some embodiments, the modulation frequencies of the first transmit light TL1 and the second transmit light TL2 may be dynamically changed. For example, in the first period t1-t2, the modulation frequencies of the first transmit light TL1 and the second transmit light TL2 may be 300MHz, and in the second period t3-t4, the modulation frequencies of the first transmit light TL1 and the second transmit light TL2 may be 50MHz.

[0233] In some embodiments, a single light source 112 with a dynamic modulation frequency (e.g., without using light source 114) may be used to emit the first transmission light TL1 and the second transmission light TL2.

[0234] Figure 7An embodiment of the sensing device 700 is illustrated. In this embodiment, the computing circuit 104 can perform one or more computer vision techniques (e.g., object recognition / classification, object tracking, etc.). For example, in some embodiments, the computing circuit 104 may include a pre-computation unit 1041, a first AI modeling unit 1042, a second AI modeling unit 1043, and a prediction unit 1044. In some embodiments, the pre-computation unit 1041 can perform integration or differentiation calculations on the sensor output signal SO and output a pre-computation result P01 to the first AI modeling unit 1042 (e.g., a first machine learning model). In other words, the pre-computation unit 1041 can obtain integration or differentiation information at different wavelengths, enabling the first AI modeling unit 1042 to perform image data analysis and output a first modeling signal S01. In some embodiments, the integration or differentiation information can be converted to the frequency domain (e.g., wavelet transform). This information can be a data seed with AI training data labels. Furthermore, the sensor output signal SO can also be provided to the second AI modeling unit 1043 (e.g., a second machine learning model), enabling the second AI modeling unit 1042 to perform image data analysis and output a second modeling signal S02. The prediction unit 1044 can be configured to predict certain states based on the first modeling signal S01 and / or the second modeling signal S02. For example, the prediction unit 1044 can predict the classification of object 140, the material of object 140, user behavior of sensing device 700, and / or health data. Finally, the prediction result generated by the prediction unit 1044 can be used as the calculation result ED and output to the adjustment circuit 106. Other types of machine learning models and network architectures can also be used to analyze the detection signals acquired by the two-dimensional and / or three-dimensional broadband sensors described herein in a similar manner.

[0235] The advantage of the state sensing device 700 according to the present application is that it can perform computer vision analysis of object 140 more accurately. For example, in conventional computer vision implementations, one or more machine learning models can use various segmentation techniques (e.g., bounding boxes) to perceive objects 140 in the environment from two-dimensional images. Similarly, three-dimensional data, such as data obtained by a ToF sensor (e.g., a LiDAR sensor), can be used to identify and track objects 140 in three-dimensional space. However, this computer vision technique classifies and tracks objects 140 based on their shape (e.g., segmentation boundaries in two-dimensional or three-dimensional space), and therefore objects with similar shapes to other objects are easily misclassified using this computer vision technique. This may lead to incorrect inferences about the expected behavior of object 140 in subsequent analysis. For example, classifying object 140 as a stationary object type (e.g., a statue) when it is not stationary (e.g., a pedestrian) may lead to incorrect predictions about the object's future movement in downstream analysis.

[0236] However, the sensing device according to the example of this application can incorporate additional information, such as data indicating material type, into the input data of the machine learning model, thereby improving the accuracy of computer vision analysis. For example, the sensor output SO may include incident light of various wavelengths (e.g., IL1, IL2) reflected by the object 140. The reflectivity of the incident light of various wavelengths (e.g., IL1, IL2) can indicate the material type of the object 140.

[0237] For example, Figure 8 This is a broadband application embodiment of the sensing device 800. For any embodiment described herein, the sensing device 800 can be a mobile phone or any other component with mobile communication capabilities, wherein the sensing device 800 can operate in various environments, such as being placed on a surface (e.g., a desktop) or held in the hand. According to one example, when the mobile phone (or other component 800) is placed on a desktop (e.g., object 140), the material type of object 140 can be wood. In this environment, a first transmitted light TL1 having a first wavelength (e.g., 940 nm) can have a first reflectivity (e.g., 90%) on object 140, and a second transmitted light TL2 having a second wavelength (e.g., 1550 nm) can have a second reflectivity (e.g., 85%). In this case, the first reflectivity and the second reflectivity are relatively close, and the first value of the first detection signal detected by the photodetector 108 and the second value of the second detection signal detected by the photodetector 110 are also relatively close. The calculation circuit 104 can obtain the calculation result ED between the first value and the second value. In one embodiment, the calculated result ED can be the ratio of a first value to a second value. Accordingly, the adjustment circuit 106 can perform adjustments to the sensing device 800 based on the calculated result ED (e.g., turning the wireless radio frequency / power / display unit / software application on / off). For example, when the material type of the object 140 is determined to be wood, the sensing device 800 can enter a low-power mode. At the same time, or alternatively, one or more applications running on the relevant components can be adjusted and / or turned off, such as stopping and / or turning off music playback applications or social media applications, which also helps to save power.

[0238] In another example, when the mobile phone (or other component 800) is held in the hand, the material type of the object 140 may be skin (e.g., body tissue). In this environment, a first transmitted light TL1 having a first wavelength (e.g., 940 nm) may have a first reflectivity (e.g., 80%) on the object 140, and a second transmitted light TL2 having a second wavelength (e.g., 1550 nm) may have a second reflectivity (e.g., 50%). In this case, the first reflectivity and the second reflectivity are not relatively close (within a threshold percentage of the other), and the first value EN1 and the second value EN2 are also not relatively close. The calculation circuit 104 may obtain a calculation result ED (e.g., ratio) of the first value EN1 and the second value EN2. In one embodiment, the calculation result ED may be the first value EN1 and the second value EN2. Upon detection of skin (or other type of material), the adjustment circuit 106 may perform adjustments to the sensing device 800 based on the calculation result ED (e.g., turning the radio frequency / power / display unit / software application on / off). For example, a software application for acquiring image data (such as fingerprint detection or verification) can be activated for the user based on skin detection. Simultaneously, or alternatively, one or more other applications (such as password prompts or music applications) can be activated or deactivated based on skin detection. In this way, the material type of the object 140 detected by the sensing device 800 can be used to determine what kind of adjustment the adjustment circuit 106 should perform. Other types of adjustments can be performed in a similar manner.

[0239] Therefore, it should be understood that the calculation result ED generated by the calculation circuit 104 will differ depending on the environment in which the sensing device 800 is used (e.g., when used with objects of different material types). Therefore, the adjustment circuit 106 is used to perform adjustments to the sensing device 800 (e.g., turning the wireless radio frequency / power / display unit on / off). For example, when the sensing device 800 is placed on a desktop, it can operate in a low-power mode, open or close applications, etc. Conversely, when the sensing device 800 is held in the hand, it can operate in a normal power mode, open or close applications, etc. Using this approach, the power consumption of the sensing device 800 can be dynamically adjusted, for example, in response to the detection of objects of different material types.

[0240] It should be understood that the configuration of adjustment circuit 106 for adjusting power supply is merely an example. Adjustment circuit 106 may continue to perform other operations based on design requirements (such as adjusting screen brightness, unlocking the screen, and access permissions, etc.).

[0241] According to another example, Figure 9This illustration depicts a broadband application embodiment of the sensing device 900. In this example, the sensing device 900 may be a health / medical component, wherein light sources 112 and 114 emit a first transmitted light TL1 having a first wavelength (e.g., 940 nm) and a second transmitted light TL2 having a second wavelength (e.g., 1350 nm) onto an object 140, respectively. For example, the object 140 may be skin (e.g., body tissue). Since most of the second transmitted light TL2 is absorbed by water-containing materials, but most of the first transmitted light TL1 is reflected, the calculation circuit 104 can calculate the blood glucose level based on the first incident light IL1 and the second incident light IL2 detected by the photodetectors 108 and 110. In this way, one or more characteristics of the object (e.g., the blood glucose level of body tissue) can be determined. Furthermore, after determining this characteristic, for example, after obtaining the blood glucose level, the value can be displayed (e.g., shown) on a panel / screen via the adjustment circuit 106.

[0242] In another example, one or more functions or the output of a medical component can be activated upon detection of a specific material. For instance, one or more characteristics of an object (e.g., skin blemishes, psoriasis patches on the human body) can be determined based on a first incident light IL1 and a second incident light IL2 detected by photodetectors 108 and 110, in a manner similar to that described herein. Subsequently, based on the detected characteristics, the functions of the medical component (e.g., laser, diagnostic tools) can be activated to treat specific parts of the object (e.g., applying laser treatment to diseased skin).

[0243] Besides mobile phones and health / medical components, the sensing device of this application can also be implemented as other components, such as wearable watches, keychains, earphones, portable computers, action cameras, drones, vehicle operating systems, and / or any other component including one or more broadband sensors. For example, in the context of autonomous vehicles, the broadband sensing device can achieve more accurate object classification and prediction through, for example, the vehicle operating system. For example, the sensor output signal SO of the broadband sensor 102 may include various data, such as image data from two-dimensional photodetectors (e.g., cameras) and / or three-dimensional photodetectors (e.g., ToF sensors, iToF sensors, structured light sensors, etc.), which may include incident light of various wavelengths (e.g., IL1, IL2). In some embodiments, the sensor output signal SO can be provided to one or more machine learning models that can use the data in the sensor output signal SO to more accurately classify object 140 and predict the trajectory of object 140. According to one example, the reflectivity of incident light of various wavelengths can indicate the material type of object 140, and this information can be used to determine the classification of object 140. As described herein, the reflectivity of incident light at different wavelengths can indicate whether a humanoid object 140 should be classified as a statue (e.g., an object made of stone, metal, or wood) or a pedestrian (e.g., an object made of skin or other body tissue, fabric, clothing, etc.). The vehicle operating system can use this method to determine whether the object 140 is likely to remain stationary or move in the environment, thereby performing more accurate trajectory prediction analysis of the object 140. Furthermore, trajectory prediction analysis can be used to determine the vehicle's motion planning, which can be implemented by the vehicle controller (e.g., adjustment circuit 106).

[0244] Sensing devices according to the exemplary embodiments of this application can also provide more immersive augmented reality (AR) and virtual reality (VR) experiences. For example, a personal operating device (e.g., a mobile phone) may include one or more broadband sensors 102 for detecting various objects 140 in the user's environment. As described herein, the broadband sensor 102 may provide a sensor output SO to a computing circuit 104, which in some embodiments may include one or more machine learning models (e.g., an AI modeling unit). The sensor output SO may include reflectance data at various wavelengths, thereby allowing the computing circuit 104 to determine the individual material type of the object 140 in the user's environment. Furthermore, the computing circuit 104 may determine one or more possible interaction methods between the user and the object 140, at least in part, based on the material type of the object 140 in the AR / VR environment. For example, in response to user interaction, an object 140 made of soft materials (such as flexible plastics, rubber, etc.) may bend, stretch, etc.; an object 140 made of fragile materials (such as glass, ceramics, etc.) may break or be destroyed; and / or an object made of hard materials (such as metal, wood, etc.) may deform, collapse, crack, etc. For example, the computing circuit 104 may, in response to user interaction, determine, at least in part, the augmented AR / VR experience with an object 140 as a calculation result ED based on the material type of the object 140, and the adjustment circuit 106 may present the augmented AR / VR experience to the user based on this calculation result ED.

[0245] Devices based on the embodiments of this application (e.g., mobile phones) may also provide optical spectral analysis. For example, a personal operating device (e.g., a mobile phone) may include one or more broadband sensors 102 for detecting various objects 140, such as food (e.g., fruit). As described herein, the broadband sensor 102 may provide sensor output SO to a computing circuit 104, which in some embodiments may include one or more hardware or software processing capabilities. The sensor output SO may include reflectance data at various wavelengths, which the computing circuit 104 may use to determine individual characteristics of the object 140 (e.g., the water content, carbohydrate and sugar content, protein content, and / or fat content of a piece of fruit). An adjustment circuit 106 may then use the characteristics of the object 140 to determine the authenticity of the food. For example, the adjustment circuit 106 may compare the measured sugar content of the fruit with a reference value; if the difference in sugar content exceeds a threshold, the adjustment circuit 106 may determine that the fruit is counterfeit.

[0246] Now refer to Figures 15 to 28 It shows several packaging implementations of a sensor 102 and / or its components. Although Figures 15 to 28 The packaging implementation and / or its components describe a particular implementation, but various packaging implementations may be adopted, rearranged, expanded, omitted and / or combined to produce more packaging implementations. Figure 15-28The sensor 102 package implementation and / or its components can be used in the sensing device 100 as described herein.

[0247] For example, Figure 15 This is a block diagram of an example sensor 102 according to the example configuration of this application. As shown, the sensor 102 may include a light-emitting unit 160, a light-detecting unit 180 (e.g., a three-dimensional light-detecting unit), and a controller 203.

[0248] The light-emitting unit 160 may include one or more light sources. For example, the light-emitting unit 160 may include a first light source 112 and a second light source 114. In some embodiments, the first light source 112 and the second light source 114 may be implemented on a discrete chip, but in other embodiments, the light sources 112 and 114 may be implemented on a single chip. In some embodiments, a single light source 112 may be included in the light-emitting unit 160. The light sources 112 / 114 may be, for example, lasers (e.g., VCSELs) or light-emitting diodes (LEDs), as disclosed herein.

[0249] In some embodiments, controller 203 (e.g., an analog or digital circuit system, or one or more processors) may be configured to generate a first control signal CS1 and a second control signal CS2 to control (e.g., drive) the light emitted by light sources 112 / 114. In some embodiments, the first light source 112 may be configured to emit a first transmitted light at a first wavelength, and the second light source 114 may emit a second transmitted light at a second wavelength. In some embodiments, a single light source 112 may be configured to emit the first transmitted light at a first wavelength and the second transmitted light at a second wavelength.

[0250] In some embodiments, the photodetector unit 180 may include a first photodetector 108 and a second photodetector 110. In some embodiments, the first photodetector 108 and the second photodetector 110 may be implemented on a discrete chip, but in other embodiments, the photodetectors 108 / 110 may be implemented on a single chip. The photodetectors 108 / 110 may be configured to provide detection signals DS1 and DS2 to the controller 203. For example, the photodetectors 108 / 110 generate detection signals DS1 and DS2 after absorbing incident light IL1 and IL2 (e.g., transmitted light TL1 and TL2 reflected by object 140). In some embodiments, both the first incident light IL1 and the second incident light IL2 may be absorbed by a single photodetector 108, as disclosed herein. In some embodiments, the photodetectors 108 / 110 may include, for example, silicon, germanium, and / or group III-V materials.

[0251] The controller 203 may be configured to output a sensor output signal SO. For example, in some embodiments, the sensor output SO may include individual values ​​EN1 and EN2 of the incident light IL1 and IL2, as disclosed herein.

[0252] Now refer to Figure 16 This is a block diagram of an example sensor 102 according to the present application. The sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings.

[0253] As shown in the figure, sensor 102 includes a first chip 401 for implementing a light-emitting unit (e.g., light source 112 / 114), a second chip 402 for implementing a light-detecting unit (e.g., photodetector 108 / 110), and a third chip 403 for implementing a controller 203. As shown, each chip 401, 402, and 403 is a separate chip, coupled together to form sensor 102. As shown, light source 112 is coupled to controller 203 via bonding line W1, light source 114 is coupled to controller 203 via bonding line W2, photodetector 108 is coupled to controller 203 via bonding line W3, and photodetector 110 is coupled to controller 203 via bonding line W4. In some embodiments, each light source 112 / 114 and each photodetector 108 / 110 may be implemented on a separate chip. Furthermore, as shown in the figure, in some embodiments, each chip 401, 402 and 403 may be implemented on a common plane.

[0254] Now refer to Figure 17 This is a block diagram of an example sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, a first light source 112 is implemented on a first chip 401, a first photodetector 108 is implemented on a second chip 402, a controller 203 is implemented on a third chip 403, a second light source 114 is implemented on a fourth chip 404, and a second photodetector 110 is implemented on a fifth chip 405. Therefore, as shown, sensor 102 is implemented in a five-chip architecture, wherein each light source 112 / 114 and each photodetector 108 / 110 is implemented on a separate chip. In some embodiments, light sources 112 / 114 may be integrated into a single chip 401 (e.g., as shown in the figure). Figure 16 As shown), and the photodetectors 108 / 110 can be integrated into a single chip 402 (e.g., as shown). Figure 16 (As shown). Furthermore, as shown in the figure, the light source 112 / 114 and the photodetector 108 / 110 are respectively wired to the controller 203 via bonding wires W1, W2, W3 and W4. Figure 17A stacked architecture implementation of sensor 102 is further described, wherein chips 401, 402, 404, and 405 are stacked on chip 403. Chips 401, 402, 404, and / or 405 can be coupled to chip 403 using any suitable method, such as epoxy bonding, mechanical fastening, etc. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane. Figure 17 The stacked architecture can reduce the space requirements of the package (e.g., reduce package overhead / overall area).

[0255] Now refer to Figure 18 This is a block diagram of an example sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a first chip 401 for implementing a light-emitting unit 160 (e.g., light source 112 / 114), a second chip 402 for implementing a light-detecting unit (e.g., photodetector 108 / 110), and a third chip 403 for implementing a controller 203. As shown, each chip 401, 402, and 403 is a separate chip, coupled to each other to form sensor 102. As shown, the first chip 401 and the second chip 402 are coupled to the third chip 403 using a flip-chip connection. As shown, solder blocks B1 and B2 enable the light source 112 / 114 to be coupled to the controller 203, and solder blocks B3 and B4 enable the photodetector 108 / 110 to be coupled to the controller 203. In some embodiments, each light source 112 / 114 and each photodetector 108 / 110 can be implemented on a discrete chip (e.g., 401, 402, 404, 405), which can be flip-chip bonded to the chip 403 implementing the controller 203 via solder pads B1-B4. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane. Implementing sensor 102 using flip-chip bonding also reduces packaging space requirements (e.g., reducing packaging overhead / overall area). Furthermore, since flip-chip bonding eliminates the need for… Figure 17 The wire bonding connections W1-W4 allow for a lower package space requirement compared to wire bonding and stacking architectures.

[0256] Now refer to Figure 19This is a block diagram of an exemplary sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a first chip 401 for implementing a light-emitting unit 160 (e.g., light source 112 / 114), a second chip 402 for implementing a light-detecting unit (e.g., light detector 108 / 110), and a third chip 403 for implementing a controller 203. As shown, each chip 401, 402, and 403 is a separate chip, coupled to each other to form sensor 102. As shown, the first chip 401 is wire-bonded to the third chip 403 using bonding wires W1 and / or W2, and the second chip 402 is stacked on top of the third chip 403 and bonded using bonding wires W3 and / or W4. In some embodiments, each light source 112 / 114 and each photodetector 108 / 110 may be implemented on separate chips (e.g., 401, 402, 404, 405), which may be stacked and / or bonded to a third chip 403 implementing the controller 203. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane, using... Figure 19 The stacked architecture can reduce the space requirements of the package (e.g., reduce package overhead / overall area).

[0257] Now refer to Figure 20 This is a block diagram of an example sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a first chip 401 for implementing a first light source 112, a second chip 402 for implementing a first photodetector 108, a third chip 403 for implementing a controller 203, a fourth chip 404 for implementing a second light source 114, and a fifth chip 405 for implementing a second photodetector 110. Therefore, as shown, sensor 102 is implemented on a five-chip architecture, wherein each light source 112 / 114 and each photodetector 108 / 110 is implemented on a separate chip. In some embodiments, light sources 112 / 114 may be integrated into a single chip 401 (e.g., as shown in the figure). Figure 16 As shown), and the photodetectors 108 / 110 can be integrated into a single chip 402 (e.g., as shown). Figure 16As shown in the figure, the first chip 401, which implements the first light source 112, is wire-bonded to the third chip 403 via bonding wire W1; the fourth chip 404, which implements the second light source 114, is wire-bonded to the third chip 403 via bonding wire W2; the second chip 402, which implements the photodetector 108, is flip-chip bonded to the third chip 403 via solder ball B3; and the fifth chip 405, which implements the photodetector 110, is flip-chip bonded to the third chip 403 via solder ball B4. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane, using... Figure 20 The stacked architecture can reduce the space requirements of the package (e.g., reduce package overhead / overall area).

[0258] Now refer to Figure 21 This is a block diagram of an exemplary sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a first chip 401 for implementing a light-emitting unit 160 (e.g., light source 112 / 114), a second chip 402 for implementing a light-detecting unit (e.g., light detector 108 / 110), and a third chip 403 for implementing a controller 203. As shown, each chip 401, 402, and 403 is a discrete chip, coupled to each other to form sensor 102. As shown, the first chip 401 is wire-bonded to the third chip 403 using bonding wires W1 and / or W2, and the second chip 402 is wafer-bonded to the third chip 403 using vias 601, 602, 603, and / or 604. For example, vias 601-604 may be interconnects between the second chip 402 and the third chip 403. Furthermore, via 460 allows bonding wires W1 and / or W2 to connect the first chip 401 to the third chip 403. In some embodiments, the light sources 112 / 114 and the photodetectors 108 / 110 can be implemented on discrete chips (e.g., 401, 402, 404, 405), which can be stacked and / or bonded to the third chip 403 for implementing the controller 203. In various embodiments, different wafer bonding technologies can be used, such as direct bonding, surface activation bonding, plasma activation bonding, anodic bonding, eutectic alloy bonding, glass frit bonding, adhesive bonding, thermocompression bonding, reactive bonding, transient liquid phase diffusion bonding, and / or other wafer bonding technologies. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane, using... Figure 21 The wafer bonding architecture can reduce the space requirements of the package (e.g., reduce package overhead / overall area).

[0259] Now refer to Figure 22 This is a block diagram of an example sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a first chip 401 for implementing a light-emitting unit 160 (e.g., light source 112 / 114), a second chip 402 for implementing a light-detecting unit (e.g., photodetector 108 / 110), and a third chip 403 for implementing a controller 203. As shown, each chip 401, 402, and 403 is a discrete chip, coupled to each other to form sensor 102. As shown, the first chip 401 is wafer-bonded to the second chip 402 using vias 605 and / or 606. Vias 460, 601, and / or 602 allow the first chip 401 to be coupled to the third chip 403 implementing the controller 203. Furthermore, the second chip 402 can be wafer-bonded to the third chip 403 via vias 603 and / or 604. In some embodiments, each light source 112 / 114 and each photodetector 108 / 110 can be implemented on discrete chips (e.g., 401, 402, 404, 405), which can be stacked and / or wafer-bonded to the third chip 403 for implementing the controller 203. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane, using... Figure 22 The wafer bonding architecture can reduce the space requirements of the package (e.g., reduce package overhead / overall area).

[0260] Now refer to Figure 23 This is a block diagram of an example sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a first chip 401 for implementing a light-emitting unit 160 (e.g., light source 112 / 114), a second chip 402 for implementing a light-detecting unit (e.g., photodetector 108 / 110), and a third chip 403 for implementing a controller 203. As shown, each chip 401, 402, and 403 is a separate chip, coupled to each other to form sensor 102. As shown, the first chip 401 is flip-chip bonded to the second chip 402 using solder balls B1 and B2. Vias 460, via 601, and / or via 602 allow the first chip 401 to be coupled to the third chip 403 for implementing the controller 203. Furthermore, the second chip 402 can be wafer-bonded to the third chip 403 via vias 603 and / or 604. In some embodiments, each light source 112 / 114 and each photodetector 108 / 110 can be implemented on discrete chips (e.g., 401, 402, 404, 405), which can be stacked and / or wafer-bonded to the third chip 403 implementing the controller 203. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane, using... Figure 23 The flip-chip and wafer bonding architecture can reduce the space requirements of the package (e.g., reduce package overhead / overall area).

[0261] Now refer to Figure 24 This is a block diagram of an example sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a single chip 2401 (e.g., a shared chip, a monolithic chip, etc.) for implementing a light-emitting unit 160 (e.g., a light source 112 / 114), a light-detecting unit (e.g., a light detector 108 / 110), and a controller 203. For example, during manufacturing, each light-emitting unit (e.g., a light source 112 / 114), light-detecting unit (e.g., a light detector 108 / 110), and controller 203 may be fabricated on a single chip. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane, using... Figure 24 The monolithic architecture can reduce the space requirements of the package (e.g., reduce package overhead / overall area).

[0262] Now refer to Figure 25 This is a block diagram of an exemplary sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a first chip 2501 for implementing a light detection unit (e.g., light detector 108 / 110) and a controller 203. For example, during manufacturing, the light detection unit (e.g., light detector 108 / 110) and controller 203 may be fabricated on chip 2501. Furthermore, a second chip 2502 may be used to implement a light-emitting unit (e.g., light source 112 / 114). As shown, the second chip 2502 may be stacked on top of the first chip 2501, and the second chip 2502 may be coupled to the first chip 2501 using bonding wires W1 and / or W2. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane, using... Figure 25 The combination of stacking and monolithic architecture can reduce the space requirements of the package (e.g., reduce package overhead / overall area). In some embodiments, a similar architecture may be used, including implementing the photodetector 108 / 110 on the second chip 2502 and the light source 112 / 114 on the first chip 2501.

[0263] Now refer to Figure 26This is a block diagram of an exemplary sensor 102 according to the present application. Sensor 102 may be, for example, the sensor 102 depicted in any of the accompanying drawings. As shown, sensor 102 may include a first chip 2601 implementing a photodetector unit (e.g., photodetector 108 / 110) and a controller 203. For example, during manufacturing, the photodetector unit (e.g., photodetector 108 / 110) and controller 203 may be fabricated on chip 2601. Furthermore, a second chip 2602 may be used to implement a light-emitting unit (e.g., light source 112 / 114). As shown, the second chip 2602 may be flip-chip bonded to the first chip 2601 via solder balls B1 and / or B2. Compared to Figure 16 The diagram shows all chips 401, 402, 403 (and / or 404, 405) arranged on a common plane, using... Figure 26 The combination of flip-chip and monolithic architectures can reduce package space requirements (e.g., reduce package overhead / overall area). In some embodiments, a similar architecture may be used, including implementing the photodetector 108 / 110 on the second chip 2602 and the light source 112 / 114 on the first chip 2601.

[0264] Now refer to Figure 27 The illustrated example is a photodetector circuit 202. As shown, the photodetector circuit 202 may include a first photodetector 108 and a second photodetector 110. In some embodiments, a single photodetector 108 / 110 may be implemented in a single photodetector circuit 202.

[0265] As shown in the figure, the first photodetector 108 can be made of silicon, and the second photodetector 110 can be made of germanium, to absorb the first incident light IL1 and the second incident light IL2, respectively. As described herein, the first incident light IL1 and the second incident light IL2 can be of different wavelengths. Furthermore, as shown in the figure, the silicon and germanium photodetectors 108 / 110 can be integrated and formed on a shared substrate 2701, such as silicon. In other embodiments, the photodetectors 108 / 110 can be implemented using other materials, such as germanium or other group III-V materials. In some embodiments, the silicon, germanium, and / or group III-V materials used to implement the photodetectors 108 / 110 can be partially recessed, fully recessed, and / or embedded in the silicon substrate 2701. Compared to manufacturing the photodetectors 108 / 110 separately, using... Figure 27 The photodetector 202 can reduce the overall area of ​​the photodetector circuit 202 and save manufacturing costs.

[0266] Now refer to Figure 28This is a cross-sectional view of an example light source 112 / 114. Light source 112 / 114 can be, for example, the light source 112 / 114 shown in any of the figures herein. As shown, light source 112 / 114 may include a silicon substrate 2801, an N+III-V material contact layer 2802 (e.g., N+ gallium arsenide (GaAs)), an N-III-V material cladding layer 2803 (e.g., N-aluminum gallium arsenide (AlGaAs)), a quantum dot (QD) material layer 2804 (e.g., InGaAs), a P-III-V material cladding layer 2805 (e.g., P-AlGaAs), and a P+III-V material contact layer 2806 (e.g., P+AlGaAs).

[0267] Now refer to Figure 29A The figure depicts a one-dimensional photodetector array 2900A. The one-dimensional photodetector array 2900A includes a substrate 2902 (e.g., a silicon substrate) and N photodetectors 2904 (e.g., germanium photodetectors) arranged in a one-dimensional array, where N can be any positive integer. The one-dimensional photodetector array 2900A can be implemented, for example, with reference to... Figure 2 The first photodetector 108 or the second photodetector 110 are described. In some other embodiments, the one-dimensional photodetector array 2900A may be divided into multiple parts to simultaneously include the first photodetector 108 and the second photodetector 110.

[0268] Figure 29B A two-dimensional photosensor array 2900B is depicted. The two-dimensional photosensor array 2900B includes a substrate 2902 (e.g., a silicon substrate) and M×N photosensors 2904 (e.g., germanium photosensors) arranged in a two-dimensional array, where M and N can be any positive integers. The two-dimensional photosensor array 2900B can be implemented, for example, with reference to... Figure 2 The first photodetector 108 or the second photodetector 110 are described. In some other embodiments, the two-dimensional photodetector array 2900B may be divided into multiple parts to simultaneously include the first photodetector 108 and the second photodetector 110.

[0269] Figure 30 A front-side-incident (FSI) photodetector array 3000 is described. The FSI photodetector array 3000 can be a one-dimensional array (e.g., Figure 29A (as shown) or two-dimensional array (such as) Figure 29B(As shown). The FSI photodetector array 3000 includes a substrate 3002 (e.g., a silicon substrate) and N photodetectors 3004 (e.g., germanium photodetectors). The FSI photodetector array 3000 further includes N optical filters 3006, each corresponding to one of the N photodetectors 3004. In some embodiments, each of the N optical filters 3006 is a bandpass filter, allowing incident light with wavelengths (e.g., λ1) within the corresponding wavelength range to pass through, while blocking light with wavelengths (e.g., λ2) outside the corresponding wavelength range. The optical filters 3006 may be implemented using absorbing materials, multilayer coatings, or in-plane periodic / aperiodic gratings. According to one example, bandpass filters 3006A, 3006B, 3006C, ..., and 3006N may allow light with wavelengths of λ1, λ2, λ3, ..., and λN to pass through, respectively. Broadband light with wavelengths from λ1 to λN is fed into an FSI photodetector array 3000, and then bandpass filters 3006A, 3006B, 3006C, ..., 3006N filter this broadband light, so that each sensor 3004A, 3004B, 3004C, and 3004N receives light with wavelengths of λ1, λ2, λ3, and λN, respectively. The signals measured at different wavelengths can be used, for example, for optical spectral analysis.

[0270] Now refer to Figure 31 The image depicts an FSI photodetector array 3100. The FSI photodetector array 3100 is similar to the FSI photodetector array 3000 and further includes a microlens array 3108 for guiding (e.g., focusing) incident light onto the photodetector 3104. The microlens array 3108 may be made of silicon, oxide, polymer, or any other suitable material. The FSI photodetector array 3100 may include a separator 3110 to pre-form a flat surface over the filter 3106 prior to forming the microlens array 3108.

[0271] Figure 32 A back-side incident (BSI) photodetector array 3200 is depicted. The BSI photodetector array 3200 can be a one-dimensional array (e.g., Figure 29A (as shown) or two-dimensional array (such as) Figure 29B(As shown). The BSI photodetector array 3200 includes a substrate 3202 (e.g., a silicon substrate) and N photodetectors 3204 (e.g., germanium photodetectors). The BSI photodetector array 3200 further includes N optical filters 3206 formed on the back side of the substrate 3202, each corresponding to one of the N photodetectors 3204. In some embodiments, each N optical filter 3206 is a bandpass filter that allows incident light of wavelength (e.g., λ1) within a corresponding wavelength range to pass through, and blocks light of wavelength (e.g., λ2) outside the corresponding wavelength range. The optical filters 3206 may be implemented using absorbing materials, multilayer coatings, or in-plane periodic / aperiodic gratings. According to one example, bandpass filters 3206A, 3206B, 3206C, ..., and 3206N may allow light of wavelengths λ1, λ2, λ3, ..., and λN to pass through. Broadband light, including wavelengths λ1 to λN, can be fed into a BSI photodetector array 3200. Bandpass filters 3206A, 3206B, 3206C, ..., 3206N can then filter this broadband light, allowing each sensor 3204A, 3204B, 3204C, and 3204N to receive light with wavelengths of λ1, λ2, λ3, and λN, respectively. The signals measured at different wavelengths can be used, for example, for optical spectral analysis.

[0272] Figure 33 A BSI photodetector array 3300 is depicted. The BSI photodetector array 3300 is similar to the BSI photodetector array 3200 and further includes a microlens array 3308 for guiding (e.g., focusing) incident light onto the photodetector 3304. The microlens array 3308 may be made of silicon, oxide, polymer, or any other suitable material. The BSI photodetector array 3300 may include a separator 3210 to form a flat surface over a filter 3206 prior to forming the microlens array 3308.

[0273] Now refer to Figure 10 This is a block diagram of an example broadband sensor 1000. The broadband sensor 1000 can be, for example, an iToF sensor. As shown, the broadband sensor 1000 may include a voltage regulator 1005, a temperature sensor 1010, an I2C (Inter-Integrated Circuit) controller 1015, a PLL (Phase Locked Loop) 1020, a timing generator 1025, a modulator 1030, a programmable delay 1035, a pixel array 1040, a readout 1045, an ADC 1050, a signal processor 1055, a memory 1060, and an output interface 1065.

[0274] A voltage regulator 1005 adjusts the voltage of the broadband sensor 1000. For example, in some embodiments, the broadband sensor 1000 may use two voltage domains, such as VDDH (e.g., >2.7V) and VDDL (e.g., about 1.8V). A temperature sensor 1010 may be used for calibration depth and power control of the broadband sensor 1000. An I2C (Integrated Bus) controller 1015 enables signal communication between components of the broadband sensor 1000. I2C control signals may provide clock synchronization, etc. A PLL (Phase-Locked Loop) 1020 may receive a reference clock signal and generate one or more output signals related to the phase of the reference clock. For example, in some embodiments, the PLL may generate a four-phase system clock for modulation 1030 and demodulation. In some embodiments, the modulation frequency may be between 10 and 500 MHz. A timing generator 1025 may control and adjust the clock signal from the PLL according to preset integration time and different operating modes. In some embodiments, a programmable delay 1035 may be added to the lighting driver path. The reflected light (e.g., IL1, IL2, etc.) can then be received by the pixel array 1040. In some embodiments, the pixel array may be a 240x180 pixel array. In other embodiments, other suitable pixel resolutions may be used. The readout 1045 may bridge the pixel array 1040 with the analog-to-digital converter (ADC) 1050. In some embodiments, the readout 1045 may include multiple sample-and-hold circuits and buffers. In some embodiments, each pixel in the pixel array 1040 may be read twice to reduce noise. In some embodiments, the ADC 1050 may have a SAR architecture. The output of the ADC may be further processed by the signal processor 1055 and memory 1060 before being provided to the output interface 1065. The output interface 1065 can then provide a sensor output (SO), for example, via a MIPI interface.

[0275] In some implementations, the pixel array 1040 may be implemented on a germanium-on-silicon (Ge-on-Si) platform to improve absorption efficiency and increase modulation speed. Figure 11 This displays an example pixel region cross-section and an example clock signal suitable for use in a pixel array 1040 on a silicon-on-germanium platform. Figure 11 The exemplary pixel array employs a germanium-on-silicon architecture to facilitate the absorption of wavelengths in the NIR and SWIR ranges. This achieves a better signal-to-noise ratio (SNR) while remaining within the maximum permissible exposure limit (MPE), for example, by enabling a light source with increased power. As an example, such as... Figure 11As shown, the top wafer (e.g., a germanium-on-silicon (Ge-on-Si) platform) can be bonded to the bottom wafer (e.g., silicon) via a wafer bonding interface and / or a through-hole via (VIA) and a metal MTL. Clock signals CLKP and CLKN can be applied to demodulation regions DEMOD1 and DEMOD2, respectively. The readout (e.g., 1045) is coupled to the readout region FD1 and includes a control transistor Msh1, a reset transistor Mrt1, a capacitor C1, a source follower Msf1, and a column select transistor Mbt1. Furthermore, the control signal SH controls the control transistor Msh1, the control signal RST controls the reset transistor Mrt1, and the control signal BS controls the column select transistor Mbt1. Similarly, the readout (e.g., 1045) is coupled to the readout region FD2 and includes a control transistor Msh2, a reset transistor Mrt2, a capacitor C2, a source follower Msf2, and a column select transistor Mbt2.

[0276] Now refer to Figure 12 The method described herein is an exemplary method 1200 according to the exemplary form of this application. One or more portions of method 1200 may be implemented by an operating system, which includes one or more operating components and / or components thereof, such as the operating system described with reference to other figures (e.g., sensing devices 100-900; broadband sensors and portions thereof 1000, 1100; operating systems 1300, 1400, etc.). Each individual portion of method 1200 may be executed by any (or any combination of) one or more operating components, sensing devices and / or broadband sensors or portions thereof. Furthermore, one or more portions of method 1200 may be implemented as algorithms of hardware components on the aforementioned components. Figure 12 The steps performed in a specific order are described for illustrative and discussion purposes. Those skilled in the art will understand from the disclosure herein that the steps of any method discussed herein can be adjusted, rearranged, expanded, omitted, combined, and / or modified in various ways without departing from the scope of this application. Figure 12 The explanations of the steps / items described with reference to other systems and diagrams are for illustrative purposes only and are not restrictive. One or more parts of method 1200 may be performed additionally or alternatively by other systems.

[0277] In step 1202, method 1200 may include emitting a first transmit light having a first wavelength. For example, in some embodiments, a light source (e.g., a VCSEL laser, an LED, etc.) may transmit the first transmit light TL1. In some embodiments, the first wavelength may be, for example, less than a threshold wavelength (e.g., <1050 nm).

[0278] In step 1204, method 1200 may include emitting a second transmit light having a second wavelength. For example, in some embodiments, a light source (e.g., a VCSEL laser, LED, etc.) may transmit the second transmit light TL2. In some embodiments, the second wavelength may be, for example, greater than a threshold wavelength (e.g., >1050 nm). In some embodiments, this light source may be the same as the light source emitting the first transmit light TL1, but in other embodiments, the two may be separate light sources. In some embodiments, the first wavelength and the second wavelength may be different wavelengths (e.g., 850 nm, 905 nm, 940 nm, 1050 nm, 1064 nm, 1310 nm, 1350 nm, 1400 nm, 1550 nm, etc.).

[0279] In step 1206, method 1200 may include absorbing first incident light having a first wavelength to generate a first detection signal. For example, in some embodiments, the photodetector unit may absorb incident light IL1 reflected by the object when the first transmitted light TL1 illuminates the object. In some embodiments, the photodetector unit may be a two-dimensional or three-dimensional photodetector unit. In some embodiments, the incident light IL1 may be reflected light emitted by a separate light source (e.g., sunlight, ambient light, etc.).

[0280] In step 1208, method 1200 may include absorbing a second incident light having a second wavelength to generate a second detection signal. For example, in some embodiments, the photodetector unit may absorb incident light IL2 reflected by the object when the second transmitted light TL2 illuminates the object. In some embodiments, the photodetector unit may be a two-dimensional or three-dimensional photodetector unit. In some embodiments, the second incident light having a second wavelength may be the same as the photodetector unit that absorbs the first incident light, but in other embodiments, the second incident light may be detected using a split-light detection unit. In some embodiments, the incident light IL2 may be reflected light emitted by a split light source (e.g., sunlight, ambient light, etc.).

[0281] In step 1210, method 1200 may include determining a calculation result based on the first detection signal and the second detection signal. For example, in some embodiments, a calculation circuit may calculate the ratio between a first value of the first detection signal and a second value of the second detection signal.

[0282] In some embodiments, determining the calculation result based on the first and second detection signals may include performing integration or differentiation calculations to output a pre-calculated result (e.g., step 1212). For example, in some embodiments, a pre-calculation unit may obtain integration or differentiation information at different wavelengths, enabling one or more machine learning models (e.g., an AI modeling unit) to perform image analysis. In some embodiments, the integration or differentiation information may be converted to the frequency domain (e.g., wavelet transform). In some embodiments, the pre-calculated result may be used as a training data seed, as detailed herein.

[0283] In step 1214, method 1200 may include performing image data analysis by an AI modeling unit to output a modeling signal. For example, in some embodiments, the sensor output signal SO may be used by one or more machine learning models (e.g., a second AI modeling unit) to perform image data analysis, thereby outputting a modeling signal.

[0284] In step 1216, method 1200 may include predicting the state of an object as a calculation result. For example, in some embodiments, the prediction unit may predict object classification, object material, user behavior of sensing devices, health data, and / or other predictive information as described herein.

[0285] In step 1218, method 1200 may include determining an output based on the calculation result to perform adjustments on one or more functions of a component. For example, in various embodiments, adjustments may include changing the device's operating mode (e.g., low power mode, normal power mode, display on / off, etc.), displaying information (e.g., health status data, etc.), adjusting the AR / VR environment (e.g., object handling), performing tasks (e.g., performing motion planning, unlocking payment confirmation, navigation, industrial manufacturing, opening / closing applications, etc.), and / or performing other adjustments as described herein.

[0286] In step 1220, method 1200 may include adjusting a component. For example, various control circuits (e.g., controllers, displays, etc.) may be configured to perform adjustments on the component.

[0287] Figure 13According to an exemplary embodiment of this application, an exemplary system 1300 is described, which includes various means for performing operations and functions. These various means can execute the methods and programs of this application. For example, an operating system 1300 may include a light-emitting unit 1302, a light-detecting unit 1304, a computing unit 1306, a pre-computation unit 1308, an AI modeling unit 1310, a prediction unit 1312, an image analysis unit 1314, a material determination unit 1316, an adjustment unit 1318, an implementation unit 1320, and / or other means for performing the operations and functions described herein. In some embodiments, one or more of the above units may be implemented separately. In some embodiments, one or more units may be part of or included in one or more other units. Such means may include a processor, a microprocessor, a graphics processing unit, logic circuits, dedicated circuits, application-specific integrated circuits, programmable array logic, field-programmable gate arrays (FPGAs), controllers, microcontrollers, and / or other suitable hardware. The means may also, or may be modified to, include software control means implemented as a processor or logic circuit system, for example. The device may include memory, or be capable of otherwise accessing memory, such as one or more non-transitory computer-readable storage media, such as random access memory, read-only memory, electronically erased programmable read-only memory, erased programmable read-only memory, flash / other memory components, data cache, database and / or other suitable hardware.

[0288] The device is programmed to execute one or more algorithms to achieve the operations and functions described herein. For example, the device (e.g., light-emitting unit 1302, etc.) may be configured to emit light, such as light of various wavelengths. In some embodiments, light-emitting unit 1302 may include a laser (e.g., VCSEL), a light-emitting diode (LEDS), and / or other types of light-emitting unit 1302. In some embodiments, light-emitting unit 1302 may emit (e.g., transmit) modulated light, delayed light, and / or other types of light, as described herein.

[0289] The device (e.g., photosensing unit 1304) may be configured to detect light, such as incident light of various wavelengths (e.g., reflected light). In some embodiments, photosensing unit 1304 may be a two-dimensional and / or three-dimensional photosensing unit 1304. In some embodiments, photosensing unit may be an NIR or SWIR photosensing unit capable of absorbing light with wavelengths greater than a threshold wavelength (e.g., 1050 nm). In some embodiments, photosensing unit may include germanium or a III-V group. In some embodiments, photosensing unit 1304 may be configured to detect reflected light, such as reflected transmitted light, reflected sunlight, reflected ambient light, and / or other types of light, as described herein.

[0290] The device (e.g., computing unit 1306) can be configured to determine a calculation result ED based on a sensor output SO. For example, the photodetector unit 1304 can absorb the incident light and then provide the sensor output SO to the computing unit 1306.

[0291] The device (e.g., pre-calculation unit 1308) may be configured to determine a pre-calculation result P01 based on a first detection signal and a second detection signal. For example, in some embodiments, the pre-calculation unit 1308 may perform integration or differentiation calculations on the sensor output signal SO and output the pre-calculation result P01 to one or more machine learning models (e.g., an AI modeling unit). In some embodiments, the integration or differentiation information may be converted to the frequency domain (e.g., wavelet transform).

[0292] The device (e.g., AI modeling unit 1310) can perform image data analysis and output modeling signals S01, S02, etc. For example, in some embodiments, AI modeling unit 1310 can perform one or more computer vision analyses, as described herein.

[0293] A device (e.g., prediction unit 1312) can predict a specific state of an object based on modeling signals from other devices (e.g., pre-computation unit 1308 and / or AI modeling unit 1310). For example, in various implementations, prediction unit 1312 can predict object classification and object material, user behavior, health data, object trajectory, AR / VR interaction, and / or other characteristics as described herein.

[0294] The device (e.g., image analysis unit 1314) can analyze image data, such as sensor output data from two-dimensional or three-dimensional light detection unit 1304. In some embodiments, image analysis unit 1314 may be implemented as part of computing unit 1306, AI modeling unit 1310, prediction unit 1312, material determination unit 1316, and / or other devices. In some embodiments, image analysis unit 1314 may be one or more machine learning models.

[0295] The device (e.g., material determination unit 1316) may be configured to determine the material type of an object. For example, in some embodiments, the material determination unit 1316 may be trained to determine the material type of an object based on the reflection values ​​of incident light at various wavelengths. In some embodiments, the material determination unit 1316 may be implemented as part of a computing unit 1306, an AI modeling unit 1310, a prediction unit 1312, an image analysis unit 1314, and / or other devices. In some embodiments, the material determination unit 1316 may be one or more machine learning models.

[0296] The device (e.g., adjustment unit 1318) may be configured to determine an adjustment based on a calculation result. For example, in various embodiments, the adjustment may include adjusting one or more functions of the sensing device, such as an operating mode, a display mode or information status, an interaction, an action, and / or other functions as described herein.

[0297] The device (e.g., implementation unit 1320) may be configured to implement one or more adjustments determined by adjustment unit 1318. For example, various controllers may be configured to perform adjustments.

[0298] Now refer to Figure 14 The figure depicts an exemplary operating system 1400 according to the exemplary embodiments of this application. Exemplary system 1400 may include operating system 1402 and machine learning operating system 1430, both communicatively coupled to each other via one or more networks 1420. As described herein, operating system 1402 may be an on-board system of a sensing device, such as a personal operating device (e.g., a mobile phone, tablet computer, etc.), a vehicle operating system, an automation / robotics operating system, and / or other operating systems.

[0299] Operating system 1402 may include one or more operating components 1404. Operating components 1404 of operating system 1402 may include processor 1406 and memory 1408. The one or more processors 1406 may be any suitable processing component (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operating in series. Memory 1408 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, one or more memory components, flash memory components, etc., and combinations thereof.

[0300] Memory 1408 may store information retrieved by one or more processors 1406. For example, memory 1408 (e.g., one or more non-transitory computer-readable storage media, memory devices) includes computer-readable instructions 1410 executable by one or more processors 1406. Instructions 1410 may be software written in any suitable programming language or may be implemented in hardware. Meanwhile, or alternatively, instructions 1410 may execute on logically and / or virtually decoupled threads on processor 1406.

[0301] For example, when the instructions 1410 stored in memory 1408 are executed by one or more processors 1406, the one or more processors 1406 (operating system 1402) may perform operations, such as any operation and function of a sensing device, and / or operations set by the sensing device, object sensing operations and / or determination of object characteristics (e.g., one or more parts of method 1200) and / or any other operation and function as described herein.

[0302] Memory 1408 can store acquired data 1412, which can be acquired in ways such as receiving, accessing, writing, processing, generating, creating, storing, etc. Data 1412 may include, for example, broadband sensor output data, control data, reflectivity data, depth data, calibration data, error correction code data, numerical data, voltage data, readout data, modulation data, timing data, prediction data, machine learning model data (e.g., AI models, parameters, etc.), object material data, calculation result data, pre-calculation result data, transmission and incident light wavelength data, modeling signal data, image analysis data, material determination data, device function data, adjustment data, implementation data, and / or other data / information described herein. In some embodiments, operating component 1404 may acquire data from one or more memories remote from operating system 1402.

[0303] Operating component 1404 may also include a communication interface 1414 for communicating with one or more other systems (e.g., other onboard and / or remote systems / components of operating system 1402). Communication interface 1414 may include any circuitry, components, software, etc., for communicating via one or more networks 1420. In some embodiments, communication interface 1414 may include, for example, one or more of the following: a communication controller, receiver, transceiver, transmitter, port, conductor, software and / or hardware for transmitting data / information, according to one aspect of this application. Operating system 1402 may store or include one or more machine learning models 1416 (e.g., AI modeling units). For example, machine learning model 1416 may be or may otherwise include various machine learning models, such as network models, subnets, neural networks (e.g., deep neural networks), support vector machines, decision trees, ensemble models, k-nearest neighbor models, Bayesian networks, or other types of models (including linear and / or nonlinear models). Exemplary neural networks include feedforward-controlled neural networks (e.g., convolutional neural networks, parameterized continuous convolutions, etc.), recurrent neural networks (e.g., long short-term memory recurrent neural networks, etc.), and / or other forms of neural networks. Machine learning model 1416 may include... Figure 7 Machine learning models and / or other models described herein.

[0304] In some implementations, operating system 1402 may receive one or more machine learning models 1416 from machine learning operating system 1430 via network 1420, and may store one or more machine learning models 1416 in memory 1408 of operating system 1402. Operating system 1402 may use one or more machine learning models 1416 or otherwise implement them (e.g., via processor 1406). Specifically, operating system 1402 may utilize machine learning models 1416 to generate pre-computed data, modeling signal data, object classification data, material identification data, image analysis data, prediction data, etc.

[0305] The machine learning operating system 1430 may include one or more processors 1432 and a memory 1434. The one or more processors 1432 may be any suitable processing component (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be one processor or multiple operatively interconnected processors. The memory 1434 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, one or more memory components, flash memory components, etc., and combinations thereof.

[0306] Memory 1434 stores information accessible by one or more processors 1432. For example, memory 1434 (e.g., one or more non-transitory computer-readable storage media, memory components) may store acquired data 1436, wherein the acquisition methods may include, for example, generating, retrieving, receiving, accessing, writing, retrieving, creating, storing, etc. In some embodiments, machine learning operating system 1430 may acquire data from one or more memory locations remote from machine learning operating system 1430.

[0307] Memory 1434 may also store computer-readable instructions 1438 that can be executed by one or more processors 1432. Instructions 1438 may be software written in any suitable programming language or may be implemented in hardware. Alternatively, instructions 1438 may be executed on logically and / or virtually decoupled threads on processor 1432. When instructions 1438 stored in memory 1434 are executed by one or more processors 1432, the one or more processors 1432 may perform operations. The machine learning operating system 1430 may include a communication interface 1440, which includes components and / or functions similar to those of the aforementioned operating system 1402.

[0308] In some implementations, the machine learning operating system 1430 may include one or more server operating components. If the machine learning operating system 1430 includes multiple server operating components, these server operating components may operate according to various operating architectures, including, for example, sequential operating architectures, parallel operating architectures, or certain combinations thereof.

[0309] In addition to model 1416 within operating system 1402, machine learning operating system 1430 may further include or use one or more machine learning models 1442 (e.g., AI modeling units). For example, machine learning model 1442 may be, or may otherwise include, various machine learning models such as network models, subnets, neural networks (e.g., deep neural networks), support vector machines, decision trees, ensemble models, k-nearest neighbor models, Bayesian networks, or other types of models (including linear and / or nonlinear models). Exemplary neural networks include feedforward controlled neural networks (e.g., convolutional neural networks, parameterized continuous convolutions, etc.), recurrent neural networks (e.g., long short-term memory recurrent neural networks, etc.), and / or other forms of neural networks. Machine learning model 1442 may be similar to and / or identical to machine learning model 1416 and / or other models described herein.

[0310] According to one example, machine learning operating system 1430 can communicate with operating system 1402 based on a master-slave relationship. For example, machine learning operating system 1430 can utilize machine learning model 1442 to provide network services to operating system 1402. Therefore, machine learning model 1442 can be located in and used in operating system 1402, and / or machine learning model 1442 can be located in and used in machine learning operating system 1430.

[0311] In some embodiments, the machine learning operating system 1430 and / or the operating system 1402 may use the model trainer 1444 to train machine learning models 1416 and / or 1442. The model trainer 1444 may use one or more training or learning algorithms to train machine learning models 1416 and / or 1442. One exemplary training technique is backwards propagation of error. In some embodiments, the model trainer 1444 may perform supervised training using a set of labeled training data. In other embodiments, the model trainer 1444 may perform unsupervised training using a set of unlabeled training data. The model trainer 1444 may perform several generalization techniques to improve the generalization ability of the trained model. Generalization techniques include weight decay, dropout, or other techniques.

[0312] Specifically, model trainer 1444 can train machine learning models 1416 and / or 1442 based on a set of training data 1446. Training data 1446 may include previously acquired sensor output (SO) data, such as data from two-dimensional and / or three-dimensional photodetectors. In some embodiments, training data 1446 may include labeled or unlabeled object classification training data. In some embodiments, training data 1446 may include labeled or unlabeled material type data. In some embodiments, training data 1446 may include labeled or unlabeled health data (e.g., blood glucose level labels). In some embodiments, training data 1446 may include labeled or unlabeled AR / VR interaction data (e.g., material response data). In some embodiments, training data 1446 may include data containing a first type of labeled data, but excluding other types of labeled data. For example, training data 1446 may include object classification labels where objects are labeled, but the material property type of the objects is not labeled. Even without material type labels, model trainer 1444 can still use training data 1446 to train models 1442 / 1416 to incorporate data representing material type (e.g., wavelength reflectance data) into object classification decisions, as described herein. Other model functions described herein can be trained in a similar manner using appropriate training data 1446. Model trainer 1444 can be implemented in hardware, firmware, and / or software that controls one or more processors.

[0313] Network 1420 can be any type of network or combination of networks that enables communication between components. In some embodiments, network 1420 may include one or more of a local area network, wide area network, Internet, secure network, mobile communication network, mesh network, peer-to-peer communication link, and / or combinations thereof, and may include any number of wired or wireless connections. Communication on network 1420 may be achieved through a network interface, for example, using any type of protocol, protection scheme, encoding, format, encapsulation, etc.

[0314] Figure 14 An exemplary system 1400 for implementing this application is illustrated, but other operating systems may also be used. For example, in some embodiments, the operating system 1402 may include a model trainer 1444 and a training dataset 1446. In this embodiment, both machine learning models 1416 / 1442 can be trained and used locally on the operating system 1402.

[0315] In this application, tasks performed on operating system 102, which is located away from machine learning operating system 1430, can be performed on machine learning operating system 1430, and vice versa. Implementation of such configurations is also within the scope of this application. A computerized system allows for various possible configurations, combinations, and task and function divisions among components. Computer-implemented operations can be executed on a single component or across multiple components. Computer-implemented tasks and / or operations can be executed sequentially or simultaneously. The data and instructions can be stored in a single memory component or across multiple memory components.

[0316] The sensing device provided in this application utilizes broadband sensors to achieve novel environmental / contextual awareness (e.g., biometric identification, 3D reconstruction, gesture / skeleton detection, target characteristic analysis, material analysis, blood glucose or user context definition) and implement further responses (e.g., unlocking / payment confirmation, navigation, UI / gesture monitoring, and wireless RF signal / power optimization), thereby creating more broadband applications for the market while solving the multipath interference problem.

[0317] In addition to using silicon as the light-absorbing material for the photodetector, this application also provides a photodetector in which the material used (e.g., germanium or III-V group) is capable of absorbing light of a longer wavelength (e.g., wavelength greater than 1050 nm).

[0318] While the preferred embodiments of this application have been illustrated above by way of example, it should be understood that this application is not limited thereto. Rather, it is intended to cover various modifications and similar arrangements and procedures, and therefore the scope of the claims should be interpreted in the broadest sense to include all such modifications and similar arrangements and procedures.

Claims

1. A sensing device, comprising: Sensors, including The first light source is configured to emit first transmitted light having a first wavelength; The second light source is configured to emit a second transmitted light having a second wavelength; and A light detection unit is configured to absorb (i) a first incident light having the first wavelength to generate a first detection signal, and (ii) a second incident light having the second wavelength to generate a second detection signal; A computing circuit, coupled to the sensor and configured to output a calculation result, wherein the computing circuit includes: The pre-calculation unit is configured to perform calculations on the first detection signal and the second detection signal, and output pre-calculation results, the pre-calculation results including integral or differential information for different wavelengths; A first AI modeling unit, coupled to the pre-computation unit, is configured to perform a first image data analysis on the pre-computation results to output a first modeling signal; The second AI modeling unit is configured to perform a second image data analysis on the first detection signal and the second detection signal to output a second modeling signal; and A prediction unit, coupled to the first AI modeling unit and the second AI modeling unit, is configured to output the calculation result based on the first modeling signal and the second modeling signal; and An adjustment circuit, coupled to the computing circuit, is configured to dynamically adjust the power mode associated with the sensing device based on the calculation results. The prediction unit includes one or more machine learning models, which are trained to determine the material type of an object based on the first modeling signal and the second modeling signal.

2. The sensing device as claimed in claim 1, wherein, The sensing device includes a wearable watch or headphones.

3. The sensing device as claimed in claim 1, wherein, The first wavelength is within the wavelength range of near-infrared (NIR) or short-wave infrared (SWIR), and the second wavelength is within the wavelength range of short-wave infrared (SWIR).

4. The sensing device as claimed in claim 1, wherein, The computing circuit is configured to calculate the magnitude difference between the first detection signal and the second detection signal.

5. The sensing device as claimed in claim 1, wherein, The computing circuit is configured to calculate the ratio between the first detection signal and the second detection signal.

6. The sensing device as claimed in claim 1, wherein, The power modes include low power mode and normal power mode.

7. The sensing device as claimed in claim 6, wherein, The sensing device is configured to play music or launch a software application in response to the adjustment circuit adjusting the power mode to enter a normal power mode.

8. The sensing device as claimed in claim 6, wherein, The sensing device is configured to stop playing music or close the software application in response to the adjustment circuit adjusting the power mode to enter a low-power mode.

9. The sensing device as claimed in claim 1, wherein, The material of the object includes skin.

10. The sensing device as claimed in claim 1, wherein, The calculation results indicate the health data of the user sensed by the sensing device.

11. The sensing device as claimed in claim 1, wherein, The calculation result indicates the water content of the object sensed by the sensing device.

12. The sensing device as claimed in claim 1, wherein, The light detection unit includes a first light detector and a second light detector, wherein the first light detector has a first light-absorbing material and the second light detector has a second light-absorbing material, and wherein the first light-absorbing material and the second light-absorbing material are different.

13. The sensing device of claim 12, wherein, The first light-absorbing material includes silicon, and the second light-absorbing material includes germanium or a group III-V material.

14. The sensing device as claimed in claim 1, wherein, The photodetector unit includes a first photodetector and a second photodetector, wherein the second photodetector has the same light-absorbing material as the first photodetector, and wherein the light-absorbing material includes germanium or a III-V group material.

15. The sensing device as claimed in claim 1, wherein, The light detection unit includes a light detector configured to absorb the first incident light in a first period and to absorb the second incident light in a second period.

16. The sensing device as claimed in claim 1, wherein, The sensor also includes a controller configured to generate a signal to control the first light source and the second light source, wherein the first light source, the second light source and the light detection unit are disposed on the controller and electrically connected to the controller via a bonding wire.

17. The sensing device of claim 16, wherein, The optical detection unit is connected to the controller via at least one bonding wire.

18. The sensing device of claim 16, wherein, The first light source is connected to the controller via at least one bonding wire.

19. The sensing device as claimed in claim 1, wherein, The sensor also includes a controller, wherein the photodetector wafer is bonded to the controller, and wherein the first light source is electrically connected to the controller via a bonding wire.

20. The sensing device of claim 19, further comprising a plurality of interconnects located between the light detection unit and the controller.

Images