Pulse processing circuit and method, sensor and electronic equipment

Abstract

The embodiment of the invention is suitable for the technical field of detection, and provides a pulse processing circuit and method, a sensor and electronic equipment, the pulse processing circuit is configured to be connected with a photoelectric detector, the photoelectric detector comprises at least one photoelectric detection unit, and the photoelectric detection unit is used for outputting a digital pulse signal for a detected single photon; the pulse processing circuit comprises a pulse response module which is connected with at least one photoelectric detection unit and is configured to output a neural network input pulse signal corresponding to the photoelectric detection unit under the condition that a digital pulse signal is received; the neural network module is connected with the pulse response module and is configured to respond to neural network input pulse signals at different moments, generate feature data and output the feature data; the feature data is used for generating a detection result. According to the embodiment of the invention, the feature data can be extracted on the premise that the digital pulse signal does not need to be stored, and the data output quantity is effectively reduced.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application belongs to the technical field of detection, and particularly relates to a pulse processing circuit and method, a sensor and an electronic device. BACKGROUND

[0002] A single-photon avalanche diode (SPAD) is a highly sensitive optical detector capable of detecting a single photon. The SPAD works under a reverse bias voltage. When the reverse bias voltage exceeds the breakdown voltage, the electron-hole pairs generated by the absorption of a single photon accelerate in a strong electric field, triggering avalanche carrier multiplication and forming a detectable current pulse. Due to its high sensitivity and fast response time, the SPAD is widely used in weak light detection and high-precision time measurement fields (such as LiDAR, quantum communication, biomedical imaging, etc.).

[0003] In some SPAD imaging application schemes, the number of photons detected by the SPAD is directly accumulated and stored, and then the corresponding pixel value is obtained through the stored number of electrons. These schemes sacrifice the high time resolution characteristics of the SPAD because the number of photons needs to be accumulated for a certain period of time.

[0004] In other SPAD imaging application schemes, in order to provide high time resolution characteristics, the photons detected by the SPAD are converted into digital signals through multiple signal conversion and processing. Due to the setting of different signal conversion devices, the overall power consumption is significantly increased. SUMMARY

[0005] Therefore, the embodiment of the present application provides a pulse processing circuit, method, sensor and electronic device to effectively extract feature data of a current pulse without storing the current pulse.

[0006] A first aspect of the embodiment of the present application provides a pulse processing circuit, which is configured to be connected to a photodetector. The photodetector includes at least one photodetection unit, and the photodetection unit is configured to output a digital pulse signal for a detected single photon. The pulse processing circuit includes:

[0007] a pulse response module connected to the at least one photodetection unit, configured to output a neural network input pulse signal corresponding to the photodetection unit when receiving the digital pulse signal;

[0008] a neural network module connected to the pulse response module, configured to generate feature data in response to the neural network input pulse signal at different time instants, and output the feature data; and the feature data is used to generate a detection result.

[0009] In some implementations of the first aspect, the impulse response module comprises a latch corresponding to each of the photodetection units;

[0010] The latch is configured to generate a neural network input pulse signal according to a preset signal feature when receiving the digital pulse signal, and send the neural network input pulse signal to the neural network module;

[0011] The preset signal feature comprises a preset amplitude and a preset pulse width.

[0012] In some implementations of the first aspect, the neural network module comprises a plurality of first neuron components, each of the first neuron components is connected to each of the latches, and each of the first neuron components comprises:

[0013] a plurality of first selector components, each of the first selector components is connected to each of the latches, and the first selector is configured to generate a first weight corresponding to the photodetection unit connected to the latch according to the neural network input pulse signal at different time points, the neural network input pulse signals output by the same latch at different time points satisfy a preset first probability distribution in the time domain, and the neural network input pulse signals output by different latches at the same time point satisfy a preset second probability distribution in the spatial domain;

[0014] a first neuron calculation unit connected to the plurality of first selector components and configured to output a first neuron output pulse signal when an integral value obtained according to the first weights at different time points satisfies a preset threshold voltage;

[0015] The feature data is determined by the first neuron output pulse signal.

[0016] In some implementations of the first aspect, the first neuron calculation unit comprises:

[0017] a first addition component configured to accumulate according to the first weights to generate an accumulated value;

[0018] a first membrane potential component connected to the first addition component and configured to generate an integral value according to the accumulated values received at different time points and a preset time correlation coefficient;

[0019] a first activation component connected to the first membrane potential component and configured to output the first neuron output pulse signal when it is determined that the integral value reaches a threshold voltage corresponding to the first activation component.

[0020] In some implementations of the first aspect, the plurality of first neuron components are located in a first neuron array, and the neural network module further comprises a second neuron array;

[0021] The second neuron array is configured to generate feature data according to the first neuron output pulse signals.

[0022] In some implementations of the first aspect, the neural network module further comprises a third neuron array arranged between the first neuron array and the second neuron array;

[0023] The third neuron array is configured to generate third neuron output pulse signals according to the first neuron output pulse signals at different time points;

[0024] The second neuron array is configured to generate feature data according to the third neuron output pulse signals.

[0025] In some implementations of the first aspect, the neural network module comprises a plurality of third neuron arrays, and the plurality of third neuron arrays are connected in sequence;

[0026] The first third neuron array is connected to the first neuron array and is configured to receive the first neuron output pulse signals output by the first neuron array;

[0027] The last third neuron array is connected to the second neuron array and is configured to output the third neuron output pulse signals corresponding to the last third neuron array to the second neuron array;

[0028] The intermediate third neuron array is configured to receive the third neuron output pulse signals output by the previous third neuron array and output the third neuron output pulse signals corresponding to the intermediate third neuron array to the next third neuron array.

[0029] In some implementations of the first aspect, the detection result is generated by a preset feature processing circuit according to the feature data; and the pulse processing circuit further comprises:

[0030] A configuration module connected to the neural network module and configured to record configuration information of the pulse processing circuit; the configuration information is obtained by model training of the pulse processing circuit and the feature processing circuit according to labeled training data;

[0031] The configuration information comprises at least one of the first weight and the threshold voltage.

[0032] In some implementations of the first aspect, the neural network module is connected to a memory; and the memory is configured to store the feature data and determine and store time domain information;

[0033] The time domain information comprises a timestamp corresponding to the feature data, and a number of neural network input pulse signals corresponding to each of the photodetection units in a unit time.

[0034] The second aspect of the embodiment of the present application provides a sensor, comprising:

[0035] The pulse processing circuit according to the first aspect, and a photodetector connected with the pulse processing circuit.

[0036] In some implementations of the second aspect, a plurality of pulse processing circuits are provided, and the photodetector comprises a plurality of pixel regions; one pulse processing circuit is connected with one pixel region.

[0037] In some implementations of the second aspect, a memory connected with each of the pulse processing circuits is further included.

[0038] In some implementations of the second aspect, a feature processing circuit connected with the memory is further included.

[0039] The third aspect of the embodiment of the present application provides an electronic device, wherein at least one sensor according to the second aspect is provided.

[0040] The fourth aspect of the embodiment of the present application provides a pulse processing method, applied to the pulse processing circuit according to the first aspect, and the method comprises:

[0041] In a case where a digital pulse signal output by the photodetection unit for a detected single photon is received, outputting a neural network input pulse signal corresponding to the photodetection unit;

[0042] In response to the neural network input pulse signals received at different time instants, generating feature data, and outputting the feature data; the feature data is used to generate a detection result.

[0043] The fifth aspect of the embodiment of the present application provides a computer program product, comprising a computer program, which, when executed, causes the pulse processing method according to the fourth aspect to be executed.

[0044] The sixth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the pulse processing method according to the fourth aspect.

[0045] The embodiment of the present application has the following beneficial effects:

[0046] In an embodiment of the present application, a pulse processing circuit is configured to connect to a photodetector. The photodetector includes at least one photodetection unit, which is configured to output a digital pulse signal in response to a detected single photon. The pulse processing circuit includes a pulse response module connectable to the photodetection unit, and a neural network module connected to the pulse response module. The pulse response module is capable of generating a neural network input pulse signal upon receiving a digital pulse signal output by the photodetection unit, and transmitting the neural network input pulse signal to the neural network module. The neural network model calculates and processes each neural network input pulse signal received at different times to generate feature data. Because the feature data is derived from the neural network input pulse signals at different times, the size of the feature data can be reduced while ensuring its validity and reliability. Since no specific device is required to store and accumulate the number of photons detected by the photodetection unit before calculating the feature data, the circuit structure is simplified. Through the embodiments of the present application, feature data can be generated when the photodetection unit outputs a digital pulse signal. The digital pulse signal is obtained by processing the current pulse generated by the photodetection unit when a single photon is detected, thereby preprocessing the current pulse and reducing the complexity of the device or circuit that subsequently uses the feature data to generate the detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0048] Figure 1 is a schematic diagram of a pulse processing circuit provided in an embodiment of the present application;

[0049] Figure 2 This is another pulse processing circuit schematic provided in an embodiment of the present application;

[0050] Figure 3 This is a schematic diagram of a latch provided in an embodiment of the present application;

[0051] Figure 4 is a schematic diagram of another pulse processing circuit provided in an embodiment of the present application;

[0052] Figure 5 is a schematic diagram of a first neuron computing unit provided in an embodiment of the present application;

[0053] Figure 6 is a schematic diagram of another neural network module provided in an embodiment of the present application;

[0054] Figure 7 is a schematic diagram of another neural network module provided by an embodiment of the present application;

[0055] Figure 8 is a schematic diagram of a sensor provided by an embodiment of the present application;

[0056] Figure 9 is a schematic diagram of another sensor provided by an embodiment of the present application;

[0057] Figure 10 is a schematic diagram of yet another sensor provided by an embodiment of the present application;

[0058] Figure 11 is a schematic diagram of a pulse processing method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0059] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0060] It is to be understood that the terminology "includes", "has", "holds", "contains" and / or "comprising", when used in this specification and in the following claims, indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0061] It is also to be understood that the terminology "and / or" when used in this specification and in the following claims, refers to at least one of the items, or any combination of one or more of the items, associated with the "and / or" term.

[0062] As used in this specification and in the claims, the term "if" can be interpreted as meaning "when", or "once", or "in response to a determination", or "in response to a detection" of, as appropriate. Similarly, the phrase "if determined", or "if detected" can be interpreted as meaning "once determined", or "in response to a determination", or "once detected", or "in response to a detection", as appropriate.

[0063] In addition, in the description of the application and in the claims, the terms "first", "second", "third", etc. are used only for distinguishing between similar elements, and cannot be interpreted as indicating or implying relative importance.

[0064] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places within specified sections of this specification are not necessarily all referring to the same embodiment, however, and can refer to one or more but less than all embodiments of the application, unless otherwise specified. The terms "including," "containing," "comprising," "having," and variations thereof herein are meant to be broad and encompass the possibility of zero, one, or more steps or components, unless otherwise indicated.

[0065] The technical solutions of the application are described below through specific embodiments.

[0066] Referring to Figure 1 , a schematic diagram of a pulse processing circuit provided by an embodiment of the application is shown, the pulse processing circuit 1000 is configured to be connected to a photodetector 2000, the photodetector includes at least one photodetection unit, and the photodetection unit is configured to output a digital pulse signal for a detected single photon.

[0067] The photodetector 2000 outputs an electrical signal based on a photoelectric effect, and the photodetector 2000 can be composed of one or more photodetection units, for example, the photodetector 2000 includes a plurality of photodetection units arranged in an array.

[0068] In the photodetector 2000, the photodetection unit can include a photodetector that can detect a single photon, and a current processing circuit connected to the photodetector, the current processing circuit can process the current pulse output by the photodetector when a single photon is detected to obtain a digital pulse signal. As an example, the photodetector described above is a single-photon avalanche diode (Single-Photon Avalanche Diode, APAD).

[0069] The pulse processing circuit 1000 includes a pulse response module 100 and a neural network module 200.

[0070] The neural network module 200 can call a neural network model to calculate the data received thereby, thereby extracting corresponding features from the data received thereby. Before the pulse processing circuit 1000 is used for detection in a real environment, the neural network model needs to be trained to determine parameters related to the neural network model, so that the neural network module 200 can extract corresponding feature data.

[0071] The pulse response module 100 is connected with at least one of the photodetector units, and is configured to output a neural network input corresponding to the photodetector unit in response to receiving the digital pulse signal.

[0072] The neural network model can include a plurality of data processing units, and the data processing units process the weight data received thereby. In an embodiment of the present application, the data output by the pulse response model is used as the input of the neural network module 200.

[0073] The neural network module 200 is connected with the pulse response module 100, and is configured to generate feature data in response to the neural network input pulse signals at different time points, and output the feature data. The feature data is used to generate a detection result.

[0074] Since the sensitivities of different photodetector units to light are not exactly the same, when different photodetector units receive light at the same time, the time of the digital pulse signals output by the photodetector units can have a certain deviation. In addition, the dead time of different photodetector units is not exactly the same, which results in that the intervals of the digital pulse signals output by the photodetector units are not necessarily exactly the same. The neural network module is configured to process the neural network input pulse signals at different time points received within a certain continuous time to generate feature data, thereby improving the effectiveness and reliability of the feature data. Thus, the feature data is an equivalent cumulative result in time.

[0075] After obtaining the feature data, the feature data can be directly or indirectly transmitted to a device or circuit for computing and processing the feature data, so that the device or circuit can generate a detection result according to the feature data. Since the pulse response module 100 can be connected with a plurality of photodetector units, it can output the neural network input pulse signals corresponding to different photodetector units at the same time. The neural network module 200 can simultaneously receive the neural network input pulse signals corresponding to different photodetector units, and process the neural network input pulse signals corresponding to one or more photodetector units at different time points to generate feature data.

[0076] In the embodiment of the present application, the pulse processing circuit 1000 comprises a pulse response module 100 connectable with the photodetector 2000, and a neural network module 200 connected with the pulse response module 100. The pulse response module 100 is capable of generating a neural network input pulse signal corresponding to the photodetection unit outputting the digital pulse signal in the case of receiving the digital pulse signal output by the photodetection unit in the photodetector 2000, and transmitting the neural network input pulse signal corresponding to each photodetection unit to the neural network module 200. The neural network model processes the neural network input pulse signal corresponding to each photodetection unit received at different times to generate feature data. Since the feature data is obtained from the neural network input pulse signal at different times, the size of the feature data can be reduced while ensuring the effectiveness and reliability of the feature data. Since there is no need to set a specific device to store and accumulate the number of photons detected by the photodetection unit and then calculate the feature data, the circuit structure is simplified. Through the embodiment of the present application, the feature data can be generated in the case that the photodetection unit outputs a digital pulse signal for a single photon. The digital pulse signal is processed by the current pulse generated by the photodetection unit. The corresponding feature data is generated to pre-process the current pulse. The feature processing circuit connected with the pre-processing circuit can call the corresponding intelligent network to process the feature data to obtain the corresponding detection result. There is no need to repeatedly extract the feature by the above-mentioned feature processing circuit, which reduces the complexity of the device or circuit using the feature data to generate the detection result.

[0077] With reference to Figure 2 , another pulse processing circuit 1000 schematic diagram provided by the embodiment of the present application is shown. In some implementation modes of the embodiment of the present application, the pulse response module 100 comprises a latch 110 corresponding to each photodetection unit.

[0078] The latch 110 is configured to generate a neural network input pulse signal different from the digital pulse signal according to a preset signal feature in the case of receiving the digital pulse signal, and send the neural network input pulse signal to the neural network module 200.

[0079] Since the pulse width of the digital pulse signal output by different photodetection units is not necessarily completely the same, if the digital pulse signal is directly used as the input and output of the neural network module 200, the neural network model needs a more complex network structure to reduce the influence of different pulse widths on the generation of feature data by the neural network module 200.

[0080] The embodiment of the present application performs asynchronous latching on the digital pulse signal through the latch 110, and then outputs a neural network input pulse signal different from the digital pulse signal according to a preset signal feature. The pulse width is determined according to the circuit structure of asynchronous latching. The preset signal feature includes a preset amplitude and a preset pulse width. By performing asynchronous latching on the digital pulse signal output by each photoelectric detection unit and then outputting the neural network input pulse signal, it is realized that each latch can output a neural network input pulse signal with the same amplitude and the same pulse width when receiving the digital pulse signal, the neural network input pulse signal input into the neural network module is standardized, and the stability of the neural network module in processing the neural network input pulse signal is improved.

[0081] With reference to Figure 3 , a schematic diagram of a latch 110 provided by the embodiment of the present application is shown. The latch is taken as a D-type latch for example. The clock end of the D-type latch is connected with the photoelectric detection unit, the D end is connected with a high potential, and the Q end is connected with the reset end of itself through a delay element, so that the D-type latch 110 can be triggered by the digital pulse signal output by the photoelectric detection unit to output a pulse signal with a fixed pulse width, i.e., a neural network input pulse signal. The pulse signal width is set by the structure of the delay element, which is not limited in the embodiment of the present application.

[0082] With reference to Figure 4 , a schematic diagram of another pulse processing circuit 1000 provided by the embodiment of the present application is shown. In some implementation modes of the embodiment of the present application, the neural network module includes a plurality of first neuron components 211. Each first neuron component 211 is connected with each latch, so that the first neuron component and the latch constitute a full connection structure. The first neuron component 211 includes:

[0083] a plurality of first selector components 2111, each first selector component 2111 being connected with each latch 110. The first selector component 2111 is configured to generate a first weight corresponding to the photoelectric detection unit connected with the latch according to the neural network input pulse signal at different time. The neural network input pulse signals at different time output by the same latch satisfy a preset first probability distribution in the time domain. The neural network input pulse signals output by different latches at the same time satisfy a preset second probability distribution in the spatial domain.

[0084] a first neuron calculation unit 2112, configured to output a second excitation signal when the integral value obtained according to the neural network input pulse signal at different time satisfies a preset threshold voltage.

[0085] The second selector component is connected with the first neuron calculation unit 2112, and is configured to output a second weight corresponding to the second selector component in response to the second excitation signal.

[0086] The enable end of the first selector component 2111 is connected with the latch 110, so that the neural network input pulse signal sent by the latch 110 is taken as the first excitation signal of the first selector component 2111. The first selector component 2111 can be composed of one or more selectors, each of which has two input ends, one of which can read preset data, and the other of which is a zero setting end. The first selector component 2111 will respond to the first excitation signal to make each first selector component 2111 output a first weight corresponding to the photodetector unit and input the first weight to the first neuron calculation unit 2112.

[0087] After the foregoing training is completed, the first weight is a determined value. When the pulse response module 100 detects the real environment based on the pulse processing circuit 1000, if the pulse response module 100 receives a digital pulse signal transmitted by a photodetector unit, that is, the photodetector unit outputting the digital pulse signal detects light, the pulse response module 100 outputs a neural network input pulse signal corresponding to the photodetector unit, and outputs the first weight after the first selector component 2111 receives the neural network input pulse signal; if the pulse response module 100 does not receive a digital pulse signal transmitted by any photodetector unit, it means that the photodetector unit does not detect light, and the pulse response module 100 does not output a neural network input pulse signal corresponding to the photodetector unit, and the first selector component 2111 does not output the first weight corresponding to the photodetector unit.

[0088] The first weight can be composed of one or more bits, for example: the first weight can be a 4-bit character, and the number of selectors in the first selection component corresponds to the number of bits of the first weight.

[0089] For example: a certain first selector component 2111 is connected with a certain photodetector unit through a latch, the first weight corresponding to the photodetector unit is "0010", the first weight includes 4 bits, therefore the first selector component 2111 is composed of four selectors, when the first selector component 2111 receives the first excitation signal, the four selectors output 0, 0, 1, 0 in turn, and then obtain the first weight "0010".

[0090] The neural network input pulse signals output by the same latch 110 at different times satisfy a preset first probability distribution in the time domain. By the neural network input pulse signals at different times, the first neuron component 211 filters out noise that does not satisfy the first probability distribution, and realizes time domain feature extraction on the neural network input pulse signal.

[0091] The neural network input pulse signals output by different latches 110 at the same time satisfy a preset second probability distribution in the spatial domain. Since different latches are respectively connected with different photodetectors, and the different photodetectors have corresponding spatial distribution characteristics, by the neural network input pulse signals of different latches 110 at the same time, the first neuron assembly 211 filters out noise that does not satisfy the first probability distribution, and realizes spatial feature extraction of the neural network input pulse signals.

[0092] The first neuron calculation unit 2112 can receive the neural network input pulse signals of multiple latches 110, and accumulate the neural network input pulse signals received at different times, so as to realize time domain feature extraction of the digital pulse signals. In a case where the accumulated value obtained by the first weight reaches the threshold voltage corresponding to the first neuron array 210, the first neuron calculation unit 2112 outputs a first neuron output pulse signal.

[0093] The feature data is determined by the first neuron output pulse signal. As an example, the first neuron output pulse signal can be further processed to obtain the feature data. As another example, the first neuron output pulse signal can be directly determined as the feature data and transmitted to other circuits connected with the neural network module (for example, a circuit for storing the feature data, or a circuit for calculating using the feature data).

[0094] Reference Figure 5 , a schematic diagram of a first neuron calculation unit 2112 provided by an embodiment of the present application is shown. In some implementation manners of the embodiment of the present application, the first neuron calculation unit 2112 is constructed based on a leakage integration model. The first neuron calculation unit 2112 includes:

[0095] A first addition assembly 21121 configured to accumulate according to the first weight to generate an accumulated value;

[0096] A first membrane potential assembly 21122 connected with the first addition assembly 21121 and configured to generate an integral value according to the accumulated value received at different times and a preset time correlation coefficient;

[0097] A first activation assembly 21123 connected with the first membrane potential assembly 21122 and configured to output a second excitation signal in a case where the integral value reaches a threshold voltage corresponding to the first activation assembly.

[0098] The first addition tree component accumulates the first weight output by each first selector component 2111 connected thereto to generate an accumulated value, and stores the accumulated value to the first membrane potential component 21122. When the first membrane potential is a non-zero value, the first membrane potential continues to integrate according to a preset time correlation coefficient, and then sums the accumulated values received at different times to obtain an integrated value. The first activation component 21123 can identify the voltage value of the integrated value currently generated by the first membrane potential component 21122, and when the voltage value of the integrated value reaches the threshold voltage corresponding to the first neuron calculation unit 2112 component, the first neuron output pulse signal is output.

[0099] The time correlation coefficient is used to reduce the value stored in the first membrane potential component 21122 according to a time-dependent decay rule when the value stored in the first membrane potential component 21122 is a non-zero value. The specific calculation formula of the time correlation coefficient is not limited in the embodiments of the present application.

[0100] As an example, after the first activation component 21123 outputs the first neuron output pulse signal, the value stored in the first membrane potential component 21122 can be reset to a specified value or reduced by a specified value.

[0101] In a specific implementation, the time resolution of the feature data can be adjusted by making corresponding settings to the first neuron calculation unit, including but not limited to adjusting the time length for the first addition component 21121 to calculate the accumulated value, or adjusting the frequency of detecting the current voltage of the integrated value, etc. to provide effective circuit support for high time resolution imaging based on the pulse processing circuit 1000 provided in the embodiments of the present application.

[0102] It should be noted that in actual application, the digital pulse signal output by the photodetector unit can exhibit sparse characteristics, for example: the time interval of the digital pulse signal output by a single photodetector unit is large, the interval of the digital pulse signals output by different photodetector units is large, etc. When the photodetector unit does not output a digital pulse signal, the neural network input pulse signal output by the latch 110 corresponding to the photodetector unit is 0, the first weight output by the first selector component 2111 connected to the asynchronous latch component 110 is also 0, and the first addition component 21121 does not need to accumulate. The first addition component 21121 in the entire preprocessing circuit does not need to consume power to perform signal flipping from "0" to "1", so that when the digital pulse signal output by the photodetector unit can exhibit sparse characteristics, the number of flips of the entire pulse processing circuit is small, and the power consumption of the pulse processing circuit is reduced.

[0103] As an example, the first membrane potential component 21122 can be one of SRAM (Static Random Access Memory), a register, RRAM (Resistive Random Access Memory), and MRAM (Magnetic Random Access Memory).

[0104] As an example, the first addition component 21121 is an addition tree structure.

[0105] Referring to Figure 6 , a schematic diagram of another pulse processing circuit 1000 provided by an embodiment of the application is shown; in some implementations of the embodiment of the application, the plurality of first neuron components 211 are located in a first neuron array, and the neural network module 200 further includes:

[0106] The second neuron array 220 is configured to generate feature data according to the first neuron output pulse signal.

[0107] The first neuron array 210 is configured to output a first neuron output pulse signal in a case where an integral value obtained according to the first weight at different times satisfies a preset threshold voltage.

[0108] The first neuron array 210 obtains an integral value by accumulating the first weight at different times, and outputs a first neuron output pulse signal in a case where the integral value is greater than or equal to a preset threshold voltage, and transmits the first neuron output pulse signal to the second neuron array 220. The second neuron array 220 processes the received first neuron output pulse signal to generate feature data.

[0109] By accumulating the neural network input pulse signals at different times, time domain feature processing is realized on the neural network input pulse signals corresponding to the photoelectric detection unit, the credibility of the second weight is improved, and the effectiveness of the feature data is further improved. The neural network module 200 processes the neural network input pulse signals at multiple times to obtain the feature data, rather than generating the feature data from the neural network input pulse signals at each time, thereby effectively reducing the data size of the feature data while maintaining high credibility.

[0110] As an example, the second neuron array 220 includes a plurality of second neuron components 221, and each second neuron component 221 includes:

[0111] a plurality of second selector components, each of the second selector components being connected with each of the first neuron components 211; the second selector component is configured to generate a second weight corresponding to the second selector component according to the first neuron output pulse signal at different time;

[0112] a second neuron calculation unit connected with the plurality of second selector components, and configured to output feature data when an integral value obtained according to the second weight at different time meets a preset threshold voltage.

[0113] After the model training of the neural network module 200, the second weight corresponding to the second selector component of the second neuron array 220 and the threshold voltage of the second neuron calculation unit can be determined.

[0114] Referring to Figure 7 , another schematic diagram of a neural network module 200 is shown, in some implementations of the embodiments of the present application, the neural network module further includes a third neuron array 230 arranged between the first neuron array 210 and the second neuron array 220.

[0115] The third neuron array 230 is configured to generate a third neuron output pulse signal according to the first neuron output pulse signal at different time.

[0116] The second neuron array 220 is configured to generate feature data according to the third neuron output pulse signal.

[0117] In specific implementations, the third neuron array 230 can be arranged between the first neuron array 210 and the second neuron array 220 according to actual needs, including but not limited to improving the accuracy of the feature data.

[0118] The third neuron array 230 can receive the first neuron output pulse signal output by the first neuron array 210, and generate a third neuron output pulse signal corresponding to the third neuron array 230 according to the third weight obtained according to the first neuron output pulse signal at different time, and the integral value obtained according to the third weight at different time meets the threshold voltage corresponding to the third neuron array 230. The third neuron array 230 can input the third neuron output pulse signal to the second neuron array 220. In the case that the neural network module 200 includes the third neuron array 230, the second neuron array 220 is configured to generate feature data according to the third neuron output pulse signal, instead of generating feature data according to the first neuron output pulse signal. By arranging the third neuron array 230 between the first neuron array 210 and the second neuron array 220, the feature data can be optimized, for example, the accuracy of the feature data can be improved.

[0119] In some implementations of the embodiments of the present application, the neural network module 200 includes a plurality of third neuron arrays 230, and the plurality of third neuron arrays 230 are sequentially connected;

[0120] The first third neuron array 230 is connected with the first neuron array 210 and is configured to receive the first neuron output pulse signal output by the first neuron array 210;

[0121] The last third neuron array 230 is connected with the second neuron array 220 and is configured to output the third neuron output pulse signal corresponding to itself to the second neuron array 220;

[0122] The intermediate third neuron array 230 is configured to receive the third weight output by the previous third neuron array 230 and output the third neuron output pulse signal corresponding to itself to the subsequent third neuron array 230.

[0123] If the neural network module 200 includes a plurality of third neuron arrays 230, and the plurality of third neuron arrays 230 are sequentially connected, the first third neuron array 230 in the sequentially connected plurality of third neuron arrays 230 is configured to receive the first neuron output pulse signal output by the first neuron array 210 and output the third neuron output pulse signal corresponding to itself to the subsequent third neuron array 230. The last third neuron array 230 in the sequentially connected plurality of third neuron arrays 230 is connected with the second neuron array 220 and is configured to output the third neuron output pulse signal corresponding to itself to the second neuron array 220, so that the second neuron array 220 generates the feature data according to the third neuron output pulse signal output by the last third neuron array 230. The third neuron array 230 other than the first third neuron array 230 and the last third neuron array 230 is the intermediate third neuron array 230, and the intermediate third neuron array 230 is configured to receive the third neuron output pulse signal output by the previous third neuron array 230 and output the third neuron output pulse signal corresponding to itself to the subsequent third neuron array 230.

[0124] As an example, the third neuron array 230 includes a plurality of third neuron components 231, and the third neuron component 231 includes:

[0125] a plurality of third selector components, each of the third selector components being connected with a neuron component in the previous neuron array; the third selector component is configured to generate a third weight corresponding to itself according to neuron output pulse signals (first neuron output pulse signals or third neuron output pulse signals output by the previous third neuron array) at different time points;

[0126] a third neuron calculation unit connected with the plurality of second selector components, and configured to output third neuron output pulse signals in a case where an integral value obtained according to the third weight at different time points meets a preset threshold voltage.

[0127] After the model training of the neural network module 200, the third weight corresponding to the third selector component in the third neuron array 230 and the threshold voltage of the third neuron calculation unit can be determined.

[0128] The first neuron component 211, the second neuron component 221 and the third neuron component 231 mainly differ in input, output and threshold voltage, and the related description can be referred to the first neuron component 211, which will not be repeated here.

[0129] The second neuron array 220 is provided with a plurality of second neuron calculation units, and the second neuron calculation unit has the same structure as the first neuron calculation unit 2112 and the third neuron calculation unit. The related description can be referred to the first neuron calculation unit 2112, which will not be repeated here.

[0130] In some implementations of the embodiments of the present application, the detection result is generated by a preset feature processing circuit according to the feature data; the pulse processing circuit 1000 further comprises:

[0131] a configuration module connected with the neural network module 200, and configured to record configuration information of the pulse processing circuit 1000; the configuration information is obtained by model training of the pulse processing circuit 1000 and the feature processing circuit according to the labeled training data;

[0132] The configuration information includes the first weight, the second weight, the third weight and the threshold voltage.

[0133] Different first selector components 2111 can correspond to different neural network input pulse signals, different second selector components can correspond to different second weights, and different first neuron calculation units 2112 can correspond to different threshold voltages.

[0134] In the case where the neural network module 200 further comprises a third neuron array 230, the configuration information further comprises the third weight corresponding to each third selector component.

[0135] In some implementations of the embodiments of the present application, the neural network module 200 is connected with a memory; the memory is configured to store the feature data, and determine and store the time domain information;

[0136] The time domain information includes a timestamp corresponding to the feature data, and a number of neural network input pulse signals corresponding to each photoelectric detection unit in a unit time.

[0137] The memory can record the timestamp corresponding to the feature data when receiving the feature data, and determine and record the number of neural network input pulse signals corresponding to each photoelectric detection unit in a unit time at a certain frequency or at a specified time.

[0138] The feature processing circuit can process in combination with the time domain information and the feature data to generate a corresponding detection result, thereby improving the accuracy of the detection result.

[0139] The feature processing circuit can use a neural network structure, for example, an SNN network structure or a CNN network structure, and the embodiments of the present application do not limit the neural network structure used by the feature processing circuit.

[0140] Referring to Figure 8 , a schematic diagram of a sensor provided by an embodiment of the present application is shown, which includes the pulse processing circuit 1000 as described above, and a photoelectric detector 2000 connected with the pulse processing circuit.

[0141] Referring to Figure 9 , a schematic diagram of another sensor provided by an embodiment of the present application is shown, in some implementations of the embodiments of the present application, the pulse processing circuit 1000 is multiple, and the photoelectric detector 2000 includes multiple pixel regions; one pulse processing circuit 1000 is connected with one pixel region.

[0142] As an example, the photoelectric detector can be provided with M pixel regions, each pixel region can be provided with N photoelectric devices, that is, one pulse processing circuit is provided with N latches connected with N photoelectric devices. When M*N is a certain value, that is, when the photoelectric detector has a certain number of photoelectric devices, the larger N is, the smaller M is, and the stronger the perception of the spatial domain feature is (for example, in a scene where the object of interest occupies a larger field of view angle, N is set to be larger); correspondingly, the larger N is, the more complex the connection between the response module and the neuron component in the neural network module in the network response module is, therefore, in actual application, for the scene that the photoelectric detector contains a certain number of photoelectric devices, the specific values of M and N can be determined according to actual needs.

[0143] In some implementations of the embodiments of the present application, the sensor further includes a memory 701 connected to each of the pulse processing circuits 1000 .

[0144] like Figure 9 As shown, in some implementations of the embodiments of the present application, the sensor further includes a feature processing circuit 702 connected to the memory 701 .

[0145] Reference Figure 10 , shows a schematic diagram of another sensor provided in an embodiment of the present application, in which a feature processing circuit 702 is disposed in the sensor.

[0146] Reference Figure 11 , shows a schematic diagram of a pulse processing method provided in an embodiment of the present application, the method comprising:

[0147] Step 1101: upon receiving a digital pulse signal output by a photoelectric detection unit in response to a detected single photon, output a neural network input pulse signal corresponding to the photoelectric detection unit;

[0148] Step 1102: generating feature data in response to the neural network input pulse signals received at different times, and outputting the feature data; the feature data is used to generate a detection result.

[0149] An embodiment of the present application further discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the pulse processing method as described in the above embodiments is implemented.

[0150] An embodiment of the present application further discloses a computer program product, including a computer program. When the computer program is executed, the pulse processing method described in the above embodiments is executed.

[0151] The above embodiments are intended only to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the above embodiments or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application and should be included within the scope of protection of the present application.

Claims

1. A pulse processing circuit, characterized in that: The pulse processing circuit is configured to be connected to a photodetector, wherein the photodetector includes at least one photodetection unit, and the photodetection unit is configured to output a digital pulse signal for a detected single photon; The pulse processing circuit comprises: an impulse response module, connected to at least one of the photoelectric detection units, and configured to output a neural network input pulse signal corresponding to the photoelectric detection unit when receiving the digital pulse signal; The neural network module is connected to the impulse response module and is configured to generate characteristic data in response to the neural network input pulse signals at different times, and to output the characteristic data; the characteristic data is used to generate the detection result.

2. The pulse processing circuit according to claim 1, wherein: The pulse response module includes a latch corresponding to each of the photodetection units; The latch is configured to generate a neural network input pulse signal different from the digital pulse signal according to a preset signal characteristic when the digital pulse signal is received; and sending the neural network input pulse signal to the neural network module; The preset signal characteristics include a preset amplitude and a preset pulse width.

3. The pulse processing circuit according to claim 2, characterized in that: The neural network module includes: a plurality of first neuron components; each of the first neuron components is connected to each of the latches; the first neuron components include: A plurality of first selector components, each of which is connected to each of the latches; the first selector is configured to generate a first weight corresponding to a photodetection unit connected to the latch based on a neural network input pulse signal at different times; the neural network input pulse signals output by the same latch at different times satisfy a preset first probability distribution in the time domain; and the neural network input pulse signals output by different latches at the same time satisfy a preset second probability distribution in the spatial domain; a first neuron calculation unit connected to the plurality of first selector components and configured to output a first neuron output pulse signal when an integral value obtained according to the first weights at different moments satisfies a preset threshold voltage; The characteristic data is determined by the pulse signal output by the first neuron.

4. The pulse processing circuit according to claim 3, characterized in that: The first neuron computing unit comprises: a first adding component configured to perform accumulation according to the first weight to generate an accumulated value; a first membrane potential component connected to the first adding component and configured to generate an integral value based on the accumulated values ​​received at different times and a preset time correlation coefficient; The first activation component is connected to the first membrane potential component and is configured to output a first neuron output pulse signal when it is determined that the integrated value reaches a threshold voltage corresponding to the first activation component.

5. The pulse processing circuit according to claim 3, characterized in that: The plurality of first neuron components are located in a first neuron array, and the neural network module further comprises: a second neuron array; The second neuron array is configured to generate characteristic data according to the pulse signal output by the first neuron.

6. The pulse processing circuit according to claim 5, characterized in that: The neural module further includes a third neuron array disposed between the first neuron array and the second neuron array; The third neuron array is configured to generate a third neuron output pulse signal based on the first neuron output pulse signal at different times; The second neuron array is configured to generate characteristic data according to the pulse signal output by the third neuron.

7. The pulse processing circuit according to claim 6, characterized in that: The neural network module includes a plurality of the third neuron arrays, and the plurality of the third neuron arrays are connected in sequence; A first third neuron array is connected to the first neuron array and is configured to receive a first neuron output pulse signal output by the first neuron array; The last third neuron array is connected to the second neuron array and is configured to output a third neuron output pulse signal corresponding to the last third neuron array to the second neuron array; The middle third neuron array is configured to receive the third neuron output pulse signal output by the previous third neuron array, and output its corresponding third neuron output pulse signal to the next third neuron array.

8. The pulse processing circuit according to any one of claims 3 to 5, characterized in that: The detection result is generated by a preset feature processing circuit based on the feature data; The pulse processing circuit further includes: a configuration module, connected to the neural network module, configured to record configuration information of the pulse processing circuit; the configuration information is obtained by performing model training on the pulse processing circuit and the feature processing circuit based on the labeled training data; The configuration information includes at least one of the first weight and the threshold voltage.

9. The pulse processing circuit according to any one of claims 1 to 5, characterized in that: The neural network module is connected to a memory; the memory is configured to store the feature data, and to determine and store time domain information; The time domain information includes a timestamp corresponding to the characteristic data and the number of neural network input pulse signals corresponding to each of the photoelectric detection units per unit time.

10. A sensor, characterized in that: include: A pulse processing circuit according to any one of claims 1 to 9, and a photodetector connected to the pulse processing circuit.

11. The sensor according to claim 10, characterized in that There are multiple pulse processing circuits, and the photodetector includes multiple pixel areas; one pulse processing circuit is connected to one pixel area.

12. The sensor according to claim 11, characterized in that Also included is a memory connected to each of the pulse processing circuits.

13. The sensor according to claim 12, characterized in that Also included is a feature processing circuit connected to the memory.

14. An electronic device, characterized in that: The electronic device is provided with at least one sensor according to any one of claims 10-13.

15. A pulse processing method, characterized in that: Applied to the pulse processing circuit according to any one of claims 1 to 9, the method comprises: When receiving a digital pulse signal output by the photoelectric detection unit in response to a detected single photon, outputting a neural network input pulse signal corresponding to the photoelectric detection unit; In response to the neural network input pulse signals received at different times, feature data is generated, and the feature data is output; the feature data is used to generate a detection result.

16. A computer program product, characterized in that The invention comprises a computer program which, when executed, causes the method according to claim 15 to be performed.

Images