Signal processing apparatus, sensor chip, signal processing method, device, and medium

Abstract

A signal processing apparatus (100; 610), a sensor chip (600), a signal processing method, a device, and a medium, which belong to the technical field of computers. The signal processing apparatus (100; 610) comprises: a photosensitive module (110; 1101), which is configured to generate, at a preset sampling moment, an electrical signal corresponding to an optical signal incident into the photosensitive module (110; 1101); a storage module (120; 1201; 1202), which is configured to store an electrical signal of the photosensitive module (110; 1101), wherein the storage module (120; 1201) comprises at least two storage sub-modules, and each storage sub-module is configured to store an electrical signal of the photosensitive module (110; 1101) at a sampling moment; and a processing module (130), which is configured to process, on the basis of a preset signal processing mode, the electrical signals stored in the storage module (120; 1201; 1202), so as to obtain a signal processing result, wherein the signal processing result comprises at least one of a direct transmission result, a time difference result, a spatial difference result and a high-order difference result.

Description

Signal processing device, sensor chip, signal processing method, equipment and medium Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a signal processing device, a sensor chip, a signal processing method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Currently, most complementary metal oxide semiconductor (CMOS) image sensors (CIS) capture video based on the frame sampling principle, primarily recording the absolute value of the incident light intensity at all pixels in the pixel array, with each frame being equally spaced. An event camera is a new type of imaging system. Unlike a CIS, each pixel in an event camera only records the change in incident light intensity at its corresponding location, and only outputs a positive or negative pulse when the intensity change exceeds a certain threshold. Summary of the Invention

[0003] The present disclosure provides a signal processing device, a sensor chip, a signal processing method, an electronic device, and a computer-readable storage medium.

[0004] In a first aspect, the present disclosure provides a signal processing device, which includes: a photosensitive module for generating an electrical signal corresponding to a light signal incident to the photosensitive module at a preset sampling moment; a storage module for storing the electrical signal of the photosensitive module, wherein the storage module includes at least two storage sub-modules, and the storage sub-module is used to store the electrical signal of the photosensitive module at a sampling moment, and the electrical signal stored for the photosensitive module at any sampling moment includes at least one of the following: the electrical signal of the photosensitive module at the sampling moment, the electrical signal of the photosensitive module at the sampling moment and at least one historical sampling moment; a processing module for processing the electrical signal stored in the storage module according to a preset signal processing mode to obtain a signal processing result, and the signal processing result includes at least one of a direct input result, a time difference result, a spatial difference result and a high-order difference result.

[0005] In a second aspect, the present disclosure provides a sensor chip, which includes at least one signal processing device; wherein the signal processing device adopts the signal processing device of an embodiment of the present disclosure.

[0006] In a third aspect, the present disclosure provides a signal processing method, which includes: determining a signal processing mode based on a received mode setting instruction; processing the electrical signal of the photosensitive module stored in the storage module according to the signal processing mode to obtain a signal processing result; wherein the signal processing device adopts the signal processing device of an embodiment of the present disclosure.

[0007] In a fourth aspect, the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor to enable the at least one processor to execute the signal processing method of an embodiment of the present disclosure.

[0008] In a fifth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the signal processing method of an embodiment of the present disclosure when executed by a processor / processing core.

[0009] In a sixth aspect, the present disclosure provides a computer program product comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes a signal processing method for implementing an embodiment of the present disclosure.

[0010] In the embodiments provided by the present disclosure, in a signal processing device, a photosensitive module is used to generate an electrical signal corresponding to a light signal incident to the photosensitive module at a preset sampling moment; a storage module is used to store the electrical signal of the photosensitive module, wherein the storage module includes at least two storage sub-modules, and the storage sub-module is used to store the electrical signal of the photosensitive module at a sampling moment, and the electrical signal stored for the photosensitive module at any sampling moment includes at least one of the following: the electrical signal of the photosensitive module at the sampling moment, the electrical signal of the photosensitive module at the sampling moment and at least one historical sampling moment; a processing module is used to process the electrical signal stored in the storage module according to a preset signal processing mode to obtain a signal processing result, and the signal processing result includes at least one of a direct input result, a time difference result, a spatial difference result and a high-order difference result. It can be seen that the electrical signal generated by the photosensitive module can be stored in the storage module, and at least two storage submodules of the storage module can respectively store the electrical signal at a sampling moment, thereby providing a data basis for subsequent signal processing; further, the signal processing device supports multiple preset signal processing modes. Therefore, when processing based on the stored electrical signal, a suitable signal processing mode can be selected from them, and then the stored electrical signal is processed based on the signal processing mode to obtain at least one of a direct input result, a time difference result, a spatial difference result, and a high-order difference result. In other words, in the embodiment of the present disclosure, different signal processing modes are configured for the photosensitive module, so that the signal processing device can reuse the photosensitive module to realize multiple signal processing modes, thereby being applicable to various signal processing scenarios and having high flexibility.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, which are shown below.

[0013] FIG1 is a block diagram of a signal processing device provided by an embodiment of the present disclosure.

[0014] FIG2 is a schematic diagram of a signal processing device provided by an embodiment of the present disclosure.

[0015] FIG3 is a schematic diagram of a signal processing device provided by an embodiment of the present disclosure.

[0016] FIG4 is a schematic diagram of a processing process of a signal processing device provided by an embodiment of the present disclosure.

[0017] FIG5 is a schematic diagram of a processing process of a signal processing device provided by an embodiment of the present disclosure.

[0018] FIG6 is a schematic diagram of a sensor chip provided in an embodiment of the present disclosure.

[0019] FIG7 is a flowchart of a signal processing method provided by an embodiment of the present disclosure.

[0020] FIG8 is a block diagram of an electronic device provided by an embodiment of the present disclosure.

[0021] FIG9 is a block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] To enable those skilled in the art to better understand the technical solutions of the present disclosure, exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0024] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0025] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0026] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0027] In related technologies, image sensors support relatively few processing modes and cannot freely configure corresponding processing modes for pixels as required, thereby limiting the application scenarios of image sensors.

[0028] In view of this, embodiments of the present disclosure provide a signal processing device, a sensor chip, a signal processing method, an electronic device, and a computer-readable storage medium.

[0029] In the embodiments provided by the present disclosure, the electrical signal generated by the photosensitive module can be stored in a storage module, and at least two storage submodules of the storage module can respectively store the electrical signal at a sampling moment, thereby providing a data basis for subsequent signal processing; further, the signal processing device supports multiple preset signal processing modes. Therefore, when processing based on the stored electrical signal, a suitable signal processing mode can be selected from them, and then the stored electrical signal can be processed based on the signal processing mode to obtain at least one of a direct input result, a time difference result, a spatial difference result, and a high-order difference result. In other words, in the embodiments of the present disclosure, different signal processing modes are configured for the photosensitive module, so that the signal processing device can reuse the photosensitive module to realize multiple signal processing modes, thereby being applicable to various signal processing scenarios and having high flexibility.

[0030] A first aspect of an embodiment of the present disclosure provides a signal processing device.

[0031] FIG1 is a block diagram of a signal processing device provided by an embodiment of the present disclosure. Referring to FIG1 , the signal processing device 100 may include the following modules.

[0032] The photosensitive module 110 is configured to generate an electrical signal corresponding to the light signal incident on the photosensitive module at a preset sampling moment.

[0033] The storage module 120 is used to store the electrical signal of the photosensitive module, wherein the storage module includes at least two storage sub-modules, each storage sub-module is used to store the electrical signal of the photosensitive module at a sampling moment, and the electrical signal stored for each photosensitive module at any sampling moment includes at least one of the following: the electrical signal of the photosensitive module at the sampling moment, the electrical signal of the photosensitive module at the sampling moment and at least one historical sampling moment.

[0034] The processing module 130 is used to process the electrical signal stored in the storage module according to a preset signal processing mode to obtain a signal processing result, which includes at least one of a direct input result, a time difference result, a spatial difference result and a high-order difference result.

[0035] In an embodiment of the present disclosure, a photosensitive module is used to generate an electrical signal corresponding to a light signal incident to the photosensitive module at a preset sampling moment; a storage module is used to store the electrical signal of the photosensitive module, wherein the storage module includes at least two storage sub-modules, each storage sub-module is used to store the electrical signal of the photosensitive module at a sampling moment, and the electrical signal stored for each photosensitive module at any sampling moment includes at least one of the following: the electrical signal of the photosensitive module at the sampling moment, the electrical signal of the photosensitive module at the sampling moment and at least one historical sampling moment; a processing module is used to process the electrical signal stored in the storage module according to a preset signal processing mode to obtain a signal processing result, and the signal processing result includes at least one of a direct input result, a time difference result, a spatial difference result and a high-order difference result. It can be seen that the electrical signal generated by the photosensitive module can be stored in the storage module, and at least two storage submodules of the storage module can respectively store the electrical signal at a sampling moment, thereby providing a data basis for subsequent signal processing; further, the signal processing device supports multiple preset signal processing modes; therefore, when processing based on the stored electrical signal, a suitable signal processing mode can be selected from it, and then the stored electrical signal can be processed based on the signal processing mode to obtain at least one of a direct input result, a time difference result, a spatial difference result, and a high-order difference result. In other words, in the embodiment of the present disclosure, different signal processing modes are configured for the photosensitive module, so that the signal processing device can reuse the photosensitive module to realize multiple signal processing modes, thereby being applicable to various signal processing scenarios and having high flexibility.

[0036] In some optional implementations, the photosensitive module is a functional module with photoelectric conversion capability, which can convert the light signal incident on the photosensitive module into an electrical signal that is more easily processed by the electronic device.

[0037] In some optional implementations, the photosensitive module may sample and convert the incident light signal according to a preset sampling period, thereby generating an electrical signal corresponding to the preset sampling moment.

[0038] For example, the preset sampling period is T, and the photosensitive module can generate electrical signals at sampling time t0, sampling time t1, sampling time t2, ..., sampling time tn, where n is an integer greater than or equal to 1, and the time difference between two adjacent sampling times is T.

[0039] In some optional implementations, the electrical signal includes a sensing signal, which is a signal generated by the photosensitive module through photoelectric sensing of the light signal, and the sensing signal is a signal including spatial and temporal dimensions; or, the electrical signal includes a sensing signal and a reset signal corresponding to the sensing signal, and the reset signal is used to reset the sensing signal.

[0040] As can be seen, the composition of the electrical signal includes two cases: the first case, the electrical signal only includes the sensing signal, and the second case, the electrical signal includes the sensing signal and the corresponding reset signal. The spatial dimension of the sensing signal can be determined by the position information of the pixel module generating the sensing signal, and the temporal dimension of the sensing signal can be determined by the sampling time of the sensing signal. For example, the sensing signal can be expressed as S(x, y, t), where x and y represent the horizontal and vertical positions of the photosensitive module generating the sensing signal, respectively, and t represents the sampling time corresponding to the sensing signal.

[0041] In some optional implementations, in the signal processing device, there may be multiple photosensitive modules, and the multiple photosensitive modules may be arranged in an array to form a corresponding photosensitive array.

[0042] In some optional implementations, the multiple photosensitive modules may correspond to the same type of photosensitive modules or to different types of photosensitive modules, which is not limited in the embodiments of the present disclosure.

[0043] Exemplarily, there is at least one of the following situations for multiple photosensitive modules: multiple photosensitive modules are color photosensitive modules; multiple photosensitive modules are grayscale photosensitive modules; some photosensitive modules are color photosensitive modules, and some photosensitive modules are grayscale photosensitive modules; among which, the color photosensitive module is a photosensitive module for collecting color information, and the grayscale photosensitive module is a photosensitive module for collecting brightness information.

[0044] Exemplarily, the color sensor module includes an RGB sensor module and / or a YUV sensor module, and the electrical signal generated by the module is a signal containing color information. "RGB" stands for Red, Green, and Blue, respectively; "Y" represents brightness (Luminance or Luma); and "U" and "V" represent chrominance (Chrominance or Chroma).

[0045] In some optional implementations, for a photosensitive array in which some photosensitive modules are color photosensitive modules and some photosensitive modules are grayscale photosensitive modules, the color photosensitive modules and the grayscale photosensitive modules can be arranged at equal intervals or at unequal intervals. The color photosensitive modules can also be concentrated in a certain array area and the grayscale photosensitive modules can be concentrated in another array area. The embodiments of the present disclosure do not limit this.

[0046] In some optional implementations, the grayscale photosensitive module can be regarded as a dynamic vision sensor (DVS) photosensitive module. Compared with the color photosensitive module, the grayscale photosensitive module has a higher sensitivity to light intensity and can collect light intensity signals with higher intensity and higher accuracy.

[0047] In some optional implementations, the storage module is a functional module for storing the electrical signal of the photosensitive module.

[0048] In some optional implementations, the storage module includes at least two storage submodules, each of which can store the electrical signal of the photosensitive module at a sampling moment. The electrical signal stored for each photosensitive module at any sampling moment includes at least one of the following: the electrical signal of the photosensitive module at the sampling moment, the electrical signal of the photosensitive module at the sampling moment, and the electrical signal of the photosensitive module at at least one historical sampling moment. The historical sampling moment is a sampling moment before the current sampling moment.

[0049] In some optional implementations, the storage module includes two storage sub-modules; wherein, at the j-1th sampling moment, the two storage sub-modules respectively store the electrical signal at the j-1th sampling moment and the electrical signal at the j-2th sampling moment; at the j-th sampling moment, the storage sub-module storing the electrical signal at the j-2th sampling moment is used as the target storage sub-module, the electrical signal at the j-th sampling moment is stored in the target storage sub-module, and the electrical signal at the j-1th sampling moment stored by the other storage sub-module is retained, where j is an integer and j>2.

[0050] Among them, at the first sampling moment, only the electrical signal of the first sampling moment can be stored. As soon as the second sampling moment is entered, the electrical signals of the first sampling moment and the second sampling moment can be obtained, thereby supporting differential operations in the time dimension. In addition, the electrical signals of different photosensitive modules can also be differentiated in the spatial dimension, and it is also possible to perform differentiation in the time dimension and the spatial dimension at the same time (i.e., high-order differentiation), which is not limited in the embodiments of the present disclosure.

[0051] In some optional implementations, there are multiple photosensitive modules, and there is a corresponding relationship between the photosensitive modules and the storage modules. The storage modules are used to store the electrical signals of the photosensitive modules with the corresponding relationship.

[0052] In some optional implementations, the storage module can be a shared functional module that can provide storage support for at least two photosensitive modules. In other words, the correspondence between the photosensitive modules and the storage modules is a many-to-one correspondence, that is, multiple photosensitive modules correspond to one storage module.

[0053] It should be noted that for a shared storage module, the electrical signals of multiple photosensitive modules stored in the storage module can be distinguished by carrying the photosensitive module identifier in the electrical signal, or setting up an independent storage space for each corresponding photosensitive module in the storage module.

[0054] Exemplarily, it is possible to determine which photosensitive modules share the same storage module based on the positions of the photosensitive modules. For example, multiple photosensitive modules in the same row (column) share a storage module, and / or multiple photosensitive modules in L adjacent rows (columns) share a storage module, and / or multiple photosensitive modules within a predetermined area share a storage module, where L>1 and the predetermined area can be a regular or irregular area.

[0055] Exemplarily, which photosensitive modules share the same storage module can be determined based on the type of the photosensitive modules. For example, color photosensitive modules share the same storage module (or multiple storage modules), and grayscale photosensitive modules share the same storage module (or multiple storage modules).

[0056] Exemplarily, it is also possible to determine which photosensitive modules share the same storage module based on the location and type of the photosensitive modules. For example, multiple color photosensitive modules within a certain preset area share a storage module, and multiple grayscale photosensitive modules within another preset area share a storage module.

[0057] In some optional implementations, the correspondence between the photosensitive modules and the storage modules can be a one-to-one correspondence, that is, each photosensitive module has a dedicated storage module, which only stores the electrical signals of the corresponding photosensitive module, and does not store the electrical signals of other photosensitive modules.

[0058] In some optional implementations, the storage module may include at least two storage sub-modules; accordingly, the processing module is also used to select a target storage sub-module from the storage module corresponding to the photosensitive module when the photosensitive module generates an electrical signal at the current sampling moment, and store the electrical signal at the current sampling moment in the target storage sub-module.

[0059] It should be noted that if the electrical signal at the current sampling moment needs to overwrite the electrical signal at one (or several) historical sampling moments when it is stored, before overwriting, it is necessary to ensure that the electrical signal at the historical sampling moment to be overwritten has been read out, or has been transferred, or has been used, or will not be used again, so as to avoid the situation where the corresponding signal processing cannot be performed due to data overwriting.

[0060] In some optional implementations, the storage submodule may include at least one storage unit, and each storage unit is provided with a switch; accordingly, the processing module may store the electrical signal in a storage unit by turning on the switch of the storage unit.

[0061] For example, the switch can be implemented by a transistor, and the transistor can be turned on or off by adjusting the voltage between the base and emitter of the transistor, thereby realizing the opening and closing of the switch.

[0062] FIG2 is a schematic diagram of a signal processing device provided by an embodiment of the present disclosure. Referring to FIG2 , the signal processing device 100 includes m photosensitive modules, each of which corresponds to a storage module. Each storage module includes multiple storage submodules (not shown in the figure), each of which corresponds to n storage cells. Each storage cell is provided with a switch, and the closing and opening of the switch determines whether data can be written to or read from the storage cell.

[0063] As shown in Figure 2 , photosensitive module 1101 corresponds to storage module 1201, which includes storage units 12011, 12012, ..., and 1201n. Photosensitive module 1102 corresponds to storage module 1202, which includes storage units 12021, 12022, ..., and 1202n. Photosensitive module 110m corresponds to storage module 120m, which includes storage units 120m1, 120m2, ..., and 120mn. For any photosensitive module, based on the electrical signal generated at sampling time ti, processing module 130 selects at least one target storage unit based on the contents stored in each storage unit of the storage module corresponding to that photosensitive module, turns on the switch of the target storage unit, and writes the electrical signal Si corresponding to ti into the target storage unit. Furthermore, except for the target memory cell, the switches of the other memory cells in the memory module are closed. Therefore, the data stored in these memory cells is retained and not lost. After determining which signal processing mode to adopt, the processing module can determine which electrical signals are required, thereby reading these electrical signals from the memory module and performing corresponding signal processing based on the read electrical signals.

[0064] In some optional implementations, the memory cell may be prepared by a capacitor device, and a plurality of memory cells may be stacked, thereby reducing the occupied chip area and improving the chip area utilization.

[0065] In some optional implementations, the photosensitive module is composed of a pinned photodiode (PPD) and several transistors and other devices, the storage module (or storage unit) is composed of several capacitors, and the switch of the storage unit is also made of transistors. Among them, PPD is a semiconductor device built based on the photoelectric conversion function. The PPD generates an initial electrical signal through photoelectric conversion, and the initial electrical signal passes through the transmission tube M TG , gain adjustment tube M DCG , reset tube M RST , source follower M SF After a series of processing, the final electrical signal is obtained under the action of the rolling shutter.

[0066] For example, the storage module may include four storage units, corresponding to capacitor C1, capacitor C2, capacitor C3 and capacitor C4 respectively, and the above four capacitors are respectively connected to the gate transistor M1. SEL1 , gate tube M SEL2 , gate tube M SEL3 and gate tube M SEL4 Controls the switch status.

[0067] After receiving the electrical signal, the processing module stores the electrical signal in the gated capacitor by gating the gate transistor of at least one capacitor. The charge in the capacitor that is not gated or whose gate transistor is off does not move, so the information in the capacitor does not change, and the stored historical information (for example, the electrical signal at the historical sampling moment) is preserved.

[0068] In some optional implementations, multiple pixel modules are arranged in an array to form a pixel array. Each pixel module includes a photosensitive module and its corresponding storage module, and each photosensitive module and its corresponding storage module are integrated together to facilitate the storage of electrical signals. For any pixel module in the pixel array, a photosensitive module circuit composed of a PPD and multiple transistors, and a storage module circuit composed of multiple capacitors and multiple gate transistors (for example, four capacitors and four gate transistors) are integrated, wherein one capacitor and one gate transistor can be regarded as a storage unit.

[0069] FIG3 is a schematic diagram of a signal processing device provided by an embodiment of the present disclosure. Referring to FIG3 , the signal processing device 100 includes m photosensitive modules (including photosensitive modules 1101, ..., 110k, ..., 110q, ..., 110m), and the m photosensitive modules are arranged in a q×k array to form an array, wherein q, k, and m are all integers greater than 1, and q < m, k < m. Furthermore, the m photosensitive modules share a storage module 120, and the processing module 130 can process the electrical signals stored in the storage module 120 according to a preset signal processing mode to obtain a signal processing result corresponding to the signal processing mode.

[0070] In some optional implementations, the electrical signal generated by the photosensitive module includes an induction signal, each photosensitive module corresponds to a storage module, and each storage module includes two storage units, namely a first storage unit and a second storage unit. For each sampling moment (for example, t1, t2, t3, t4, etc.), the information stored in a storage module is shown in Table 1.

[0071] Table 1 Schematic diagram of storage information of storage module

[0072] As shown in Table 1, for any photosensitive module, a sensing signal S1 is generated at sampling time t1. Since both storage cells of the storage module corresponding to the photosensitive module can store data, it is sufficient to enable the switch of either storage cell to store S1 in the enabled storage cell (this storage cell is the target storage cell). As shown in Table 1, if the switch of the first storage cell is enabled, S1 can be stored in the first storage cell. At this time, the first storage cell stores S1, and the second storage cell does not store any data.

[0073] At sampling time t2, before the corresponding sensing signal S2 is stored, since the first storage unit stores S1 and the second storage unit does not store data, the second storage unit is determined as the target storage unit, and the switch of the second storage unit is turned on to store S2 in the second storage unit, while the switch of the first storage unit is turned off to retain S1 stored in the first storage unit.

[0074] For sampling time t3, before the corresponding sensing signal S3 is stored, the first storage unit stores S1 and the second storage unit stores S2. Since the historical sampling time of S1 is earlier than the historical sampling time of S2, the first storage unit storing S1 is selected as the target storage unit, the switch of the first storage unit is turned on to store S3 in the first storage unit, and the switch of the second storage unit is turned off to retain S2 stored in the second storage unit.

[0075] At sampling time t4, before the corresponding sensing signal S4 is stored, the first storage unit stores S3 and the second storage unit stores S2. Since the historical sampling time of S2 is earlier than the historical sampling time of S3, the second storage unit storing S2 is selected as the target storage unit, and the switch of the second storage unit is turned on to store S4 in the second storage unit, while the switch of the first storage unit is turned off to retain S3 stored in the first storage unit.

[0076] By analogy, the sensing signals stored in the storage module at each sampling moment can be obtained for subsequent signal processing.

[0077] Table 2 shows the information stored at each sampling time (eg, t1, t2, t3, t4, etc.) when the storage module includes three storage units (a first storage unit, a second storage unit, and a third storage unit).

[0078] Table 2 Schematic diagram of storage information of storage module

[0079] As shown in Table 2, at sampling time t1, a sensing signal S1 is generated. Since all three storage cells can store data, turning on the switch of any one of them can store S1 in the selected storage cell (this storage cell is the target storage cell). As shown in Table 2, if the switch of the first storage cell is turned on, S1 can be stored in the first storage cell. At this time, the first storage cell stores S1, while the second and third storage cells do not store data.

[0080] At sampling time t2, before the corresponding sensing signal S2 is stored, since the first storage unit stores S1 and the second and third storage units do not store data, the second storage unit can be determined as the target storage unit, and the switch of the second storage unit is turned on to store S2 in the second storage unit, while the switch of the first storage unit is turned off to retain S1 stored in the first storage unit.

[0081] At sampling time t3, before the corresponding sensing signal S3 is stored, the first storage unit stores S1, the second storage unit stores S2, and the third storage unit does not store any data. Therefore, the third storage unit is selected as the target storage unit, and the switch of the third storage unit is turned on to store S3 in the first storage unit. At the same time, the switches of the first storage unit and the second storage unit are turned off to retain S1 stored in the first storage unit and S2 stored in the second storage unit.

[0082] For sampling time t4, before the corresponding sensing signal S4 is stored, the first storage unit stores S1, the second storage unit stores S2, and the third storage unit stores S3. Since the historical sampling time of S1 is earlier than the historical sampling times of S2 and S3, the first storage unit storing S1 is selected as the target storage unit, the switch of the first storage unit is turned on, and S4 is stored in the first storage unit. At the same time, the switches of the second storage unit and the third storage unit are turned off to retain S2 stored in the second storage unit and S3 stored in the third storage unit.

[0083] By analogy, the sensing signals stored in the storage module at each sampling moment can be obtained for subsequent signal processing.

[0084] In some optional implementations, each photosensitive module corresponds to a storage module, and the storage module includes two storage sub-modules, the first storage sub-module includes a first storage unit and a second storage unit, and the second storage sub-module includes a third storage unit and a fourth storage unit; accordingly, the processing module is also used to: for the i-th sampling moment, when the first storage unit and the second storage unit respectively store the sensing signal and the reset signal of the i-1-th sampling moment, and the third storage unit and the fourth storage unit respectively store the sensing signal and the reset signal of the i-2-th sampling moment, turn off the switches of the first storage unit and the second storage unit, and turn on the switches of the third storage unit and the fourth storage unit, so that the third storage unit stores the sensing signal of the i-th sampling moment, and the fourth storage unit stores the reset signal corresponding to the sensing signal of the i-th sampling moment, where i is an integer and i>2.

[0085] Table 3 shows the information stored at each sampling time (eg, t1, t2, t3, t4, etc.) when the storage module includes four storage units (a first storage unit, a second storage unit, a third storage unit, and a fourth storage unit).

[0086] Table 3 Schematic diagram of storage information of storage module

[0087] As shown in Table 3, at sampling time t1, a sensing signal S1 and a reset signal R1 are generated. Since all four storage cells can store data, it is sufficient to enable the switches of any two of them to store S1 and R1 in the two selected storage cells (these storage cells are the target storage cells). As shown in Table 3, if the switches of the first and second storage cells are enabled, S1 can be stored in the first storage cell and R1 can be stored in the second storage cell. At this time, the first storage cell stores S1, the second storage cell stores R1, and the third and fourth storage cells do not store data.

[0088] For the sampling time t2, before the corresponding sensing signal S2 and reset signal R2 are stored, since the first storage unit stores S1, the second storage unit stores R1, and the third storage unit and the fourth storage unit do not store data, the third storage unit and the fourth storage unit can be determined as target storage units, and the switch of the third storage unit is turned on to store S2 in the third storage unit, and the switch of the fourth storage unit is turned on to store R2 in the fourth storage unit, and at the same time, the switches of the first storage unit and the second storage unit are turned off to retain S1 stored in the first storage unit and R1 stored in the second storage unit.

[0089] For the sampling time t3, before the corresponding sensing signal S3 and reset signal R3 are stored, since the first storage unit stores S1, the second storage unit stores R1, the third storage unit stores S2, and the fourth storage unit stores R2, and S3 and R3 need to occupy two storage units, the historical sampling time of S1 and R1 is earlier than the historical sampling time of S2 and R2. Therefore, the first storage unit and the second storage unit can be determined as the target storage units, and the switch of the first storage unit is turned on to store S3 in the first storage unit, the switch of the second storage unit is turned on to store R3 in the second storage unit, and the switches of the third storage unit and the fourth storage unit are turned off to retain S2 stored in the third storage unit and R2 stored in the fourth storage unit.

[0090] For the sampling time t4, before the corresponding sensing signal S4 and reset signal R4 are stored, since the first storage unit stores S3, the second storage unit stores R3, the third storage unit stores S2, and the fourth storage unit stores R2, and S4 and R4 need to occupy two storage units, the historical sampling time of S2 and R2 is earlier than the historical sampling time of S3 and R3. Therefore, the third storage unit and the fourth storage unit can be determined as the target storage units, and the switch of the third storage unit is turned on to store S4 in the third storage unit, and the switch of the fourth storage unit is turned on to store R4 in the fourth storage unit. At the same time, the switches of the first storage unit and the second storage unit are turned off to retain S3 stored in the first storage unit and R3 stored in the second storage unit.

[0091] By analogy, the sensing signals stored in the storage module at each sampling moment can be obtained for subsequent signal processing.

[0092] In some optional implementations, the pixel module further includes a gain adjustment unit, and the gain adjustment unit is configured to adjust the conversion gain according to the ambient brightness of the pixel module and generate a sensing signal corresponding to the conversion gain.

[0093] In some optional implementations, the gain adjustment unit is used to generate an nth gain sensing signal based on the nth conversion gain when the ambient brightness is greater than the n-1th brightness threshold and less than or equal to the nth brightness threshold, wherein n≥2, and the gain adjustment unit generates a first gain sensing signal based on the first conversion gain when the ambient brightness is less than the first brightness threshold.

[0094] In some optional implementations, the conversion gain corresponding to the photosensitive module includes a first gain and a second gain; the sensing signal includes a first gain sensing signal corresponding to the first gain and / or a second gain sensing signal corresponding to the second gain, and the reset signal includes a first gain reset signal corresponding to the first gain and / or a second gain reset signal corresponding to the second gain.

[0095] It should be noted that by setting different conversion gains, the signal readout noise can be reduced and the signal-to-noise ratio can be improved in dark light scenes, and the charge capacity can be increased and the saturation can be reduced in bright light scenes.

[0096] Table 4 shows the information stored for each sampling time (for example, t1, t2, t3, t4, etc.) when the storage module includes four storage units (a first storage unit, a second storage unit, a third storage unit, and a fourth storage unit) and the gain adjustment unit corresponds to the first gain and the second gain.

[0097] Table 4 Schematic diagram of storage information of storage module

[0098] At sampling time t1, the photosensitive module obtains a corresponding sensing signal HS1 and a reset signal HR1 corresponding to HS1 based on the first gain. The photosensitive module obtains a corresponding sensing signal LS1 and a reset signal LR1 corresponding to LS1 based on the second gain. Furthermore, since all four storage cells can store data, the switches of these four storage cells can be turned on to store HS1 in the first storage cell, HR1 in the second storage cell, LS1 in the third storage cell, and LR1 in the fourth storage cell.

[0099] At sampling time t2, the gain adjustment unit determines the conversion gain to be used based on the ambient brightness and generates a sensing signal and a reset signal corresponding to sampling time t2 based on the determined conversion gain. Furthermore, if the second gain is determined to be used, a sensing signal LS2 and a reset signal LR2 corresponding to t2 can be generated. Because the sampling times of the signals stored in the first through fourth storage units are the same before LS2 and LR2 are stored, the first and second storage units are randomly selected as target storage units. The switch of the first storage unit is turned on to store LS2 in the first storage unit, the switch of the second storage unit is turned on to store LR2 in the second storage unit, and the switches of the third and fourth storage units are turned off to retain LS1 stored in the third storage unit and LR1 stored in the fourth storage unit.

[0100] At sampling time t3, the gain adjustment unit determines the conversion gain to be used based on the ambient brightness and generates a sensing signal and a reset signal corresponding to sampling time t3 based on the determined conversion gain. Furthermore, if the second gain is determined to be used, a sensing signal LS3 and a reset signal LR3 corresponding to t3 can be generated. Because, before LS3 and LR3 are stored, the first and second storage units store LS2 and LR2, respectively, and the third and fourth storage units store LS1 and LR1, respectively, and the historical sampling times corresponding to LS1 and LR1 are earlier than those corresponding to LS2 and LR2, the third and fourth storage units are selected as target storage units, and the switch of the third storage unit is turned on to store LS3 in the third storage unit, and the switch of the fourth storage unit is turned on to store LR3 in the fourth storage unit. Simultaneously, the switches of the first and second storage units are turned off to retain LS2 stored in the first storage unit and LR2 stored in the second storage unit.

[0101] At sampling time t4, the gain adjustment unit determines the conversion gain to be used based on the ambient brightness and generates a sensing signal and a reset signal corresponding to sampling time t4 based on the determined conversion gain. Furthermore, if the first gain is determined to be used, a sensing signal HS4 and a reset signal HR4 corresponding to t4 can be generated. Because, before HS4 and HR4 are stored, the first and second storage units store LS2 and LR2, respectively, and the third and fourth storage units store LS3 and LR3, respectively, and the historical sampling times corresponding to LS2 and LR2 are earlier than those corresponding to LS3 and LR3, the first and second storage units are selected as target storage units, and the switch of the first storage unit is turned on to store HS4 in the first storage unit, the switch of the second storage unit is turned on to store HR4 in the second storage unit, and the switches of the third and fourth storage units are turned off to retain LS3 stored in the third storage unit and LR3 stored in the fourth storage unit.

[0102] By analogy, the sensing signals stored in the storage module at each sampling moment can be obtained for subsequent signal processing.

[0103] In some optional implementations, considering that the ambient light has a certain continuity, the conversion gain may be adjusted once every period of time, rather than before each sampling moment.

[0104] For example, the conversion gain may be reset after a preset reset period, and signal sampling may be performed at a new sampling time based on the reset conversion gain. For example, the sampling time corresponds to a sampling period of T1, a reset period of T2, and T2 = N × T1, where N > 1.

[0105] Table 5 shows the information stored at each sampling time (for example, t1, t2, t3, t4, etc.) when the storage module includes four storage units (a first storage unit, a second storage unit, a third storage unit, and a fourth storage unit) and the gain adjustment unit resets the conversion gain based on a preset reset period.

[0106] Table 5 Schematic diagram of storage information of storage module

[0107] As shown in Table 5, the conversion gain is reset every 30 sampling moments. At time t1, the photosensitive module obtains a corresponding sensing signal HS1 and a reset signal HR1 corresponding to HS1 based on the first gain. The photosensitive module obtains a corresponding sensing signal LS1 and a reset signal LR1 corresponding to LS1 based on the second gain. Since all four storage cells can store data, the switches of these four storage cells can be turned on to store HS1 in the first storage cell, HR1 in the second storage cell, LS1 in the third storage cell, and LR1 in the fourth storage cell.

[0108] Furthermore, the processing module compares the sensing signals of the different conversion gains to determine which conversion gain is more suitable, and selects one of them as the conversion gain for the subsequent 29 sampling moments. For example, if the second gain is determined to be more suitable than the first gain, then at sampling moment t2, signal acquisition and conversion are performed based on the second gain and stored in the corresponding target storage unit. The selection of the target storage unit can be found in the relevant content of the embodiments of the present disclosure and will not be described in detail here.

[0109] At sampling time t31, a new reset cycle begins, and the conversion gain needs to be reset. Similarly, at sampling time t61, a new reset cycle begins, and the conversion gain needs to be reset again, and so on. For details on resetting the conversion gain, please refer to the relevant content from sampling time t1 to sampling time t30, and will not be described in detail here.

[0110] It should be noted that for the case of three or more conversion gains, the processing process is similar to the process shown in Table 5. Similarly, when each reset cycle is reached, the conversion gain to be selected in the current reset cycle is re-determined, and signal acquisition is performed based on the re-determined conversion gain.

[0111] In some optional implementations, the signal processing device supports multiple preset signal processing modes. The processing module can select a signal processing mode to process the electrical signal stored in the storage module based on instructions, experience, statistical data, simulation results, etc., and can also replace the current signal processing mode with another signal processing mode as needed.

[0112] In some optional implementations, the signal processing mode includes at least one of a direct input mode, a temporal difference mode, a spatial difference mode, and a high-order difference mode; the signal processing result includes at least one of a direct input result, a temporal difference result, a spatial difference result, and a high-order difference result; accordingly:

[0113] The processing module is used to read the electrical signals of the multiple photosensitive modules at the same sampling moment from the storage module when the signal processing mode is the direct input mode, and obtain the direct input result corresponding to the sampling moment based on the electrical signals of the multiple photosensitive modules at the same sampling moment;

[0114] The processing module is configured to, when the signal processing mode is the time difference mode, read from the storage module the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the photosensitive module at at least one historical sampling moment, and perform time difference based on the electrical signal at the current sampling moment and the electrical signal at at least one historical sampling moment to obtain a time difference result corresponding to the photosensitive module;

[0115] The processing module is configured to read, from the storage module, the electrical signal of the photosensitive module and the electrical signal of the adjacent photosensitive module at the same sampling moment when the signal processing mode is the spatial difference mode, and perform spatial difference based on the electrical signal of the photosensitive module and the electrical signal of the adjacent photosensitive module to obtain a spatial difference result corresponding to the photosensitive module;

[0116] The processing module is used to read the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the adjacent photosensitive module at at least one historical sampling moment from the storage module when the signal processing mode is a high-order differential mode, and perform time differentiation based on the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the adjacent photosensitive module at at least one historical sampling moment to obtain a high-order differential result corresponding to the photosensitive module.

[0117] It should be noted that the term "adjacent photosensitive module" is a broad concept, encompassing both photosensitive modules adjacent to the current photosensitive module and photosensitive modules that are not adjacent to the current photosensitive module but are relatively close. For example, if multiple photosensitive modules form a pixel array, then photosensitive modules adjacent to the current photosensitive module are considered adjacent photosensitive modules. Photosensitive modules that are closer than a preset distance threshold to the current photosensitive module are also considered adjacent photosensitive modules.

[0118] In some optional implementations, the first execution mode of the time difference mode includes at least one of the following: global synchronous execution with a fixed time difference step, global synchronous execution with an adjustable time difference step, and asynchronous execution; the second execution mode of the spatial difference mode includes at least one of the following: global synchronous execution with a fixed spatial difference step, global synchronous execution with an adjustable spatial difference step, and asynchronous execution.

[0119] It can be seen from this that for multiple photosensitive modules, time differentiation can be performed globally and synchronously, or asynchronously and dispersedly, and the time differentiation step can be a fixed value or an adjustable non-fixed value, and the time differentiation step can be globally the same or not completely the same, and the embodiments of the present disclosure do not limit this. Similarly, when performing spatial differentiation for multiple photosensitive modules, it can also be performed globally and synchronously, or asynchronously and dispersedly, and the spatial differentiation step can be a fixed value or an adjustable non-fixed value, and the spatial differentiation step can be globally the same or not completely the same, and the embodiments of the present disclosure do not limit this.

[0120] In some optional implementations, the signal processing device also includes a processing path corresponding to the signal processing mode; accordingly, the processing module is used to select the processing path corresponding to the signal processing mode, and process the electrical signal stored in the storage module based on the selected processing path to obtain a signal processing result.

[0121] It can be seen from this that in the embodiment of the present disclosure, corresponding processing paths are configured for different signal processing modes. After the signal processing mode is determined, the corresponding signal processing can be performed through the processing path corresponding to the signal processing mode.

[0122] In some optional implementations, the signal processing mode includes at least one of a direct input mode, a temporal differential mode, a spatial differential mode, and a high-order differential mode; and the processing pathway includes at least one of a direct input pathway corresponding to the direct input mode, a temporal differential pathway corresponding to the temporal differential mode, a spatial differential pathway corresponding to the spatial differential mode, and a high-order differential pathway corresponding to the high-order differential mode. The temporal differential pathway is equivalent to a dynamic vision sensor (DVS) processing pathway, enabling event-based signal output.

[0123] For example, the direct input path may include a color path, which is mainly used to process the signal of the color photosensitive module. The color path outputs the sampling time t n In this case, the absolute value of the electrical signal of the photosensitive module corresponding to the position (x, y) can be expressed as: RGB(x, y, t n )=I(x,y,t n ) (1)

[0124] In formula (1), I(x,y,t n ) represents the electrical signal (also known as the visual signal), RGB (x, y, t n ) indicates the output color signal.

[0125] The time difference path is mainly used to output the time difference value of the electrical signal of the photosensitive module corresponding to the position (x, y) at different sampling times, which can be expressed as: TD(x, y, t n )=Q TD (I(x,y,t n )-I(x,y,t n-1 )) (2)

[0126] In formula (2), t n and t n-1 Represents two sampling moments, TD(x,y,t n ) represents the time difference signal, Q TD Indicates the quantization method of the time difference signal. This quantization method can be multi-valued (>1 bit, that is, multi-valued time difference mode) or single-valued (for example, positive and negative pulses). Among them, the sampling time t of the electrical signal involved in the time difference is n , t n-1 The whole array may be synchronized with the same time interval, or the whole array may be synchronized with a variable time interval, or the whole array may be asynchronous.

[0127] The spatial differential path is mainly used to output the sampling time t n The spatial difference value of the electrical signal between the photosensitive module corresponding to the position (x, y) and its adjacent photosensitive module (which can be oblique, horizontal or vertical). For the horizontal spatial difference, it can be expressed as: SD x (x,y,t n )=Q SD (I(x,y,t n )-I(x-1,y,t n )) (3)

[0128] In formula (3), SD x (x,y,t n ) represents the spatial differential signal in the horizontal direction.

[0129] For the vertical spatial difference, it can be expressed as: SD y (x,y,t n )=Q SD (I(x,y,t n )-I(x,y-1,t n )) (4)

[0130] In formula (4), SD y (x,y,t n ) represents the spatial differential signal in the vertical direction.

[0131] For oblique spatial differences, it can be expressed as: SD↙ (x,y,t n )=Q SD (I(x,y,t n )-I(x-1,y-1,t n )) (5) SD ↘ (x,y,t n )=Q SD (I(x,y,t n )-I(x+1,y-1,t n )) (6)

[0132] In formulas (5) and (6), SD ↙ (x,y,t n ) represents the spatial differential signal in the 135 direction, SD ↘ (x,y,t n ) represents the spatial differential signal in the 45-degree direction (with the three o'clock direction as the starting direction and the clockwise direction as the positive direction of rotation).

[0133] In addition, Q SD The quantization method of the spatial differential signal can be multi-valued (>1 bit, i.e. multi-valued spatial differential mode) or single-valued (e.g. positive and negative pulses). n ,t n-1 The whole array may be synchronized with the same time interval, or the whole array may be synchronized with a variable time interval, or the whole array may be asynchronous.

[0134] It should be noted that the above signal processing modes and processing paths are merely examples, and the embodiments of the present disclosure do not limit them.

[0135] It can be seen from this that the signal processing device of the embodiment of the present disclosure supports multiple signal processing modes. It can directly output electrical signals, or output differential processing results after performing differential processing on the stored electrical signals in the time dimension and / or space dimension.

[0136] Exemplarily, the processing module is used to read the electrical signal at the first sampling moment from the storage module corresponding to the photosensitive module for each photosensitive module when the signal processing mode is the direct input mode, and obtain the direct input result corresponding to the first sampling moment based on the electrical signals of multiple photosensitive modules at the first sampling moment.

[0137] For example, the electrical signal of the photosensitive module P1 at sampling time t1 is S1, the electrical signal at sampling time t2 is S2, ..., and the electrical signal at sampling time tn is Sn. In direct input mode, the processing module directly outputs a signal sequence: S1, S2, ..., Sn. If multiple photosensitive modules form a pixel array, and each pixel corresponds to the direct input mode, the signal sequence output by each photosensitive module is arranged according to the time dimension and combined according to the position of the photosensitive module, n video frames can be obtained, each of which includes the electrical signals of multiple photosensitive modules at the corresponding sampling time. Furthermore, if the photosensitive module is a color photosensitive module, the electrical signal is a color signal, and accordingly, the video frame is a color video frame. If the photosensitive module is a grayscale photosensitive module, the electrical signal is a grayscale signal (corresponding to brightness information), and accordingly, the video frame is a grayscale video frame.

[0138] Exemplarily, the processing module is used to read the electrical signal at the second sampling moment and the electrical signal at the third sampling moment from the storage module corresponding to the photosensitive module when the signal processing mode is the time difference mode, and perform time difference based on the electrical signal at the second sampling moment and the electrical signal at the third sampling moment to obtain a time difference result corresponding to the photosensitive module, and the second sampling moment and the third sampling moment are different sampling moments.

[0139] For example, taking Table 1 as an example, if the time difference step is one sampling period, a time difference can be performed based on S2 and S1 to obtain a time difference result, and a time difference can be performed based on S3 and S2 to obtain a time difference result, and so on.

[0140] For example, still taking Table 1 as an example, if the time difference step is two sampling periods, a time difference can be performed based on S3 and S1 to obtain a time difference result, and a time difference can be performed based on S4 and S2 to obtain a time difference result, and so on.

[0141] In some optional implementations, the time difference mode includes a polarity time difference mode and / or a multi-value time difference mode. Among them, the polarity time difference mode means that the time difference result is a result with a positive or negative sign, and does not represent a specific numerical value. For example, based on the polarity time difference mode, the electrical signals S2 and S1 are time-differentiated. If S2 is greater than S1, a positive signal P1 is output, and if S2 is less than or equal to S1, a negative signal P2 is output. The multi-value time difference mode refers to pre-setting multiple value intervals and assigning a unified value to each value interval. As long as the signal difference of the electrical signals at different sampling moments falls into a certain value interval, the time difference result is determined according to the unified value corresponding to the value interval. The smaller the value interval, the higher the accuracy of the multi-value time difference result. The value interval can be determined according to actual needs, and the embodiments of the present disclosure do not impose any restrictions on this.

[0142] Exemplarily, when the signal processing mode is the spatial difference mode, the processing module is configured to read the electrical signal at the fourth sampling moment from the storage module corresponding to the first photosensitive module, read the electrical signal at the fourth sampling moment from the storage module of the second photosensitive module adjacent to the first photosensitive module, and perform spatial difference based on the electrical signal at the fourth sampling moment of the first photosensitive module and the electrical signal at the fourth sampling moment of the second photosensitive module to obtain a spatial difference result corresponding to the first photosensitive module. In other words, in the spatial difference mode, a difference operation can be performed on the electrical signals corresponding to the same sampling moment of different photosensitive modules in the spatial dimension to obtain a corresponding spatial difference result.

[0143] In some optional implementations, the spatial differential mode includes a polarity spatial differential mode and / or a multi-value spatial differential mode. Among them, the polarity spatial differential mode means that the spatial differential result is a result with positive and negative signs, and does not represent a specific numerical value. The multi-value spatial differential mode refers to pre-setting multiple value intervals and assigning a unified value to each value interval. As long as the signal difference of the electrical signals of different photosensitive modules at the same sampling moment falls into a certain value interval, the spatial differential result is determined according to the unified value corresponding to the value interval. The smaller the value interval, the higher the accuracy of the multi-value spatial differential result. The value interval can be determined according to actual needs, and the embodiments of the present disclosure do not impose any restrictions on this.

[0144] In some optional implementations, the spatial difference mode includes at least one of a horizontal, vertical, and oblique direction. In other words, any photosensitive module can perform spatial difference calculations with photosensitive modules located above or below it, with photosensitive modules located to its left or right, or with photosensitive modules located diagonally opposite to it (e.g., the upper left corner, lower left corner, upper right corner, and lower right corner).

[0145] Exemplarily, when the signal processing mode is the high-order difference mode, the processing module is configured to read the electrical signal at the fifth sampling moment from the storage module corresponding to the third photosensitive module, read the electrical signal at the sixth sampling moment from the storage module of the fourth photosensitive module adjacent to the third photosensitive module, and perform a time difference based on the electrical signal at the fifth sampling moment of the third photosensitive module and the electrical signal at the sixth sampling moment of the fourth photosensitive module to obtain a high-order difference result corresponding to the third photosensitive module. It can be seen that the high-order difference mode essentially combines time difference and spatial difference, thereby enabling time difference to be performed for photosensitive modules at different locations.

[0146] It should be noted that before the processing module performs signal processing based on the stored electrical signal, it is usually necessary to read the required electrical signal from the storage module. For storage units prepared using devices such as capacitors, if a new electrical signal is to be stored in a certain capacitor, charging and discharging behavior may occur, thereby overwriting the electrical signal originally stored in the capacitor. Based on this, if the electrical signal Si at the sampling time ti is stored in the capacitor Cj, and the electrical signal Si is used for signal processing, it is necessary to read the electrical signal Si from it before charging and discharging the capacitor Cj.

[0147] For example, after the switch of the capacitor Cj is turned on, before the capacitor Cj is charged or discharged based on the electrical signal Si+e at the sampling time ti+e, the electrical signal Si stored in the capacitor Cj is read out first, where e≥1.

[0148] Figure 4 is a schematic diagram of a processing process of a signal processing device provided by an embodiment of the present disclosure. Referring to Figure 4 , if the first storage unit shown in Table 5 corresponds to capacitor C1 , Figure 4 schematically illustrates the process of writing and reading an electrical signal from capacitor C1 .

[0149] As shown in FIG4 , since the sensing signal HS1 at sampling time t1 is to be written into C1, the switch of C1 should be set to the on state (“on” represents the on state and “off” represents the off state in the figure) before t1. Based on this, after HS1 is generated, it can be written into C1.

[0150] Furthermore, since the sensing signal LS2 at sampling time t2 is to be written into C1, the switch of C1 should be set to the enabled state after t1 and before t2. Furthermore, HS1 should be read from C1 before writing LS2. For example, the reading operation of HS1 can be performed between t1 and t2.

[0151] After LS2 is written to C1, there's no need to write the sensing signal at time t3 to C1, so C1 can be turned off. However, since the sensing signal LS4 at sampling time t4 needs to be written to C1, the switch on C1 should be turned on after t3 and before t4. Furthermore, LS2 should be read from C1 before writing LS4. For example, the read operation on LS2 can be performed between t3 and t4.

[0152] Similarly, after writing LS4 to C1, since the sensing signal at time t5 does not need to be written to C1, C1 can be turned off. However, since the sensing signal LS6 at sampling time t6 needs to be written to C1, the switch of C1 should be turned on after t5 and before t6. Furthermore, LS4 should be read from C1 before writing LS6. For example, the reading operation of LS4 can be performed between t5 and t6.

[0153] Furthermore, after LS6 is written into C1, since there is no need to write the sensing signal at time t7 into C1, C1 can be turned off to retain the stored LS6. Similarly, the writing and reading of electrical signals can be performed sequentially, providing a data basis for subsequent processing modules to perform signal processing based on the signal processing mode.

[0154] In some optional implementations, the difference direction is horizontal and the spatial difference step size is 1. The spatial difference processing can be characterized as follows: It indicates that a differential operation is performed on the electrical signals of the two photosensitive modules at adjacent positions on the left and right corresponding to the first row.

[0155] In some optional implementations, the difference direction is horizontal and the spatial difference step size is 2. The spatial difference processing can be characterized as follows: This means that among the three photosensitive modules corresponding to the first row, a differential operation is performed on the electrical signals of the leftmost photosensitive module and the rightmost photosensitive module. Similarly, when the differential direction is horizontal and the spatial differential step size is greater than 2, a similar method can be used to represent the spatial differential processing method.

[0156] In some optional implementations, the difference direction is the vertical direction, and the spatial difference step size is 1. The spatial difference processing can be characterized as follows: It indicates that a differential operation is performed on the electrical signals of the two photosensitive modules at adjacent upper and lower positions corresponding to the first column.

[0157] In some optional implementations, the difference direction is the vertical direction, and the spatial difference step size is 2. The spatial difference processing can be characterized as follows: Indicates that among the three photosensitive modules corresponding to the first column, the electrical signals of the top photosensitive module and the bottom photosensitive module are differentially calculated. Similarly, when the differential direction is vertical and the spatial differential step size is greater than 2, a similar method can be used to represent the spatial differential processing method.

[0158] In some optional implementations, the difference direction is 45 degrees and the spatial difference step size is 1. The spatial difference processing can be characterized as follows: It indicates that a differential operation is performed on the electrical signals of the photosensitive module at the upper left corner of the first row and the photosensitive module at the lower right corner of the second row.

[0159] In some optional implementations, the difference direction is 45 degrees and the spatial difference step size is 2. The spatial difference processing can be characterized as follows: This means that the difference operation is performed on the electrical signals of the photosensitive module in the upper left corner of the first row and the photosensitive module in the lower right corner of the third row. Similarly, when the difference direction is 45 degrees and the spatial difference step size is greater than 2, a similar method can be used to represent the spatial difference processing method.

[0160] In some optional implementations, the difference direction is 135 degrees and the spatial difference step size is 1. The spatial difference processing can be characterized as follows: It indicates that a differential operation is performed on the electrical signals of the photosensitive module at the upper right corner of the first row and the photosensitive module at the lower left corner of the second row.

[0161] In some optional implementations, the difference direction is 135 degrees and the spatial difference step size is 2. The spatial difference processing can be characterized as follows: This means that the difference operation is performed on the electrical signals of the photosensitive module in the upper right corner of the first row and the photosensitive module in the lower left corner of the third row. Similarly, when the difference direction is 135 degrees and the spatial difference step size is greater than 2, a similar method can be used to represent the spatial difference processing method.

[0162] It should be noted that the above spatial difference method is only an example, and the embodiments of the present disclosure do not limit this.

[0163] Figure 5 is a schematic diagram of the processing process of a signal processing device provided by an embodiment of the present disclosure, which includes (a), (b), (c), and (d), and exemplarily illustrates the signal processing process of a 4×4 array of pixels, where each pixel includes a photosensitive module and a storage module. In the pixel array shown in (a), the photosensitive modules of all pixels are grayscale photosensitive modules; in the pixel array shown in (b), the photosensitive modules of all pixels are color photosensitive modules, including red, green, and blue photosensitive modules; and the pixel arrays shown in (c) and (d) include both color photosensitive modules and grayscale photosensitive modules, with the photosensitive modules arranged differently.

[0164] As shown in (a), first, taking the grayscale pixel module 0 as an example, it can be spatially differentiated with the grayscale pixel module 1, and the spatial differentiation step size is 1 at this time, and the differentiation direction is the horizontal direction (or 0 degree direction); the grayscale pixel module 0 can also be spatially differentiated with the grayscale pixel module 2, and the spatial differentiation step size is 2 at this time, and the differentiation direction is the horizontal direction; the grayscale pixel module 0 can also be spatially differentiated with the grayscale pixel module 3, and the spatial differentiation step size is 3 at this time, and the differentiation direction is the horizontal direction.

[0165] Secondly, the grayscale pixel module 0 can be spatially differentiated with the grayscale pixel module 4, and the spatial differentiation step size is 1 at this time, and the differentiation direction is the vertical direction (or 90 degrees); the grayscale pixel module 0 can also be spatially differentiated with the grayscale pixel module 8, and the spatial differentiation step size is 2 at this time, and the differentiation direction is the vertical direction; the grayscale pixel module 0 can also be spatially differentiated with the grayscale pixel module 12, and the spatial differentiation step size is 3 at this time, and the differentiation direction is the vertical direction.

[0166] Again, grayscale pixel module 0 can be spatially differentiated with grayscale pixel module 5, where the spatial differentiation step size is 1 and the differentiation direction is the lower right corner (or 45 degrees). Grayscale pixel module 0 can also be spatially differentiated with grayscale pixel module 10, where the spatial differentiation step size is 2 and the differentiation direction is the lower right corner. Grayscale pixel module 0 can also be spatially differentiated with grayscale pixel module 15, where the spatial differentiation step size is 3 and the differentiation direction is the lower right corner. Other grayscale pixel modules are similar to grayscale pixel module 0 and will not be described in detail here.

[0167] As shown in (b), if (q, v) represents the v-th pixel module in the q-th row, then taking the red pixel module at position (1, 1) as an example, it can be spatially differentiated with the red pixel module at position (1, 3), with a spatial differentiation step of 2 and a differentiation direction in the horizontal direction (or 0 degrees); the red pixel module at position (1, 1) can also be spatially differentiated with the red pixel module at position (3, 1), with a spatial differentiation step of 2 and a differentiation direction in the vertical direction (or 90 degrees); the red pixel module at position (1, 1) can also be spatially differentiated with the red pixel module at position (3, 3), with a spatial differentiation step of 2 and a differentiation direction in the lower right corner direction (or 45 degrees). Other pixel modules are similar and will not be described in detail here.

[0168] In (c), the spatial difference method of the red pixel module, the green pixel module, and the blue pixel module can refer to (b). For the grayscale pixel module in (c), it can be spatially differentiated with the adjacent grayscale pixel modules. For example, taking the grayscale pixel module at position (2, 1) as an example, it can be spatially differentiated with the grayscale pixel module at position (2, 3), and the spatial difference step size is 2 at this time, and the difference direction is the horizontal direction (or 0 degree direction). It can also be spatially differentiated with the grayscale pixel module at position (4, 1), and the spatial difference step size is 2 at this time, and the difference direction is the vertical direction (or 90 degree direction). It can also be spatially differentiated with the grayscale pixel module at position (4, 3), and the spatial difference step size is 2 at this time, and the difference direction is the lower right corner direction (or 45 degree direction).

[0169] In (d), the spatial differentiation method of the red pixel module, green pixel module and blue pixel module can refer to (b) or (c). For the grayscale pixel module in (d), it can be spatially differentiated with the adjacent grayscale pixel modules. Taking the grayscale pixel module 16 as an example, it can be spatially differentiated with the grayscale pixel module 17, and the spatial differential step size is 1 at this time, and the differential direction is the horizontal direction (or 0 degree direction); the grayscale pixel module 16 can also be spatially differentiated with the grayscale pixel module 18, and the spatial differential step size is 1 at this time, and the differential direction is the vertical direction (or 90 degree direction); the grayscale pixel module 16 can also be spatially differentiated with the grayscale pixel module 19, and the spatial differential step size is 1 at this time, and the differential direction is the lower right corner direction (or 45 degree direction); the grayscale pixel module 16 can also be spatially differentiated with the grayscale pixel module 20, and the spatial differential step size is 2 at this time, and the differential direction is the lower right corner direction (or 45 degree direction); the grayscale pixel module 16 can also be spatially differentiated with the grayscale pixel module 23, and the spatial differential step size is 3 at this time, and the differential direction is the lower right corner direction (or 45 degree direction). Of course, the grayscale pixel module 16 can also perform spatial differentiation with the grayscale pixel module 21 and the grayscale pixel module 22, and the embodiment of the present disclosure does not limit this.

[0170] A second aspect of the embodiments of the present disclosure provides a sensor chip.

[0171] FIG6 is a schematic diagram of a sensor chip provided by an embodiment of the present disclosure. Referring to FIG6 , the sensor chip 600 includes at least one signal processing device 610. The signal processing device 610 may be a signal processing device according to an embodiment of the present disclosure.

[0172] In some optional implementations, the sensor chip 600 can be applied to image imaging fields, video shooting fields, etc., and can adjust the signal processing mode according to needs, thereby outputting at least one of color signals, light intensity signals, time difference signals, spatial difference signals and high-order difference signals, which is suitable for a variety of imaging scenarios.

[0173] In the embodiments provided by the present disclosure, the sensor chip includes at least one signal processing device, and the signal processing device includes a photosensitive module, a storage module and a processing module, wherein the photosensitive module is used to generate an electrical signal corresponding to the light signal incident on the photosensitive module at a preset sampling moment; the storage module is used to store the electrical signal of the photosensitive module, wherein the storage module includes at least two storage sub-modules, each storage sub-module is used to store the electrical signal of the photosensitive module at a sampling moment, and the electrical signal stored for each photosensitive module at any sampling moment includes at least one of the following: the electrical signal of the photosensitive module at the sampling moment, the electrical signal of the photosensitive module at the sampling moment and at least one historical sampling moment; the processing module is used to process the electrical signal stored in the storage module according to a signal processing mode to obtain a signal processing result, and the signal processing result includes at least one of a direct input result, a time difference result, a spatial difference result and a high-order difference result. It can be seen that the electrical signal generated by the photosensitive module can be stored in the storage module, and at least two storage submodules of the storage module can respectively store the electrical signal at a sampling moment, thereby providing a data basis for subsequent signal processing; further, the signal processing device supports multiple preset signal processing modes. Therefore, when processing based on the stored electrical signal, a suitable signal processing mode can be selected from them, and then the stored electrical signal is processed based on the signal processing mode to obtain at least one of a direct input result, a time difference result, a spatial difference result, and a high-order difference result. In other words, in the embodiment of the present disclosure, different signal processing modes are configured for the photosensitive module, so that the signal processing device can reuse the photosensitive module to realize multiple signal processing modes, thereby being applicable to various signal processing scenarios and having high flexibility.

[0174] A third aspect of the embodiments of the present disclosure provides a signal processing method.

[0175] Figure 7 is a flow chart of a signal processing method provided by an embodiment of the present disclosure. Referring to Figure 7 , the method may include the following steps.

[0176] Step S701: Determine a signal processing mode according to a received mode setting instruction.

[0177] Step S702: Process the electrical signal of the photosensitive module stored in the storage module according to the signal processing mode to obtain a signal processing result.

[0178] The signal processing device may adopt the signal processing device of any one of the embodiments of the present disclosure.

[0179] In some optional implementations, the mode setting instruction may be an instruction issued according to processing requirements, etc., and is used to indicate a signal processing mode for the electrical signal.

[0180] In some optional implementations, the signal processing mode includes at least one of a direct input mode, a time difference mode, a space difference mode, and a high-order difference mode.

[0181] For example, in order to avoid image blur when shooting scenes of high-speed moving objects, the time difference mode can be selected and the corresponding mode setting instructions can be issued so that when performing signal processing, signals are output only for pixel areas with large light intensity changes, and no signals are output for pixel areas with small light intensity changes.

[0182] For example, in some landscape photography scenes, in order to improve the shooting effect, the direct input mode can be selected and the corresponding mode setting instructions can be issued to accurately convert the light signal into the corresponding color signal during signal processing and output it externally.

[0183] In some optional implementations, the signal processing result includes at least one of a direct input result, a time difference result, a spatial difference result, and a high-order difference result; accordingly, according to the signal processing mode, the electrical signal of the photosensitive module stored in the storage module is processed to obtain the signal processing result, including:

[0184] When the signal processing mode is the direct input mode, the electrical signals of the multiple photosensitive modules at the same sampling moment are read from the storage module, and the direct input results corresponding to the sampling moment are obtained according to the electrical signals of the multiple photosensitive modules at the same sampling moment;

[0185] When the signal processing mode is the time difference mode, the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the photosensitive module at at least one historical sampling moment are read from the storage module, and a time difference is performed based on the electrical signal at the current sampling moment and the electrical signal at at least one historical sampling moment to obtain a time difference result corresponding to the photosensitive module;

[0186] When the signal processing mode is the spatial difference mode, the electrical signal of the photosensitive module and the electrical signal of the adjacent photosensitive module at the same sampling time are read from the storage module, and spatial difference is performed based on the electrical signal of the photosensitive module and the electrical signal of the adjacent photosensitive module to obtain the spatial difference result corresponding to the photosensitive module;

[0187] When the signal processing mode is a high-order differential mode, the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the adjacent photosensitive module at at least one historical sampling moment are read from the storage module, and time differentiation is performed based on the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the adjacent photosensitive module at at least one historical sampling moment to obtain a high-order differential result corresponding to the photosensitive module.

[0188] In the embodiment of the present disclosure, the signal processing mode for the electrical signal can be determined based on the mode setting instruction, and the required electrical signal can be read from the storage module, and then the read electrical signal can be processed according to the determined signal processing mode to obtain the corresponding signal processing result, so that it can be applicable to various signal processing scenarios and has high flexibility.

[0189] It is understood that the above-mentioned embodiments mentioned in this disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step and the setting of the functional module should be determined by its function and possible internal logic.

[0190] In addition, the present disclosure also provides an electronic device and a computer-readable storage medium.

[0191] FIG8 is a block diagram of an electronic device provided by an embodiment of the present disclosure.

[0192] 8 , an embodiment of the present disclosure provides an electronic device, comprising: at least one processor 801; at least one memory 802; and one or more I / O interfaces 803 connected between the processor 801 and the memory 802; wherein the memory 802 stores one or more computer programs that can be executed by the at least one processor 801, and the one or more computer programs are executed by the at least one processor 801 so that the at least one processor 801 can execute the signal processing method of the embodiment of the present disclosure.

[0193] FIG9 is a block diagram of an electronic device provided by an embodiment of the present disclosure.

[0194] 9 , an embodiment of the present disclosure provides an electronic device comprising a plurality of processing cores 901 and an on-chip network 902 , wherein the plurality of processing cores 901 are connected to the on-chip network 902 , and the on-chip network 902 is used to exchange data between the plurality of processing cores and external data.

[0195] One or more instructions are stored in one or more processing cores 901, and the one or more instructions are executed by one or more processing cores 901, so that the one or more processing cores 901 can perform the signal processing method of the embodiment of the present disclosure. In some embodiments, the electronic device can be a brain-like chip.

[0196] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the signal processing method of the present disclosure. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0197] An embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the signal processing method of the embodiment of the present disclosure.

[0198] It will be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium).

[0199] As is well known to those skilled in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those skilled in the art, communication media typically contains computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0200] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0201] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0202] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0203] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0204] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0205] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0206] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0207] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A signal processing device, wherein: include: A photosensitive module, configured to generate an electrical signal corresponding to the light signal incident on the photosensitive module at a preset sampling moment; a storage module, configured to store the electrical signal of the photosensitive module, wherein the storage module includes at least two storage submodules, each of which is configured to store the electrical signal of the photosensitive module at a sampling moment, wherein the electrical signal stored for the photosensitive module at any sampling moment includes at least one of the following: the electrical signal of the photosensitive module at the sampling moment, and the electrical signal of the photosensitive module at the sampling moment and at least one historical sampling moment; A processing module is used to process the electrical signal stored in the storage module according to a preset signal processing mode to obtain a signal processing result, wherein the signal processing result includes at least one of a direct input result, a time difference result, a spatial difference result and a high-order difference result.

2. The device according to claim 1, wherein The processing module is further configured to select a target storage submodule from a storage module corresponding to the photosensitive module when the photosensitive module generates an electrical signal at the current sampling moment, and store the electrical signal at the current sampling moment in the target storage submodule.

3. The device according to claim 1, wherein The storage module includes two storage sub-modules; Among them, at the j-1th sampling moment, the two storage sub-modules respectively store the electrical signal at the j-1th sampling moment and the electrical signal at the j-2th sampling moment; at the j-th sampling moment, the storage sub-module storing the electrical signal at the j-2th sampling moment is used as the target storage sub-module, the electrical signal at the j-1th sampling moment is stored in the target storage sub-module, and the electrical signal at the j-1th sampling moment stored by the other storage sub-module is retained, where j is an integer and j>2.

4. The device according to claim 1, wherein The electrical signal includes a sensing signal, which is a signal generated by the photosensitive module through photoelectric sensing of the light signal, and the sensing signal is a signal with spatial and temporal dimensions; Alternatively, the electrical signal includes the sensing signal and a reset signal corresponding to the sensing signal, and the reset signal is used to reset the sensing signal.

5. The device according to claim 4, wherein The photosensitive module corresponds to one storage module, and the storage module includes two storage submodules, the first storage submodule includes a first storage unit and a second storage unit, and the second storage submodule includes a third storage unit and a fourth storage unit; The processing module is further configured to: for an i-th sampling moment, when the first storage unit and the second storage unit respectively store the sensing signal and the reset signal at the i-1-th sampling moment, and the third storage unit and the fourth storage unit respectively store the sensing signal and the reset signal at the i-2-th sampling moment, turn off switches of the first storage unit and the second storage unit, and turn on switches of the third storage unit and the fourth storage unit, so that the third storage unit stores the sensing signal at the i-th sampling moment, and the fourth storage unit stores the reset signal corresponding to the sensing signal at the i-th sampling moment, where i is an integer and i>2.

6. The device according to claim 5, wherein The conversion gain corresponding to the photosensitive module includes a first gain and a second gain; The sensing signal includes a first gain sensing signal corresponding to the first gain and / or a second gain sensing signal corresponding to the second gain, and the reset signal includes a first gain reset signal corresponding to the first gain and / or a second gain reset signal corresponding to the second gain.

7. The device according to claim 6, wherein The photosensitive module further includes a gain adjustment unit, and the gain adjustment unit is used to adjust the conversion gain according to the ambient brightness of the photosensitive module, so that the photosensitive module generates an electrical signal corresponding to the conversion gain.

8. The device according to claim 1, wherein The signal processing mode includes at least one of a direct input mode, a time difference mode, a spatial difference mode and a high-order difference mode; The processing module is configured to read the electrical signals of the plurality of photosensitive modules at the same sampling moment from the storage module when the signal processing mode is the direct input mode, and obtain a direct input result corresponding to the sampling moment based on the electrical signals of the plurality of photosensitive modules at the same sampling moment; The processing module is configured to, when the signal processing mode is the time difference mode, read the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the photosensitive module at at least one historical sampling moment from the storage module, and perform time difference based on the electrical signal at the current sampling moment and the electrical signal at the at least one historical sampling moment to obtain a time difference result corresponding to the photosensitive module; The processing module is configured to, when the signal processing mode is the spatial difference mode, read the electrical signal of the photosensitive module and the electrical signal of the adjacent photosensitive module at the same sampling moment from the storage module, and perform spatial difference based on the electrical signal of the photosensitive module and the electrical signal of the adjacent photosensitive module to obtain a spatial difference result corresponding to the photosensitive module; The processing module is used to read the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the adjacent photosensitive module at at least one historical sampling moment from the storage module when the signal processing mode is the high-order differential mode, and perform time differentiation based on the electrical signal of the photosensitive module at the current sampling moment and the electrical signal of the adjacent photosensitive module at at least one historical sampling moment to obtain a high-order differential result corresponding to the photosensitive module.

9. The device according to claim 8, wherein There is a corresponding relationship between the photosensitive module and the storage module; The processing module is configured to, when the signal processing mode is the direct input mode, read, for the photosensitive module, an electrical signal at a first sampling moment from a storage module corresponding to the photosensitive module, and obtain a direct input result corresponding to the first sampling moment based on the electrical signals of the plurality of photosensitive modules at the first sampling moment; The processing module is configured to, when the signal processing mode is the time difference mode, read the electrical signal at the second sampling moment and the electrical signal at the third sampling moment from the storage module corresponding to the photosensitive module, and perform time difference based on the electrical signal at the second sampling moment and the electrical signal at the third sampling moment to obtain a time difference result corresponding to the photosensitive module, wherein the second sampling moment and the third sampling moment are different sampling moments; The processing module is configured to, when the signal processing mode is the spatial difference mode, read the electrical signal at a fourth sampling moment from the storage module corresponding to the first photosensitive module, read the electrical signal at the fourth sampling moment from the storage module of the second photosensitive module adjacent to the first photosensitive module, and perform spatial difference based on the electrical signal at the fourth sampling moment of the first photosensitive module and the electrical signal at the fourth sampling moment of the second photosensitive module to obtain a spatial difference result corresponding to the first photosensitive module; The processing module is used to read the electrical signal at the fifth sampling moment from the storage module corresponding to the third photosensitive module when the signal processing mode is the high-order differential mode, read the electrical signal at the sixth sampling moment from the storage module of the fourth photosensitive module adjacent to the third photosensitive module, and perform time differentiation based on the electrical signal at the fifth sampling moment of the third photosensitive module and the electrical signal at the sixth sampling moment of the fourth photosensitive module to obtain a high-order differential result corresponding to the third photosensitive module.

10. The device according to claim 8, wherein The first execution mode of the time difference mode includes at least one of the following: global synchronous execution with a fixed time difference step, global synchronous execution with an adjustable time difference step, and asynchronous execution; The second execution mode of the spatial difference mode includes at least one of the following: global synchronous execution and using a fixed spatial difference step size, global synchronous execution and using an adjustable spatial difference step size, and asynchronous execution.

11. The device according to claim 8, wherein The plurality of photosensitive modules are in at least one of the following situations: the plurality of photosensitive modules are color photosensitive modules; the plurality of photosensitive modules are grayscale photosensitive modules; some of the photosensitive modules are color photosensitive modules, and some of the photosensitive modules are grayscale photosensitive modules; The color photosensitive module is a photosensitive module for collecting color information, and the grayscale photosensitive module is a photosensitive module for collecting brightness information.

12. The device according to claim 1, wherein The signal processing mode includes at least one of a direct input mode, a time difference mode, a spatial difference mode and a high-order difference mode; The processing path of the signal processing device includes at least one of a direct input path corresponding to the direct input mode, a time differential path corresponding to the time differential mode, a spatial differential path corresponding to the spatial differential mode, and a high-order differential path corresponding to the high-order differential mode.

13. A sensor chip, wherein: The sensor chip includes at least one signal processing device; Wherein, the signal processing device adopts the signal processing device according to any one of claims 1-12.

14. A signal processing method, wherein: Applied to a signal processing device, the method includes: determining a signal processing mode according to a received mode setting instruction; Processing the electrical signal of the photosensitive module stored in the storage module according to the signal processing mode to obtain a signal processing result; Wherein, the signal processing device adopts the signal processing device according to any one of claims 1-12.

15. An electronic device, wherein: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the signal processing method according to claim 14.

16. A computer-readable storage medium having a computer program stored thereon, wherein: The computer program implements the signal processing method as claimed in claim 14 when executed by a processor.

17. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein: When the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device implements the signal processing method according to claim 14 .

Images