Method, device and apparatus for encoding space-time signal

By providing different transformation parameters and intensity thresholds for different encoding positions, the problem of pulse sequence image sensor in the dynamic range of light intensity is solved, efficient capture of extremely strong light and weak light is achieved, and the light intensity expression is enhanced. Linearity.

CN117793543BActive Publication Date: 2024-11-15SPIKE VISION (BEIJING) TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311747343.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-18
Publication Date
2024-11-15
Estimated Expiration
2043-12-18

AI Technical Summary

Technical Problem

During the spatiotemporal signal encoding process of existing pulse sequence image sensors, due to the conflict between the speed requirements of the filter and the output circuit, it is difficult to effectively expand the dynamic range of light intensity, especially when the transformation time interval is large, it is difficult to express the extremely strong light intensity. Light intensity.

Method used

By providing different transformation parameters for different encoding locations, such as different transformation time intervals and intensity thresholds, denser pulse data for the region of interest and sparse pulse data for non-interest areas, and fast in low light by controlling the intensity threshold. Read out, read out more and more intense light, expanding the dynamic range of light intensity.

Benefits of technology

Taking into account the filter and output circuit speed, the expressive dynamic range of light intensity is effectively expanded, the capture ability of extremely strong light and weak light is improved, and the linearity of light intensity expression is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117793543B_ABST
    Figure CN117793543B_ABST
Patent Text Reader

Abstract

The disclosed embodiment discloses a method, device and equipment for encoding space-time signals, wherein the method includes: collecting space-time signals corresponding to each local spatial position in the area to be monitored, and the space-time signals corresponding to each local spatial position are collected by a pixel unit with an encoding position; converting each space-time signal into a signal intensity value corresponding to each encoding position; accumulating the signal intensity values ​​corresponding to each encoding position to obtain the intensity accumulation value corresponding to each encoding position; transforming the intensity accumulation value corresponding to each encoding position according to the transformation parameters corresponding to each encoding position to obtain the pulse data corresponding to each encoding position; based on the pulse data corresponding to each encoding position in the current cycle, determining the pulse sequence array corresponding to the area to be monitored as the encoding result of the space-time signal of the area to be monitored. The disclosed embodiment can effectively expand the dynamic range of the expressible light intensity.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to pulse vision technology, and in particular to a method, device and apparatus for encoding spatiotemporal signals. Background Art

[0002] The pulse train image sensor is a new type of neuromorphic vision sensor. It imitates the imaging method of the retina in primates and records the continuous light intensity information in the scene by issuing high-density single-bit pulse sequences, which is called the encoding of space-time signals. It can capture and record high-speed motion and reconstruct the texture details in the scene. Therefore, it has great application value in the fields of machine vision and dynamic scene capture.

[0003] In the related art, when encoding the spatiotemporal signal, the acquisition, accumulation and filtering of the spatiotemporal signal are usually realized based on the collector, accumulator and filter to obtain pulse data. All filters use the same and fixed transformation time interval and intensity threshold to filter the accumulated value of the accumulator to obtain pulse data that characterizes whether this transformation generates a pulse signal. The same and fixed transformation interval and intensity threshold result in a conflict between the light intensity range that can be expressed and the speed of the filter and the output circuit. That is, if the transformation time interval is small, although the light intensity range that can be expressed is larger, the speed requirements of the filter and the output circuit are higher. If the transformation time interval is large, it is difficult to express the light intensity under extremely strong light intensity. Summary of the invention

[0004] The embodiments of the present disclosure provide a method, apparatus and device for encoding a spatiotemporal signal, so as to expand the dynamic range of expressible light intensity while taking into account the speed of the filter and the output circuit.

[0005] One aspect of an embodiment of the present disclosure provides a method for encoding a spatiotemporal signal, comprising: collecting a spatiotemporal signal corresponding to each local spatial position in a to-be-monitored area, wherein the spatiotemporal signal corresponding to each local spatial position is collected by a pixel unit having an encoding position;

[0006] Converting each of the spatiotemporal signals into a signal strength value corresponding to each of the encoding positions;

[0007] Accumulating the signal strength values ​​corresponding to the encoding positions to obtain the intensity accumulation values ​​corresponding to the encoding positions;

[0008] According to the transformation parameters corresponding to the encoding positions, the intensity accumulation values ​​corresponding to the encoding positions are transformed to obtain the pulse data corresponding to the encoding positions;

[0009] Based on the pulse data corresponding to each of the encoding positions in the current cycle, a pulse sequence array corresponding to the area to be monitored is determined as the encoding result of the spatiotemporal signal of the area to be monitored.

[0010] Another aspect of the embodiments of the present disclosure provides a device for encoding a spatiotemporal signal, including:

[0011] A signal collector is used to collect the spatiotemporal signals corresponding to each local spatial position in the monitored area, wherein the spatiotemporal signals corresponding to each local spatial position are collected by a pixel unit having a coding position; and convert each of the spatiotemporal signals into a signal strength value corresponding to each of the coding positions;

[0012] A signal accumulator, used to accumulate the signal strength values ​​corresponding to each of the encoding positions to obtain the intensity accumulation values ​​corresponding to each of the encoding positions;

[0013] A filter, used for transforming the intensity accumulation values ​​corresponding to the encoding positions according to the transformation parameters corresponding to the encoding positions, to obtain the pulse data corresponding to the encoding positions;

[0014] The signal decoder is used to determine the pulse sequence array corresponding to the area to be monitored based on the pulse data corresponding to each of the encoding positions in the current cycle, as the encoding result of the spatiotemporal signal of the area to be monitored.

[0015] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a device for encoding a space-time signal provided in any of the above embodiments, and controlling the device for encoding a space-time signal by executing instructions to implement a method for encoding a space-time signal provided in any of the above embodiments.

[0016] Another aspect of the embodiments of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, and when the instructions are executed, the computer executes the space-time signal encoding method provided in any of the above embodiments.

[0017] The encoding method, device and equipment of the spatiotemporal signal provided by the embodiment of the present disclosure can collect the spatiotemporal signals corresponding to each local spatial position in the monitored area, convert each spatiotemporal signal into a signal intensity value corresponding to each encoding position; accumulate the signal intensity values ​​corresponding to each encoding position to obtain the intensity accumulation value corresponding to each encoding position; transform the intensity accumulation value corresponding to each encoding position according to the transformation parameters corresponding to each encoding position to obtain the pulse data corresponding to each encoding position; based on the pulse data corresponding to each encoding position in the current cycle, determine the pulse sequence array corresponding to the monitored area as the encoding result of the spatiotemporal signal of the monitored area. The embodiment of the present disclosure facilitates providing different transformation parameters, such as different transformation time intervals and / or different intensity thresholds, for pixel units at different encoding positions by corresponding transformation parameters to each encoding position, thereby facilitating the output of denser pulse data for the area of ​​interest and sparser pulse data for the area of ​​non-interest, so as to effectively expand the dynamic range of the expressible light intensity while taking into account the filter and output circuit speed. It is also possible to control different intensity thresholds to facilitate rapid readout of weak light and slow readout of strong light, thereby further expanding the dynamic range of light intensity.

[0018] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0021] Figure 1 is a flowchart of a method for encoding a spatiotemporal signal provided by an exemplary embodiment of the present disclosure;

[0022] Figure 2 is a flowchart of a method for encoding a spatiotemporal signal provided by another exemplary embodiment of the present disclosure;

[0023] Figure 3 is a flowchart of a method for encoding a spatiotemporal signal provided by yet another exemplary embodiment of the present disclosure;

[0024] Figure 4 is a flowchart of a method for encoding a spatiotemporal signal provided by another exemplary embodiment of the present disclosure;

[0025] Figure 5 is a flowchart of a method for encoding a spatiotemporal signal provided by yet another exemplary embodiment of the present disclosure;

[0026] Figure 6 is a flowchart of a method for encoding a spatiotemporal signal provided by another exemplary embodiment of the present disclosure;

[0027] Figure 7 is a schematic diagram of a dynamic range of light intensity provided by an exemplary embodiment of the present disclosure;

[0028] Figure 8 is a schematic structural diagram of a device for encoding a space-time signal provided by an exemplary embodiment of the present disclosure;

[0029] Fig. 9 is a schematic structural diagram of a device for encoding a space-time signal provided by another exemplary embodiment of the present disclosure;

[0030] Fig.10 is a structural diagram of a device for encoding a space-time signal provided by another exemplary embodiment of the present disclosure;

[0031] Fig.11 is a structural schematic diagram of a space-time signal encoding device provided by another exemplary embodiment of the present disclosure;

[0032] Fig.12 is a schematic structural diagram of an application embodiment of the electronic device disclosed herein;

[0033] Fig.13 It is a structural schematic diagram of a pulse camera provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0035] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0036] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0037] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0038] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0039] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0040] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0041] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0042] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0043] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0044] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, and autonomous driving systems, which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0045] Electronic devices such as terminal devices, computer systems, servers, and autonomous driving systems can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, target programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0046] SUMMARY OF THE DISCLOSURE

[0047] In the process of realizing the present disclosure, the inventors found that the pulse sequence image sensor is a new type of neuromorphic visual sensor. By imitating the imaging method of the retina in primates, the continuous light intensity information in the scene is recorded by issuing a high-density single-bit pulse sequence, which is called the encoding of spatiotemporal signals. It can capture and record high-speed motion and reconstruct the texture details in the scene. Therefore, it has great application value in the fields of machine vision and dynamic scene capture. In the related art, when encoding spatiotemporal signals, the acquisition, accumulation and filtering of spatiotemporal signals are usually realized based on a collector, an accumulator and a filter to obtain pulse data. All filters use the same and fixed transformation time interval and intensity threshold to filter the accumulated value of the accumulator to obtain pulse data that characterizes whether this transformation generates a pulse signal. The same and fixed transformation interval and intensity threshold result in a conflict between the light intensity range that can be expressed and the speed of the filter and the output circuit. That is, if the transformation time interval is small, although the light intensity range that can be expressed is large, the speed requirements of the filter and the output circuit are high. If the transformation time interval is large, it is difficult to express the light intensity under extremely strong light intensity.

[0048] Figure 1 FIG. 1 is a flow chart of a method for encoding a spatiotemporal signal provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the encoding method of the space-time signal of the embodiment of the present disclosure includes:

[0049] Step 210: collecting space-time signals corresponding to each local spatial position in the area to be monitored.

[0050] The spatiotemporal signal corresponding to each local spatial position is collected by a pixel unit having a coding position.

[0051] In some optional embodiments, the area to be monitored may be the entire area monitored by the pulse sequence image sensor (which may be referred to as the sensor), or a partial area. For example, the pulse sequence image sensor includes a 1000*1000 pixel array, and the area to be monitored may be the monitoring area corresponding to the 1000*1000 pixel array, or the monitoring area corresponding to a part of the pixel array, such as the monitoring area corresponding to the middle 100*100 pixel array as the area to be monitored. The specific area to be monitored may be set according to actual needs.

[0052] In some optional embodiments, each local spatial position corresponds to a pixel (or pixel unit) in the sensor, and the position of each pixel unit of the sensor can be encoded to obtain the encoding position corresponding to each pixel unit. For example, if the area to be monitored corresponds to a 1000*1000 pixel array, the encoding position is represented by i, j, where i represents the i-th row in the pixel array, j represents the j-th column in the pixel array, i=0, 1, ..., 999, j=0, 1, ..., 999. If the area to be monitored corresponds to a 100*100 local pixel array in the middle of the 1000*1000 pixel array, the encoding position is represented by i, j, where i represents the i-th row in the 100*100 pixel array, j represents the j-th column in the 100*100 pixel array, i=0, 1, ..., 99, j=0, 1, ..., 99. The specific encoding position representation method can also be any other practicable representation method, not limited to the above-mentioned example method. For example, the encoding position is represented by pij, which is not specifically limited.

[0053] In some optional embodiments, the pixel units corresponding to each encoding position may be controlled to collect the spatiotemporal signals corresponding to each local spatial position.

[0054] In some optional embodiments, the spatiotemporal signal is an optical signal.

[0055] In some optional embodiments, each pixel unit may include a signal collector for collecting a spatiotemporal signal of a local spatial position corresponding to the pixel unit.

[0056] Step 220: convert each spatiotemporal signal into a signal strength value corresponding to each encoding position.

[0057] In some optional embodiments, after the spatiotemporal signals corresponding to each local spatial position are collected, each spatiotemporal signal may be converted into a signal strength value corresponding to each encoding position.

[0058] In some optional embodiments, the signal strength value is an electrical signal value converted from the optical signal. The electrical signal value may be, for example, a voltage value.

[0059] In some optional embodiments, after the signal collector of each pixel unit collects the spatiotemporal signal of the local spatial position corresponding to the pixel unit, the spatiotemporal signal is converted into a corresponding signal intensity value. The signal collector can be, for example, a photosensitive device, such as a photodiode. It can be set according to actual needs. The signal collectors are arranged into a photosensitive array to monitor the area to be monitored.

[0060] Step 230: accumulate the signal strength values ​​corresponding to each coding position to obtain the strength accumulation value corresponding to each coding position.

[0061] In some optional embodiments, the signal collector corresponding to each encoding position may correspond to one or more signal accumulators. Each signal accumulator is used to accumulate the signal strength values ​​of the signal collector corresponding to the signal accumulator to obtain the strength accumulation value of the signal collector.

[0062] In some optional embodiments, a signal accumulator may receive signal strength values ​​input by one or more signal collectors. For example, the signal strength values ​​of multiple signal collectors are accumulated on a signal accumulator according to a preset weight ratio (which may be a uniform weight ratio or a non-uniform weight ratio).

[0063] Step 240 , transform the intensity accumulation values ​​corresponding to the respective coding positions according to the transformation parameters corresponding to the respective coding positions, and obtain the pulse data corresponding to the respective coding positions.

[0064] In some optional embodiments, the transformation parameter corresponding to any coding position may include at least one of a transformation time interval and an intensity threshold. The transformation parameters of different coding positions may be the same or different.

[0065] In some optional embodiments, the transformation time intervals of different coding positions may be the same or different. They may be set specifically according to actual needs. For example, a smaller value may be set for the transformation time interval of the coding position of the region of interest, such as 1 microsecond, 2 microseconds, etc., to perform a more intensive collection of the spatiotemporal signals of the region of interest. A larger value may be set for the transformation time interval of the coding position of the non-interested region, such as 1 millisecond, 10 milliseconds, etc., to perform a more sparse collection of the spatiotemporal signals of the non-interested region.

[0066] In some optional embodiments, the intensity thresholds of different coding positions may be the same or different. For example, according to the different strengths of the light signals collected by the signal collector, the intensity threshold corresponding to the coding position with stronger light intensity may be set to a larger threshold, and the intensity threshold corresponding to the coding position with weaker light intensity may be set to a smaller threshold, so that weak light can be read out as quickly as possible, thereby effectively improving the expressible dynamic range of light intensity.

[0067] In some optional embodiments, the transformation time interval and intensity threshold corresponding to each coding position can be determined according to a preset coding rule, or can be analyzed in real time based on the historical pulse sequence array obtained before the current cycle, and the transformation time interval and intensity threshold corresponding to each coding position of the current cycle can be determined based on the analysis results.

[0068] In some optional embodiments, within a cycle, the transformation parameters corresponding to any coding position may change over time. For example, multiple groups of transformation parameter values ​​may be set, each group of transformation parameter values ​​may include at least one of a transformation time interval value and an intensity threshold value, and the multiple groups of transformation parameter values ​​may be used in turn (or alternately) within any cycle. For example, taking the transformation time interval as an example, multiple time interval values ​​may be set, and the multiple time interval values ​​may be used in turn within the cycle. For example, for a certain coding position, the change rule of the transformation time interval corresponding to the coding position is: 1 microsecond, 1 millisecond, 10 milliseconds, 1 microsecond, 1 millisecond, 10 milliseconds, .... Then for the coding position, the intensity accumulation value of the coding position is transformed according to the transformation time interval of the change rule. For example, the first transformation is performed at an interval of 1 microsecond, the second transformation is performed at an interval of 1 millisecond, the third transformation is performed at an interval of 10 milliseconds, the fourth transformation is performed at an interval of 1 microsecond, and so on, and each transformation obtains a pulse data corresponding to the coding position. The pulse data characterizes whether the transformation generates a pulse signal. For example, if a pulse signal is generated, the pulse data is 1, and if no pulse signal is generated, the pulse data is 0. In practical applications, the changing rule of the transformation time interval is not limited to the above rule, and the time interval value of each transformation is not limited to the above time interval value, and can be set according to actual needs. Within a cycle, the intensity threshold corresponding to each encoding position can also be set to one or more thresholds. When multiple thresholds are set, they can change according to a certain time rule.

[0069] In some optional embodiments, the transformation of the intensity accumulation values ​​corresponding to each coding position can be implemented based on a certain filter function. The transformation of the intensity accumulation value corresponding to any coding position can obtain the transformation coefficient corresponding to the coding position, and the pulse data corresponding to the coding position is obtained based on the comparison between the transformation coefficient and the intensity threshold. For example, if the transformation coefficient is greater than or equal to the intensity threshold, a pulse signal is output, that is, the pulse data is data indicating the generation of a pulse signal (for example, represented by a binary 1). If the transformation coefficient is less than the intensity threshold, no pulse signal is output, that is, the pulse data is data indicating that no pulse signal is generated (for example, represented by a binary 0).

[0070] In some optional embodiments, the intensity accumulation values ​​corresponding to each encoding position may be transformed by a filter.

[0071] In some optional embodiments, a signal accumulator can be connected to one or more filters, and the intensity accumulation value of the signal accumulator is used as the input of the filter. A filter can also be connected to one or more signal accumulators to transform any input intensity accumulation value to obtain corresponding pulse data.

[0072] In some optional embodiments, in the case where a filter is connected to multiple signal accumulators, for the intensity accumulation value input by any signal accumulator, the filter transforms the intensity accumulation value according to the transformation time interval and intensity threshold of the coding position corresponding to the signal accumulator, and obtains the pulse data corresponding to the intensity accumulation value. Among them, the correspondence between the input intensity accumulation value and the transformation time interval and the intensity threshold can be controlled by a coding rule. For example, within one cycle, the filter outputs a mixed pulse data sequence of each signal accumulator, in which the pulse data of each signal accumulator is arranged according to a certain coding rule. Subsequently, according to the decoding rule corresponding to the coding rule, the pulse data sequence corresponding to each coding position can be decoded from the pulse data sequence output by the filter. Exemplarily, the pulse data sequence output by the filter is 001011000001. The filter is connected to three signal accumulators. According to the transformation time interval of each signal accumulator, the encoding rule of the filter can be determined, so that the correspondence between each bit in the pulse data sequence and each signal accumulator can be determined. Based on the correspondence between each signal accumulator and the encoding position, the pulse data corresponding to each encoding position in the pulse data sequence output by the filter can be determined, so as to obtain the pulse data sequences corresponding to each encoding position.

[0073] In some optional embodiments, for any signal accumulator, if the pulse data corresponding to the intensity accumulation value transformation result of the signal accumulator is a preset value (or preset state), for example, the pulse data is 1, the intensity accumulation value of the signal accumulator is reset to enter the next accumulation stage and re-accumulate.

[0074] Step 250, based on the pulse data corresponding to each encoding position in the current cycle, determine the pulse sequence array corresponding to the area to be monitored as the encoding result of the spatiotemporal signal of the area to be monitored.

[0075] In some optional embodiments, the current cycle refers to the current data reading cycle, that is, the pulse data is output according to a certain cycle for image reconstruction of the area to be monitored and downstream tasks.

[0076] In some optional embodiments, the pulse data corresponding to each coding position in the current period may include a pulse data sequence output by each filter in the current period. The pulse sequence corresponding to each coding position is obtained by decoding the pulse data sequence, and then the pulse sequence corresponding to each coding position is arranged according to the spatial position relationship of each coding position to obtain a pulse sequence array in the current period as the encoding result of the space-time signal of the monitored area.

[0077] In some optional embodiments, for the case where each coding position corresponds to a signal collector, each signal collector is connected to a unique corresponding signal accumulator, and each signal accumulator is connected to a unique corresponding filter, the pulse data sequence output by each filter in the current cycle includes the pulse data sequence of the coding position corresponding to the filter in the current cycle (which can be called the pulse sequence corresponding to the coding position), and the pulse data sequence can include the pulse data obtained by each transformation of the intensity accumulation value of the coding position in the current cycle. The pulse sequences corresponding to each coding position are arranged according to the spatial position relationship of each coding position to obtain the pulse sequence array in the current cycle as the coding result of the spatiotemporal signal of the area to be monitored.

[0078] The encoding method of the spatiotemporal signal provided by the embodiment of the present disclosure can collect the spatiotemporal signals corresponding to each local spatial position in the monitored area, convert each spatiotemporal signal into a signal intensity value corresponding to each encoding position; accumulate the signal intensity values ​​corresponding to each encoding position to obtain the intensity accumulation values ​​corresponding to each encoding position; transform the intensity accumulation values ​​corresponding to each encoding position according to the transformation parameters corresponding to each encoding position to obtain the pulse data corresponding to each encoding position; based on the pulse data corresponding to each encoding position in the current cycle, determine the pulse sequence array corresponding to the monitored area as the encoding result of the spatiotemporal signal of the monitored area. The embodiment of the present disclosure facilitates providing different transformation parameters, such as different transformation time intervals and / or different intensity thresholds, for pixel units at different encoding positions by corresponding transformation parameters to each encoding position, thereby facilitating the output of denser pulse data for the area of ​​interest and sparser pulse data for the area of ​​non-interest, so as to effectively expand the dynamic range of the expressible light intensity while taking into account the filter and output circuit speed. It is also possible to control different intensity thresholds to facilitate rapid readout of weak light and slow readout of strong light, thereby further expanding the dynamic range of light intensity.

[0079] Figure 2 It is a flowchart of a method for encoding a spatiotemporal signal provided by another exemplary embodiment of the present disclosure.

[0080] In some optional embodiments, step 250 determines the pulse sequence array corresponding to the area to be monitored based on the pulse data corresponding to each encoding position in the current cycle, including:

[0081] Step 2510, arranging the pulse data corresponding to each coding position in the current cycle into a sequence according to time order, and obtaining the pulse sequence corresponding to each coding position.

[0082] In some optional embodiments, for any coding position, each pulse data corresponding to the coding position in the current cycle has a certain time sequence, such as the time sequence of the filter output. Then, the pulse data can be arranged in sequence according to the time sequence to obtain the pulse sequence corresponding to each coding position. For example, the pulse data output by the filter is stored in a register or other memory. For each coding position, the pulse data can be arranged in sequence according to the storage order to obtain the pulse sequence corresponding to the coding position.

[0083] Step 2520: Arrange the pulse sequences corresponding to the respective coding positions according to the spatial position relationship of the respective coding positions to obtain a pulse sequence array corresponding to the area to be monitored.

[0084] In some optional embodiments, the spatial position relationship of each coding position can be determined according to the specific coding position. By arranging the pulse sequences corresponding to each coding position into a pulse sequence array, a binary expression of the light intensity of the area to be monitored can be obtained. Since the transformation time intervals of different coding positions may be different, the pulse sequence lengths of different coding positions in the obtained pulse sequence array may be different.

[0085] Figure 3 It is a flowchart of a method for encoding a spatiotemporal signal provided by yet another exemplary embodiment of the present disclosure.

[0086] In some optional embodiments, step 240 transforms the intensity accumulation values ​​corresponding to the respective encoding positions according to the transformation parameters corresponding to the respective encoding positions to obtain the pulse data corresponding to the respective encoding positions, including:

[0087] Step 2410, determining the transformation time interval and intensity threshold corresponding to each encoding position.

[0088] Among them, the transformation time interval and intensity threshold corresponding to each coding position can be obtained based on a certain determination method. The determination method can be any implementable method, for example, it can be a preset determination method (or preset determination rule), or it can be a real-time determination method. The preset determination method can include at least one transformation time interval and a change rule of the transformation time interval corresponding to each coding position, at least one intensity threshold and a change rule of the intensity threshold. For example, for each coding position, multiple transformation time intervals and multiple intensity thresholds can be pre-set, in each cycle, multiple transformation time intervals change according to the change rule of the transformation time interval, and multiple intensity thresholds change according to the change rule of the intensity threshold, so that at each transformation moment of each cycle, the transformation time interval and intensity threshold corresponding to each coding position at the transformation moment can be determined according to the preset determination method. For another example, signal analysis can also be performed based on the pulse sequence array obtained in the previous cycle (which can be called the historical pulse sequence array), and the transformation time interval and intensity threshold of the previous cycle are adjusted according to the analysis results to obtain the transformation time interval and intensity threshold of the current cycle, and then the transformation time interval and intensity threshold corresponding to each coding position are determined according to the transformation time interval and intensity threshold of the current cycle determined in real time. In practical applications, the transformation time interval and intensity threshold corresponding to each encoding position may also be determined by other methods, which are not limited in this embodiment.

[0089] Step 2420, transform the intensity accumulation value corresponding to each coding position according to the transformation time interval and intensity threshold corresponding to each coding position, and obtain the pulse data corresponding to each coding position.

[0090] In some optional embodiments, the transformation time interval refers to the time interval between two adjacent transformations. For example, after a transformation is performed, the timing starts from the transformation time t1 of the transformation, and when the timing time reaches the current transformation time interval (for example, △t=10 microseconds), the transformation is performed again. Alternatively, the transformation time t2=t1+△t of the next transformation can be determined based on the transformation time t1 and the transformation time interval, and the timer can continuously count, and when the timing reaches the transformation time t2, another transformation is triggered. There is no limitation on the specific triggering and control method of the transformation.

[0091] In some optional embodiments, the intensity threshold is used to compare with the transformation coefficient obtained by transformation during the transformation process. For details, please refer to the above embodiments and will not be described in detail here.

[0092] This embodiment can determine the transformation time interval and intensity threshold corresponding to each coding position in a certain manner during the process of encoding the spatiotemporal signal, so as to provide different transformation time intervals and intensity thresholds for each coding position in at least one dimension of space (i.e., at least one of the transformation time intervals and intensity thresholds of different coding positions may be different) and time (i.e., at least one of the transformation time intervals and intensity thresholds of the same coding position at different times may be different), thereby helping to expand the dynamic range of light intensity that can be expressed by the encoding result.

[0093] Figure 4 It is a flowchart of a method for encoding a spatiotemporal signal provided by yet another exemplary embodiment of the present disclosure.

[0094] In some optional embodiments, determining the transformation time interval and intensity threshold corresponding to each encoding position in step 2410 includes:

[0095] Step 24110, determine the transformation time interval and intensity threshold corresponding to each encoding position according to the preset determination rule.

[0096] In some optional embodiments, the preset determination rule can be set according to actual needs. For example, for the transformation time interval, the preset determination rule may include at least one transformation time interval and a change rule of the transformation time interval corresponding to each coding position, at least one intensity threshold and a change rule of the intensity threshold. For details, please refer to the above embodiment, which will not be described here.

[0097] This embodiment determines the change time interval and intensity threshold corresponding to each coding position according to a preset determination rule, which helps the user to set the corresponding change time interval and intensity threshold according to the monitoring requirements of the monitored area, meets the needs of different users, and improves the user experience.

[0098] Figure 5 It is a flowchart of a method for encoding a spatiotemporal signal provided by yet another exemplary embodiment of the present disclosure.

[0099] In some optional embodiments, determining the transformation time interval and intensity threshold corresponding to each encoding position in step 2410 includes:

[0100] Step 24120, obtaining a historical pulse sequence array of at least one historical period.

[0101] Among them, at least one historical cycle may include one or more historical cycles before the current cycle. For example, it may include the previous historical cycle adjacent to the current cycle. It may also include an earlier historical cycle. There is no specific limitation. The acquisition process of the historical pulse sequence array of each historical cycle is consistent with the pulse sequence array of the current cycle, which will not be repeated here.

[0102] Step 24130, based on the historical pulse sequence array, determine the transformation time interval and intensity threshold corresponding to each encoding position.

[0103] In some optional embodiments, signal analysis may be performed on the historical pulse sequence array to determine the density of pulse signals and the intensity of light at each coding position in the historical pulse sequence array. Thus, the historical transformation time interval and historical intensity threshold corresponding to each coding position in the historical period that meets the adjustment conditions may be adjusted according to the density of pulse signals and the intensity of light at each coding position. The adjusted transformation time interval and intensity threshold of these coding positions may be used as the transformation time interval and intensity threshold of the current period.

[0104] This embodiment determines the transformation time interval and intensity threshold of each coding position in the current cycle through the historical pulse sequence array of the historical cycle, realizes the implementation and determination of the transformation time interval and intensity threshold, effectively improves the accuracy and effectiveness of the transformation time interval and intensity threshold, and facilitates the reduction of the transformation time interval in the pulse signal dense area in the historical cycle, realizes high-frequency transformation, and obtains the relationship between the light intensity of the area and time more accurately. The intensity threshold can be increased for the area with strong light intensity, and the intensity threshold can be reduced for the area with weak light intensity, so that the weak light can be read out as soon as possible, further expanding the dynamic range of light intensity.

[0105] In some optional embodiments, determining the transformation time interval and intensity threshold corresponding to each encoding position based on the historical pulse sequence array in step 24130 includes:

[0106] Perform target signal detection on the historical pulse sequence array to obtain the target coding position corresponding to the target signal; based on the target signal, adjust at least one of the historical transformation time interval and the historical intensity threshold corresponding to the target coding position to obtain the adjusted transformation time interval and the adjusted intensity threshold corresponding to the target coding position; determine the transformation time interval and the intensity threshold corresponding to each coding position in the current period based on the adjusted transformation time interval and the adjusted intensity threshold corresponding to the target coding position and the historical transformation time interval and the historical intensity threshold corresponding to other coding positions except the target coding position.

[0107] Among them, the target signal can be set according to actual needs. For example, the target signal can include at least one of a signal with frequent pulse changes, a signal with a high pulse signal issuance frequency, etc. A signal with frequent pulse changes means that the changes of 0 and 1 in the pulse sequence corresponding to the coding position are relatively frequent, such as 010101, and the pulse data of any two adjacent bits are changed. A signal with frequent pulse changes can be obtained by comparing the pulse change situation with the frequent condition, and the specific frequent condition can be set according to actual needs. A signal with a high pulse signal issuance frequency means that the frequency of 1 in the pulse sequence corresponding to the coding position is high, for example, a certain coding position has 1 multiple times in a row within a cycle, such as 011111. A signal with a high pulse signal issuance frequency can be determined based on the comparison between the issuance frequency of the pulse signal in the pulse sequence array and the frequency threshold. The specific frequency threshold can be set according to actual needs, and the present disclosure is not limited thereto.

[0108] In some optional embodiments, for a detected target signal, a coding position corresponding to the target signal (referred to as a target coding position) can be obtained, and then based on the target signal, at least one of the historical transformation time interval and the historical strength threshold corresponding to the target coding position is adjusted to obtain the adjusted transformation time interval and the adjusted strength threshold corresponding to the target coding position, and then based on the adjusted transformation time interval and the adjusted strength threshold corresponding to the target coding position, and the historical transformation time interval and the historical strength threshold corresponding to other coding positions except the target coding position, the transformation time interval and the strength threshold corresponding to each coding position in the current period can be determined.

[0109] Exemplarily, for target signals with frequent pulse changes or target signals with high pulse signal emission frequency, the transformation time interval corresponding to the target coding position can be reduced, and the target coding position can be transformed at high frequency to more accurately obtain the relationship between the change of light intensity at the target coding position and time. For target signals with too low pulse signal emission frequency, the transformation time interval corresponding to the target coding position can be increased to reduce the transformation frequency of the target coding position, which helps to balance the dynamic range of light intensity with the speed of the filter and output circuit. The intensity of light can also be judged according to the density of pulse signals at the target coding position. If the light intensity is weak, the intensity threshold can be lowered so that the weak light can be read out as soon as possible. If the light intensity is strong, the intensity threshold can be increased to improve the dynamic range of light intensity.

[0110] Figure 6 It is a flowchart of a method for encoding a spatiotemporal signal provided by yet another exemplary embodiment of the present disclosure.

[0111] In some optional embodiments, the method of the embodiment of the present disclosure further includes:

[0112] Step 410: For any coding position, in response to the pulse data corresponding to the coding position being in a preset state, the intensity accumulation value corresponding to the coding position is reset to enter the next accumulation stage.

[0113] The preset state is a state indicating that a pulse signal is generated, for example, the pulse data is a binary 1, which indicates that the pulse data is in the preset state. That is, after the pulse signal is generated, the intensity accumulation value is reset and the next accumulation stage is entered for re-accumulation, thereby achieving continuous monitoring of the area to be monitored.

[0114] In some optional embodiments, before collecting the spatiotemporal signals corresponding to each local spatial position in the area to be monitored in step 210, the method of the embodiment of the present disclosure further includes:

[0115] Step 420 , position encoding is performed on the target pixel unit to obtain the encoding position corresponding to the target pixel unit.

[0116] In some optional embodiments, the target pixel unit may include all pixel units of the pixel array of the sensor, or may include some pixel units in the pixel array. For example, the sensor includes a 1000*1000 pixel array, and the target pixel unit may include each pixel unit in the 1000*1000 pixel array, or may include some pixel units in the 1000*1000 pixel array, such as the target pixel unit includes each pixel unit in a 100*100 sub-pixel array in the middle of the 1000*1000 pixel array. The specific target pixel unit may be set according to actual needs.

[0117] In some optional embodiments, collecting the spatiotemporal signals corresponding to each local spatial position in the monitored area in step 210 includes:

[0118] Step 2110, controlling the target pixel unit to collect the spatiotemporal signal corresponding to the local spatial position monitored by the target pixel unit.

[0119] This embodiment can set the target pixel unit according to the monitoring requirements, so as to control all or part of the pixel units in the sensor pixel array to enter the working state, collect the spatiotemporal signals corresponding to the corresponding local spatial positions, and improve the flexibility of sensor application.

[0120] The encoding method of the spatiotemporal signal of the embodiment of the present disclosure can realize dense acquisition of the region of interest in the monitored area by densely encoding the change moments of the region of interest (i.e., the average change time interval is small), and realize sparse acquisition by sparsely encoding the change moments of the non-interested region, thereby realizing dense acquisition of the region of interest and sparse acquisition of the non-interested region under limited transmission bandwidth. Moreover, under a fixed transmission bandwidth, a non-uniform pulse sequence output can be obtained by encoding the change moments. For example, Figure 7 FIG. 1 is a schematic diagram of a dynamic range of light intensity provided by an exemplary embodiment of the present disclosure. Figure 7 As shown, t represents time. For the encoding method (which can be referred to as uniform coding) that uses equal intervals (i.e., the same transformation time interval) for transformation output in the related art, within a period of time Δt, if the number of output pulse signals (pulse data is 1) is 3, then the time intervals of the two output pulse signals are both Δt / 2, so the highest light intensity is 2K / Δt, and the weakest light intensity is K / Δt, where K is a function related to the intensity threshold. If the encoding method (which can be referred to as non-uniform coding) that uses non-equal intervals (i.e., variable transformation time intervals) for transformation output is used in the embodiment of the present disclosure, the adjacent intervals of the output pulse signals are Δt1 and Δt2, Δt1+Δt2=Δt, assuming Δt1=Δt / 100, then the highest light intensity that can be expressed is 100K / Δt, and the weakest light intensity is K / Δt. It can be seen that the method of the embodiment of the present disclosure can effectively expand the dynamic range of the light intensity that can be expressed. And by using different intensity thresholds, the dynamic range of expression can be made more linear. As shown Figure 7 As shown, there are three types of light intensity signals when non-uniform coding is used, namely K1 / Δt1, K2 / Δt2, and K2 / Δt. Therefore, by reasonably setting K1, K2, and Δt1, K1 / Δt1, K2 / Δt2, and K2 / Δt can be in a linear growth relationship, thereby making the light intensity expression more linear.

[0121] The above-mentioned embodiments of the present disclosure may be implemented separately or in any combination without conflict. The specific configuration may be based on actual needs and the present disclosure does not limit this.

[0122] Any of the spatiotemporal signal encoding methods provided in the embodiments of the present disclosure may be implemented based on at least one of hardware and software. For example, it may be implemented based on a combination of hardware and software. For example, a signal collector, a signal accumulator, and a filter may be implemented by a hardware circuit, the encoding of a local spatial position, the control of a signal collector, a signal accumulator, and a filter may be implemented by combining hardware and software, and the decoding of a signal may be implemented by software, etc.

[0123] Any of the space-time signal encoding methods provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any of the space-time signal encoding methods provided in the embodiments of the present disclosure may be executed by a processor, such as a processor executing any of the space-time signal encoding methods mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0124] Figure 8 1 is a schematic diagram of the structure of a space-time signal encoding device provided by an exemplary embodiment of the present disclosure. The space-time signal encoding device of the embodiment of the present disclosure can be used to implement the above-mentioned space-time signal encoding method embodiments of the present disclosure. Figure 8 As shown, the device for encoding a space-time signal in this embodiment includes: a signal collector 520 , a signal accumulator 530 , a filter 540 and a signal decoder 550 .

[0125] The signal collector 520 is used to collect the spatiotemporal signals corresponding to each local spatial position in the monitored area. The spatiotemporal signals corresponding to each local spatial position are collected by pixel units with coding positions; and each spatiotemporal signal is converted into a signal strength value corresponding to each coding position.

[0126] The signal accumulator 530 is used to accumulate the signal strength values ​​corresponding to each encoding position to obtain the strength accumulation value corresponding to each encoding position.

[0127] The filter 540 is used to transform the intensity accumulation value corresponding to each encoding position according to the transformation parameters corresponding to each encoding position, so as to obtain the pulse data corresponding to each encoding position.

[0128] The signal decoder 550 is used to determine the pulse sequence array corresponding to the area to be monitored based on the pulse data corresponding to each encoding position in the current cycle, as the encoding result of the spatiotemporal signal of the area to be monitored.

[0129] In some optional embodiments, the signal collector 520 may be a photosensitive device, such as a photodiode, and is used to convert the spatiotemporal signal (ie, the optical signal) into an electrical signal, which may represent the intensity of the optical signal and thus become a signal intensity value.

[0130] In some optional embodiments, the signal accumulator 530 may be a device with a time integration function, such as a capacitor connected to the signal collector, which can accumulate the electrical signal collected by the signal collector. For example, the photoelectric charge generated by the photodiode is accumulated to make the voltage at one end of the photodiode change continuously.

[0131] In some optional embodiments, the filter 540 may be a binary pulse filter. The signal accumulator 530 and the filter 540 constitute a time-delayed binary pulse filter.

[0132] In some optional embodiments, the signal decoder 550 can be implemented using any device with signal processing function. By configuring decoding rules for the signal decoder, the signal decoder can implement the corresponding signal decoding function based on the decoding rules, decode the pulse data sequence output by the filter 540 and arrange it into a pulse sequence array corresponding to the monitored area.

[0133] The specific work of each module or component of the device of this embodiment can be found in the aforementioned method embodiment and will not be described in detail here.

[0134] In some optional embodiments, the signal decoder 550 is specifically used to: arrange the pulse data corresponding to each coding position in the current cycle into a sequence in time order to obtain a pulse sequence corresponding to each coding position; arrange the pulse sequence corresponding to each coding position according to the spatial position relationship of each coding position to obtain a pulse sequence array corresponding to the monitored area.

[0135] Fig. 9 It is a structural diagram of a space-time signal encoding device provided by another exemplary embodiment of the present disclosure.

[0136] In some optional embodiments, the apparatus of the embodiment of the present disclosure further includes: a processing module 610 and a filtering control module 620 .

[0137] The processing module 610 is used to determine the transformation time interval and intensity threshold corresponding to each encoding position.

[0138] The processing module 610 may determine the transformation time interval and intensity threshold corresponding to each coding position based on a preset determination rule, or may determine the transformation time interval and intensity threshold corresponding to each coding position in real time based on a historical pulse sequence array. The specific setting may be based on actual needs.

[0139] The filter control module 620 is used to control the filter 540 to transform the intensity accumulation value corresponding to each encoding position according to the transformation time interval and intensity threshold corresponding to each encoding position.

[0140] Among them, the filter control module 620 may include a transformation time interval control unit and an intensity threshold control unit, which are respectively used to control the transformation time interval and intensity threshold of the filter 540. For example, the transformation time interval control unit can be implemented based on a timer. The transformation time of each transformation of the filter 540 is controlled by the timer, thereby realizing the control of the transformation time interval. The intensity threshold control unit can be implemented based on a digital-to-analog converter (DAC) to provide the filter 540 with an intensity threshold corresponding to the current transformation. The specific circuit structure of the filter control module 620 can be set according to actual needs, and is not limited to the structure of the above example.

[0141] In some optional embodiments, the processing module 610 is specifically used to: determine the transformation time interval and intensity threshold corresponding to each encoding position according to a preset determination rule.

[0142] In this embodiment, the processing module 610 is a processing module of the preconfigured coding control algorithm logic, which determines the transformation time interval and intensity threshold corresponding to each coding position according to the preconfigured determination rule (which may be called the coding rule).

[0143] In some optional embodiments, the processing module 610 is specifically used to: obtain a historical pulse sequence array of at least one historical period; and determine the transformation time interval and intensity threshold corresponding to each encoding position based on the historical pulse sequence array.

[0144] In this embodiment, the processing module 610 may be a processing module with a signal processing function, which is used to perform signal detection analysis on the historical pulse sequence array, so as to determine the transformation time interval and intensity threshold corresponding to each encoding position in the current cycle according to the analysis result.

[0145] In some optional embodiments, the processing module 610 is specifically used to:

[0146] Perform target signal detection on the historical pulse sequence array to obtain the target coding position corresponding to the target signal; based on the target signal, adjust at least one of the historical transformation time interval and the historical intensity threshold corresponding to the target coding position to obtain the adjusted transformation time interval and the adjusted intensity threshold corresponding to the target coding position; determine the transformation time interval and the intensity threshold corresponding to each coding position in the current period based on the adjusted transformation time interval and the adjusted intensity threshold corresponding to the target coding position and the historical transformation time interval and the historical intensity threshold corresponding to other coding positions except the target coding position.

[0147] In some optional embodiments, the filter 540 is further used to: for any coding position, in response to the pulse data corresponding to the coding position being in a preset state, reset the intensity accumulation value corresponding to the coding position to enter the next accumulation stage.

[0148] The reset of the intensity accumulation value can be achieved by the reset circuit in the filter 540. The reset circuit can be set according to actual needs, and the embodiment of the present disclosure is not limited. For example, the pulse data output by the filter is connected to the reset circuit, and the reset circuit is connected to the signal accumulator 530. When the filter outputs a pulse signal (that is, the pulse data is 1), the pulse signal triggers the reset circuit to reset the accumulated intensity value of the signal accumulator 530.

[0149] In some optional embodiments, the apparatus of the embodiment of the present disclosure further includes: an encoding control module 630, configured to perform position encoding on the target pixel unit to obtain an encoding position corresponding to the target pixel unit.

[0150] The coding control module 630 may be implemented by any implementable control device, and the coding control module 630 may implement corresponding coding functions by configuring corresponding coding control algorithm logic for the coding control module 630.

[0151] In some optional embodiments, the encoding control module 630 may have a spatial position control function in addition to the function of encoding the position of the target pixel unit, for controlling the signal collectors that need to work to work. For example, the signal collectors corresponding to each encoding position may be controlled to work through row and column decoders.

[0152] In some optional embodiments, the encoding device of the space-time signal may also be other implementable structures. Fig.10 FIG. 1 is a schematic diagram of a structure of a coding device for a space-time signal provided by another exemplary embodiment of the present disclosure. Fig.10As shown, the device includes: a control module 730, a signal collector 520, a signal accumulator 530, a filter 540 and a signal decoder 550. The control module 730 includes a coding control unit 7310, a spatial position control unit 7320 and a filtering control unit 7330. The filtering control unit 7330 includes a time interval control subunit 73310 and an intensity threshold control unit 73320. Among them, the algorithm logic includes a coding control algorithm logic, and the algorithm logic is mapped to the coding control unit 7310 to realize the corresponding algorithm logic function. The coding control unit 7310 is used to perform position coding on the target pixel unit to obtain the coding position corresponding to the target pixel unit. The spatial position control unit 7320 is used to control the signal collector 520 corresponding to each coding position to enter work under the control of the coding control unit 7310. The coding control unit 7310 is further used to determine the transformation time interval and intensity threshold corresponding to each coding position according to the algorithm logic, and then control the transformation time interval of the filter 540 through the time interval control subunit 73310, and control the intensity threshold of the filter 540 through the intensity threshold control unit 73320. The specific work of other modules can refer to the above embodiments.

[0153] In some optional embodiments, Fig.11 FIG. 1 is a schematic diagram of a structure of a coding device for a space-time signal provided by another exemplary embodiment of the present disclosure. Fig.11 As shown, the apparatus of the embodiment of the present disclosure further includes a signal processing module 560. The historical pulse sequence array of at least one historical period obtained by the signal decoder 550 can be transmitted to the signal processing module 560, and the signal processing module 560 determines the transformation time interval and intensity threshold corresponding to each encoding position of the current period based on the historical pulse sequence array.

[0154] The beneficial technical effects corresponding to the exemplary embodiment of the present device can be found in the corresponding beneficial technical effects of the above exemplary method section, which will not be repeated here.

[0155] In addition, an embodiment of the present disclosure further provides an electronic device, comprising: the device for encoding a space-time signal provided in any of the above embodiments. The electronic device may also include: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, the output signal of the device for encoding a space-time signal described in any of the above embodiments of the present disclosure is processed to achieve the desired function.

[0156] Fig.12 FIG. 1 is a schematic diagram of a structure of an application embodiment of the electronic device disclosed in the present invention. Fig.12 As shown, the electronic device includes one or more processors 11 and a memory 12, and the encoding device 50 of the space-time signal provided in the above-mentioned embodiment.

[0157] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0158] The memory 12 may store one or more computer program products, and the memory 12 may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor may run the computer program products to implement the desired functions described above.

[0159] In one example, the electronic device may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0160] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0161] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0162] Of course, to simplify, Fig.12 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0163] In another embodiment of the present disclosure, an electronic device is also provided, which includes a pixel circuit of a pulse sequence image sensor, and / or an encoding device including the above-mentioned space-time signal, and / or a chip having the above-mentioned space-time signal encoding device.

[0164] Specifically, the device includes at least one of the following: camera, webcam, audio / video player, navigation device, fixed location terminal, entertainment device, smart phone, communication device, mobile device, transportation vehicle or facility, industrial equipment, medical equipment, security equipment, flight equipment, household appliance.

[0165] Specifically, the device includes at least one of the following: a pulse camera, a high-speed camera, a visual camera, an audio player, a video player, a navigation device, a fixed-position terminal, an entertainment unit, a smart phone, a communication device, a mobile device, a device in a motor vehicle, a vehicle-mounted camera, a mobile phone camera, a sports or wearable camera, a traffic camera, an industrial inspection camera, a camera mounted on a flying object, a medical camera, a security camera, or a home appliance camera.

[0166] In the embodiments of the present disclosure, the camera includes but is not limited to a pulse camera, a high-speed camera, an industrial inspection camera, etc. The camera includes but is not limited to: a car camera, a mobile phone camera, a traffic camera, a camera mounted on a flying object, a medical camera, a security camera, or a home appliance camera.

[0167] Taking a pulse camera as an example, the device provided by the embodiment of the present disclosure is described in detail. Fig.13 is a schematic diagram of the structure of a pulse camera provided by an exemplary embodiment of the present disclosure. Fig.13 As shown, the pulse camera includes: a lens 1201, a pulse signal circuit 1202, a data processing circuit 1203, a non-volatile memory 1204, a power circuit 1205, a volatile memory 1206, a control circuit 1207 and an I / O interface 1208.

[0168] The lens 1201 is used to receive incident light from the object, that is, the optical signal.

[0169] The pulse signal circuit 1202 is used to convert the optical signal received by the lens 1201 into an electrical signal, and generate a pulse signal according to the electrical signal. The pulse signal circuit 1202 includes, for example, the pixel circuit of the pulse sequence image sensor described above, and / or the pulse sequence image sensor described above, and / or a chip having the pixel circuit of the pulse sequence image sensor described above.

[0170] The data processing circuit 1203 is used to control the pulse signal reading process. The data processing circuit 1203 includes, for example: an arithmetic processing unit (for example, a CPU) and / or an image processing unit (GPU). For example, it controls the pulse signal reading process of the pulse signal reading circuit, controls the read row selector therein to send a row read signal, controls the reset row selector to send a column reset signal, etc.

[0171] 1206 is a volatile memory, such as a random access memory (RAM), and 1204 is a non-volatile storage device, such as a solid state disk (SSD), a hybrid hard disk (HHD), a secure digital (SD) card, a mini SD card, etc.

[0172] In one embodiment of the present invention, the pulse camera further comprises: a display unit for real-time / playback display of the pulse signal / image information. The pulse camera described in the embodiment of the present invention may further comprise at least one of the following: a wired / wireless transmission interface, such as a WiFi interface, a Bluetooth interface, a USB interface, an RJ45 interface, a mobile industry processor interface (MIPI) interface, a low voltage differential signal (LVDS) interface, and other interfaces with wired or wireless transmission functions.

[0173] The pulse camera provided in the embodiment of the present invention can be used to detect visible light, infrared light, ultraviolet light, X-rays, etc., and can be applied to various scenarios. Commonly used scenarios include but are not limited to:

[0174] It can be used as an on-board camera installed in various types of transportation vehicles or facilities, such as for vehicle-road collaboration, smart transportation, and information acquisition and control of autonomous driving. For example, it can be installed in rail transportation vehicles such as high-speed rail or on rail transportation lines as a high-speed rail driving recorder; it can also be installed in autonomous vehicles or vehicles equipped with advanced driver assistance systems (ADAS), such as for detection and alarm of vehicle, pedestrian, lane, driver and other information.

[0175] It can be used as a traffic camera installed on traffic signal poles to capture, warn, and coordinate control of vehicles and pedestrians on urban roads and highways.

[0176] It can be used as an industrial inspection camera, for example, installed on high-speed rail transit lines for high-speed rail line inspection and for high-speed rail safety inspection; it can also be used for coal mine conveyor belt break detection, substation arc detection, real-time detection of wind turbine blades, high-speed turbine non-stop detection and other specific industrial scenarios detection, early warning, etc.

[0177] It is installed on flying objects, such as airplanes, satellites, etc., and is used for high-definition imaging of objects in high-speed flight or even high-speed rotation scenarios.

[0178] Industry (machine vision in intelligent manufacturing, etc.), civil (judicial evidence collection, sports refereeing, etc.) and consumer electronics (cameras, film and television media, etc.).

[0179] It can be used as a medical camera to perform high-definition medical imaging in clinical diagnosis and treatment such as medicine, beauty, and health care.

[0180] It can be used as a sports camera or a wearable camera, for example, a head-mounted camera or a camera embedded in a watch, and can be used to shoot various sports competitions, daily leisure sports and other scenes.

[0181] It can also be used as a security camera, mobile phone camera or home appliance camera, etc.

[0182] The flow charts and block diagrams in the accompanying drawings of the present disclosure show the possible architecture, functions and operations of the systems, methods and computer program products according to the various embodiments disclosed in the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the boxes can also occur in the order of the standards in different drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform steps in the desired functions according to various embodiments of the present disclosure described in the above part of this specification.

[0184] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0185] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to perform the steps of the desired functions according to the embodiment of the present disclosure described in the above part of this specification.

[0186] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0187] A person of ordinary skill in the art can understand that all or part of the steps for realizing the above-mentioned desired functions can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps including the above-mentioned desired functions; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0188] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0189] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0190] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0191] The method and apparatus of the present disclosure may be implemented in many ways. For example, the circuits and desired functions of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the desired functions is for illustration only, and the steps of the desired functions of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the desired functions according to the present disclosure. Therefore, the present disclosure also covers recording media storing programs for executing methods according to the present disclosure.

[0192] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0193] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0194] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for encoding a spatiotemporal signal, characterized in that: include: Collecting the spatiotemporal signals corresponding to each local spatial position in the area to be monitored, wherein the spatiotemporal signals corresponding to each local spatial position are collected by a pixel unit having a coding position; Converting each of the spatiotemporal signals into a signal strength value corresponding to each of the encoding positions; Accumulating the signal strength values ​​corresponding to the encoding positions to obtain the intensity accumulation values ​​corresponding to the encoding positions; According to the transformation parameters corresponding to the encoding positions, the intensity accumulation values ​​corresponding to the encoding positions are transformed to obtain the pulse data corresponding to the encoding positions; Based on the pulse data corresponding to each of the encoding positions in the current cycle, a pulse sequence array corresponding to the area to be monitored is determined as the encoding result of the spatiotemporal signal of the area to be monitored; The transforming of the intensity accumulation values ​​corresponding to the respective encoding positions according to the transformation parameters corresponding to the respective encoding positions includes: Performing target signal detection on a historical pulse sequence array of at least one historical period to obtain a target coding position corresponding to the target signal; Based on the target signal, adjusting the historical transformation time interval and the historical intensity threshold corresponding to the target coding position to obtain the adjusted transformation time interval and the adjusted intensity threshold corresponding to the target coding position; Determine the transformation time interval and intensity threshold respectively corresponding to each of the coding positions in the current period based on the adjusted transformation time interval and the adjusted intensity threshold corresponding to the target coding position, and the historical transformation time interval and the historical intensity threshold corresponding to other coding positions except the target coding position; The intensity accumulation values ​​respectively corresponding to the encoding positions are transformed according to the transformation time interval and the intensity threshold respectively corresponding to the encoding positions.

2. The method according to claim 1, characterized in that The step of determining the pulse sequence array corresponding to the area to be monitored based on the pulse data corresponding to each of the encoding positions in the current cycle includes: Arrange the pulse data corresponding to each of the encoding positions in the current cycle into a sequence in time order to obtain a pulse sequence corresponding to each of the encoding positions; The pulse sequences corresponding to the encoding positions are arranged according to the spatial position relationship of the encoding positions to obtain a pulse sequence array corresponding to the area to be monitored.

3. The method according to claim 1 or 2, characterized in that: The transforming of the intensity accumulation values ​​corresponding to the respective encoding positions according to the transform parameters corresponding to the respective encoding positions also includes: The historical pulse train array of the at least one historical period is acquired.

4. The method according to claim 1 or 2, characterized in that: Before collecting the spatiotemporal signals corresponding to each local spatial position in the monitored area, the method further includes: Performing position encoding on the target pixel unit to obtain the encoding position corresponding to the target pixel unit; After the pulse data corresponding to each of the encoding positions are obtained, the method further includes: For any of the encoding positions, in response to the pulse data corresponding to the encoding position being in a preset state, the intensity accumulation value corresponding to the encoding position is reset to enter the next accumulation stage.

5. A device for encoding a space-time signal, characterized in that: include: A signal collector is used to collect the spatiotemporal signals corresponding to each local spatial position in the monitored area, wherein the spatiotemporal signals corresponding to each local spatial position are collected by a pixel unit having a coding position; and convert each of the spatiotemporal signals into a signal strength value corresponding to each of the coding positions; A signal accumulator, used to accumulate the signal strength values ​​corresponding to each of the encoding positions to obtain the intensity accumulation values ​​corresponding to each of the encoding positions; A filter, used for transforming the intensity accumulation values ​​corresponding to the encoding positions according to the transformation parameters corresponding to the encoding positions, to obtain the pulse data corresponding to the encoding positions; A signal decoder, used to determine the pulse sequence array corresponding to the area to be monitored based on the pulse data corresponding to each of the encoding positions in the current cycle, as the encoding result of the spatiotemporal signal of the area to be monitored; The device also includes: A processing module, configured to: perform target signal detection on a historical pulse sequence array of at least one historical period to obtain a target coding position corresponding to the target signal; adjust a historical transformation time interval and a historical intensity threshold corresponding to the target coding position based on the target signal to obtain an adjusted transformation time interval and an adjusted intensity threshold corresponding to the target coding position; determine a transformation time interval and an intensity threshold corresponding to each coding position in the current period based on the adjusted transformation time interval and the adjusted intensity threshold corresponding to the target coding position and the historical transformation time interval and the historical intensity threshold corresponding to other coding positions except the target coding position; The filter control module is used to control the filter to transform the intensity accumulation value corresponding to each encoding position according to the transformation time interval and the intensity threshold corresponding to each encoding position.

6. An electronic device, characterized in that: The invention comprises a coding device for a space-time signal as described in claim 5, and controls the coding device for a space-time signal by executing instructions to implement the coding method for a space-time signal as described in any one of claims 1-4.