Pulse signal coding processing method and device, pulse camera, equipment and medium

By converting photon streams into pulse signals and generating pulse sequences, the problems of low efficiency and storage and transmission pressure in high-speed motion imaging of traditional cameras are solved, and continuous imaging with high temporal sensitivity and high frequency expression is realized.

CN118214487BActive Publication Date: 2024-11-19SPIKE VISION (BEIJING) TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211620656.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-15
Publication Date
2024-11-19
Estimated Expiration
2042-12-15

AI Technical Summary

Technical Problem

Traditional cameras and camcorders are inefficient at representing the high-speed changes in photon flow, and high frame rate imaging results in excessively short exposure times and insufficient exposure, making it difficult to achieve high-quality high-speed motion imaging. Furthermore, high data rates create storage and transmission pressures.

Method used

By employing a pulse vision model, photon streams are converted into pulse signals, generating pulse sequences. These sequences are sampled at a preset sampling frequency and encoded within the data frame output period, enabling continuous recording and digital representation of the optical process while reducing storage and transmission pressure.

Benefits of technology

It achieves continuous imaging with high temporal sensitivity and high-frequency expression capability, enabling the recording of high-speed motion, reducing data storage and transmission pressure, and supporting scientific research and high-precision detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118214487B_ABST
    Figure CN118214487B_ABST
Patent Text Reader

Abstract

The embodiments of the present disclosure disclose a pulse signal encoding processing method and apparatus, a pulse camera, a device and a medium, which convert a received photon stream into a pulse signal characterizing a light intensity change process; based on the generation time of the pulse signal corresponding to the spatial region, a pulse sequence for expressing the light signal and its change process in the spatial region is generated according to a timing relationship; the pulse sequence is sampled according to a preset sampling frequency to obtain the pulse characteristic value at each sampling time; in response to the end of a data frame output cycle, the data to be encoded in the data frame output cycle is obtained according to the current encoding method and the pulse characteristic value at each sampling time in the data frame output cycle; based on the current encoding method, the data to be encoded in the data frame output cycle is encoded to obtain the encoding information corresponding to the data frame output cycle, thereby realizing continuous imaging and reducing the pressure of data storage and transmission.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to pulse vision technology and coding technology, and in particular to a pulse signal coding processing method and device, a pulse camera, equipment and medium. Background Art

[0002] The photon stream moving at the speed of light is a discrete random process that conforms to the Poisson distribution. The sampling theorem for continuous signals is not suitable for expressing the photon stream. Traditional cameras and video cameras use a timed accumulation mode for imaging, which cannot effectively express the high-speed change process of the photon stream. The pulse vision model uses a quantitative accumulation mode to accumulate a certain number of photons into a pulse and express the photon stream with a bit stream. It can better exert the physical potential of photoelectric detection devices and achieve high-precision expression and continuous recording of high-speed light processes. The pulse vision model verifies the high-speed continuous imaging capability. However, although the pulse vision model can achieve high-speed imaging and output, the corresponding pulse sampling frequency usually needs to reach one million hertz, and the camera's output bit rate is also huge, which is difficult for ordinary camera systems to bear. Summary of the invention

[0003] The embodiments of the present disclosure provide a pulse signal coding processing method and apparatus, a pulse camera, a device and a medium to solve or at least partially solve the above-mentioned technical problems.

[0004] In one aspect of an embodiment of the present disclosure, a method for encoding and processing a pulse signal is provided, comprising:

[0005] Convert the received photon stream into a pulse signal representing the light intensity change process;

[0006] Based on the generation time of the pulse signal corresponding to the spatial region, a pulse sequence for expressing the optical signal of the spatial region and its change process is generated according to the time sequence relationship, wherein the pulse sequence includes a pulse characteristic value for indicating whether a pulse signal is generated at each time;

[0007] Sampling the pulse sequence according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment;

[0008] In response to the end of a data frame output cycle, the data to be encoded in the data frame output cycle is obtained according to the current encoding mode and the pulse characteristic value at each sampling moment in the data frame output cycle; wherein the data frame output cycle is determined based on a preset data frame output frequency;

[0009] Based on the current encoding mode, the data to be encoded within the data frame output period is encoded to obtain encoding information corresponding to the data frame output period.

[0010] Optionally, in any of the above method embodiments of the present disclosure, sampling the pulse sequence according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment includes:

[0011] Follow 2 n The sampling frequency of Hz is used to sample the pulse sequence to obtain the 1 / 2 n The pulse characteristic value within the time unit, the 1 / 2 n The pulse characteristic value within the time unit is used to characterize 1 / 2 n An indicator of whether there is a pulse signal within a time unit; wherein the value of n is an integer greater than or equal to 1.

[0012] Optionally, in any of the above method embodiments of the present disclosure, the preset data frame output frequency is 2 m The data frame output frequency in Hertz; wherein the value of m is an integer greater than or equal to 0, and the value of m is less than the value of n.

[0013] Optionally, in any of the above method embodiments of the present disclosure, the method further includes:

[0014] Obtaining the current encoding mode;

[0015] The current coding method includes at least one of the following: a pulse quantity coding method, an instant pulse coding method, and a strongest pulse coding method.

[0016] Optionally, in any of the above method embodiments of the present disclosure, when the current encoding mode is a pulse number encoding mode, obtaining the data to be encoded in the data frame output period according to the current encoding mode and the pulse characteristic values ​​at each sampling moment in the data frame output period includes:

[0017] Based on the pulse characteristic values ​​at each sampling moment in the data frame output period, determining the number of pulse signals generated in the data frame output period as the data to be encoded in the data frame output period;

[0018] The value range of the number of pulse signals generated within the data frame output period is [0,2 n-m -1].

[0019] Optionally, in any of the above method embodiments of the present disclosure, encoding the data to be encoded in the data frame output period based on the current encoding mode to obtain the encoding information corresponding to the data frame output period includes:

[0020] The number of pulse signals generated within the data frame output period is encoded into encoding information with a bit depth of nm bits as the encoding information corresponding to the data frame output period.

[0021] Optionally, in any of the above method embodiments of the present disclosure, when the current encoding mode is an instantaneous pulse encoding mode, acquiring the data to be encoded in the data frame output period according to the current encoding mode and the pulse characteristic values ​​at each sampling moment in the data frame output period includes:

[0022] Based on the pulse characteristic values ​​at each sampling moment in the data frame output period, the pulse width of the pulse signal generated in the data frame output period and closest to the current moment is obtained as the data to be encoded in the data frame output period.

[0023] Optionally, in any of the above method embodiments of the present disclosure, encoding the data to be encoded in the data frame output period based on the current encoding mode to obtain the encoding information corresponding to the data frame output period includes:

[0024] The pulse width of the most recent pulse signal is encoded into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

[0025] Optionally, in any of the above method embodiments of the present disclosure, when the current encoding mode is the strongest pulse encoding mode, obtaining the data to be encoded in the data frame output period according to the current encoding mode and the pulse characteristic values ​​at each sampling moment in the data frame output period includes:

[0026] Based on the pulse characteristic values ​​at each sampling moment in the data frame output period, the shortest pulse width among the pulse widths of the pulse signal generated in the data frame output period is obtained as the data to be encoded in the data frame output period.

[0027] Optionally, in any of the above method embodiments of the present disclosure, encoding the data to be encoded in the data frame output period based on the current encoding mode to obtain the encoding information corresponding to the data frame output period includes:

[0028] The shortest pulse width is encoded into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

[0029] Optionally, in any of the above method embodiments of the present disclosure, obtaining the current encoding mode includes:

[0030] Obtaining a coding mode corresponding to the preset coding mode information as the current coding mode; or,

[0031] Determine the encoding mode corresponding to the received encoding mode information as the current encoding mode; or,

[0032] Selecting a coding mode as the current coding mode from the preset coding mode information according to a preset selection method; or,

[0033] Obtain an encoding method corresponding to the current application scenario as the current encoding method.

[0034] Optionally, in any of the above method embodiments of the present disclosure, the method further includes:

[0035] The bit rate of the encoding information is determined based on the preset sampling frequency, the preset data frame output frequency, and the bit depth of the encoding information.

[0036] Optionally, in any of the above method embodiments of the present disclosure, after obtaining the coding information corresponding to the data frame output period, the method further includes:

[0037] A preset compression coding algorithm is used to compress and encode the coding information corresponding to the data frame output period to obtain compressed coding information.

[0038] Optionally, in any of the above method embodiments of the present disclosure, after obtaining the compressed coding information, the method further includes:

[0039] The preset compression coding algorithm is used to decode the compression coding information to obtain the coding information corresponding to the data frame output period.

[0040] Optionally, in any of the above method embodiments of the present disclosure, the method further includes:

[0041] Determine the preset sampling frequency or adjust the preset sampling frequency based on the size of the local storage space, the bit depth of the encoding information, and the number of pixels included in the device for implementing the encoding processing method of the pulse signal; or,

[0042] Determine the preset data frame output frequency or adjust the preset data frame output frequency based on the size of the local storage space, the code rate of the encoding information and the transmission bandwidth; or,

[0043] The preset sampling frequency is determined or adjusted according to the requirements of the current application scenario.

[0044] Optionally, in any of the above method embodiments of the present disclosure, converting the received photon flow into a pulse signal representing a light intensity variation process includes:

[0045] Collecting optical signals in a spatial region, and accumulating the optical signals in the spatial region to obtain a signal accumulation value;

[0046] In response to the signal accumulation value reaching a preset threshold, a pulse signal is generated, and the signal accumulation value is reset so as to be accumulated again.

[0047] Optionally, in any of the above method embodiments of the present disclosure, after accumulating the optical signals in the spatial region to obtain the signal accumulation value, the method further includes:

[0048] In response to the end of a data frame output cycle, if the signal accumulation value within the data frame output cycle does not reach a preset threshold, the signal accumulation value is output.

[0049] Optionally, in any of the above method embodiments of the present disclosure, the method further includes:

[0050] After the signal accumulation value is output, the signal accumulation value is reset to zero to re-accumulate.

[0051] Optionally, in any of the above method embodiments of the present disclosure, after obtaining the coding information corresponding to the data frame output period, the method further includes:

[0052] Determining the number of pulse signals generated within the data frame output period based on the coding information corresponding to the data frame output period;

[0053] The number of pulse signals generated within the data frame output period is converted into a digital image with a bit depth of nm bits.

[0054] Optionally, in any of the above method embodiments of the present disclosure, after obtaining the coding information corresponding to the data frame output period, the method further includes:

[0055] Based on the current encoding mode and the encoding information corresponding to the data frame output period, a light intensity value corresponding to the encoding information is determined.

[0056] Optionally, in any of the above method embodiments of the present disclosure, when the current encoding mode is a pulse number encoding mode, determining the light intensity value corresponding to the encoding information based on the current encoding mode and the encoding information corresponding to the data frame output period includes:

[0057] Determining the number of pulse signals generated within the data frame output period based on the coding information corresponding to the data frame output period;

[0058] Based on the data frame output period and the number of pulse signals generated within the data frame output period, an average light intensity value within the data frame output period is acquired.

[0059] Optionally, in any of the above method embodiments of the present disclosure, when the current encoding mode is an instantaneous pulse encoding mode, determining the light intensity value corresponding to the encoding information based on the current encoding mode and the encoding information corresponding to the data frame output period includes:

[0060] Determine the pulse width of the pulse signal generated within the data frame output period and closest to the current moment based on the coding information corresponding to the data frame output period;

[0061] Based on the pulse width of the most recent pulse signal, an instantaneous light intensity value corresponding to the pulse width of the most recent pulse signal is obtained.

[0062] Optionally, in any of the above method embodiments of the present disclosure, when the current coding mode is the strongest pulse coding mode, determining the light intensity value corresponding to the coding information based on the current coding mode and the coding information corresponding to the data frame output period includes:

[0063] Determine the shortest pulse width among the pulse widths of the pulse signals generated within the data frame output period based on the coding information corresponding to the data frame output period;

[0064] Based on the shortest pulse width, a maximum light intensity value corresponding to the shortest pulse width is acquired.

[0065] Optionally, in any of the above method embodiments of the present disclosure, after determining the light intensity value corresponding to the coding information based on the current coding mode and the coding information corresponding to the data frame output period, the method further includes:

[0066] Target detection is performed based on light intensity values ​​corresponding to the coded information corresponding to one or more spatial regions in the observation scene.

[0067] Optionally, in any of the above method embodiments of the present disclosure, after determining the light intensity value corresponding to the coding information based on the current coding mode and the coding information corresponding to the data frame output period, the method further includes:

[0068] Using a preset pulse reconstruction algorithm, a reconstructed image of the observation scene is generated based on the light intensity values ​​corresponding to the encoded information corresponding to all spatial areas in the observation scene; wherein the grayscale value of each pixel in the reconstructed image represents the light intensity value of the spatial area corresponding to each pixel.

[0069] Another aspect of the embodiments of the present disclosure provides a pulse signal encoding processing device, including:

[0070] A conversion unit, used for converting the received photon flow into a pulse signal representing a light intensity variation process;

[0071] A generating unit, configured to generate a pulse sequence for expressing the optical signal of the spatial region and its changing process according to a time sequence relationship based on the generation time of the pulse signal corresponding to the spatial region, wherein the pulse sequence includes a pulse characteristic value for indicating whether a pulse signal is generated at each time;

[0072] A sampling unit, used to sample the pulse sequence according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment;

[0073] An acquisition unit, configured to, in response to the end of a data frame output cycle, acquire the data to be encoded in the data frame output cycle according to the current encoding mode and the pulse characteristic values ​​at each sampling moment in the data frame output cycle; wherein the data frame output cycle is determined based on a preset data frame output frequency;

[0074] The encoding unit is used to encode the data to be encoded within the data frame output period based on the current encoding method to obtain encoding information corresponding to the data frame output period.

[0075] Another aspect of the embodiments of the present disclosure provides a pulse camera, comprising the device described in any of the above embodiments of the present disclosure.

[0076] Another aspect of the embodiments of the present disclosure provides an electronic device, including:

[0077] Memory for storing computer programs;

[0078] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method described in any of the above embodiments of the present disclosure is implemented.

[0079] Another aspect of the embodiments of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in any of the above embodiments of the present disclosure is implemented.

[0080] Based on the pulse signal encoding processing method and apparatus, pulse camera, equipment and medium provided by the above-mentioned embodiments of the present disclosure, by converting the received photon stream into a pulse signal that characterizes the light intensity change process, based on the generation time of the pulse signal corresponding to the spatial region, a pulse sequence for expressing the light signal of the spatial region and its change process is generated according to the timing relationship; the pulse sequence is sampled according to the preset sampling frequency to obtain the pulse characteristic value at each sampling time; whenever a data frame output cycle ends, the data to be encoded in the data frame output cycle is obtained according to the current encoding method and the pulse characteristic value at each sampling time in the data frame output cycle, and then, the data to be encoded in the data frame output cycle is encoded based on the current encoding method to obtain the encoding information corresponding to the data frame output cycle. Therefore, the embodiments of the present disclosure have at least the following beneficial technical effects:

[0081] The continuous recording and digital expression of the light process is achieved through the pulse sequence. The pulse sequence generated by each pixel can be used to estimate the light intensity at any time and obtain the image at any time, thus achieving continuous imaging. It has high time sensitivity, high-frequency expression ability and super expression ability for high-speed motion process. It can effectively record high-speed motion, such as rotating wheels, high-speed table tennis and even football, and can capture the motion details in the monitoring scene, which can support special needs such as scientific research and high-precision detection.

[0082] In addition, after the pulse sequence is generated, the embodiment of the present disclosure does not output the pulse sequence in real time, but encodes it at a lower frequency according to the data frame output period and then outputs it, thereby reducing the pressure of data storage and transmission.

[0083] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0085] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0086] Figure 1 The present invention is a flowchart of an embodiment of a pulse signal encoding processing method disclosed in the present invention.

[0087] Figure 2 FIG. 4 is an example diagram of a pulse sequence in an embodiment of the present disclosure.

[0088] Figure 3 The present invention is a flow chart of another embodiment of the pulse signal encoding processing method disclosed in the present invention.

[0089] Figure 4The present invention is a flowchart of another embodiment of the pulse signal encoding processing method disclosed in the present invention.

[0090] Figure 5 The present invention is a flowchart of another embodiment of the pulse signal encoding processing method disclosed in the present invention.

[0091] Figure 6 The figure is a schematic diagram of the structure of an embodiment of a pulse signal encoding and processing device disclosed in the present invention.

[0092] Figure 7 The figure is a schematic diagram of the structure of another embodiment of the pulse signal encoding processing device disclosed in the present invention.

[0093] Figure 8 The figure is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0094] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0095] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0096] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0097] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0098] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0099] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0100] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0101] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0102] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0103] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0104] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0105] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0106] As described in the background technology section, the upper limit of traditional camera equipment in expressing high-speed motion is its frame rate. The frame rate of conventional film and television camera equipment is only tens of hertz, which cannot express high-speed processes. The frame rate of high-speed camera equipment can reach one thousand or even ten thousand frames per second or even higher, but the substantial increase in frame rate means that the exposure time of a single frame image is too short and the exposure is seriously insufficient, resulting in a significant decrease in image quality. Even with expensive optical systems and special optoelectronic devices, it is difficult to fundamentally solve the problem. Moreover, high frame rate means high data rate, so high-speed camera equipment can often only record for a short time, and it is difficult to support real-time output and long-term storage. In addition, the image acquisition and processing circuit design is expensive or even impossible. The inventors of the present disclosure have found through extensive research that all these problems are caused by the traditional video acquisition and representation using the equal time interval acquisition method of "first space, then time", and this method is only a technical choice made based on the human visual persistence characteristics when movies appear, and it does not mean that it is the best solution for expressing dynamic images.

[0107] Each pixel of the complementary metal oxide semiconductor (CMOS) sensor is an independent photodetector that can independently record the high-speed change process of light. Using pulse modulation, the pixel starts to accumulate charge from the clear state. When the voltage difference reaches the specified threshold, a simple pulse is generated and automatically reset, and charge accumulation begins again. The time interval between two pulses is the cumulative duration of the latter pulse, which is called the pulse width. In this way, each pixel converts the received extremely high-speed photon stream into an ultra-high-speed pulse stream (Spiking). The pulse stream is the pulse sequence formed by the pulse on the time axis, which is the digital expression of the photocurrent process. The pulse sequence generated by each pixel is arranged in the pixel space to form a pulse sequence array, which is the digital expression of the incident light flow on the image surface. It is a visual information expression that is closer to the optical physical process than images and videos. This is the Spike Vision Model (SVM). The camera that works in this way is called a Spike Camera. The pulse camera realizes the continuous recording and expression of the light process: from the pulse sequence generated by the quantitative accumulation of each pixel, the light intensity at any time can be estimated, and thus the image at any time can be obtained, that is, continuous imaging is realized.

[0108] Pulse cameras naturally have high time sensitivity. Obviously, the shorter the pulse width, the stronger the ability to express high-speed motion processes. The time sensitivity of pulse cameras to strong light can reach 0.1us, which is the perception ability of tens of millions of hertz. The time sensitivity to weak light is not high, but it can still distinguish the intensity of different weak light according to the difference in cumulative time. SVM expresses photon flow / electron flow in the form of pulse sequence. However, this high-frequency expression ability of pulse sequence requires extremely high data rate, which puts great pressure on data storage and transmission.

[0109] Therefore, how to utilize the high-frequency expression capability of pulse sequences while reducing storage and transmission pressure is a technical problem that needs to be solved urgently.

[0110] Figure 1 FIG. 1 is a flow chart of an embodiment of a pulse signal coding processing method disclosed in the present invention. Figure 1 As shown, the encoding processing method of the pulse signal in this embodiment includes:

[0111] 102, converting the received photon stream into a pulse signal representing the light intensity variation process.

[0112] 104. Based on the generation time of the pulse signal corresponding to the spatial region, a pulse sequence for expressing the optical signal of the spatial region and its change process is generated according to the time sequence relationship. The pulse sequence includes a pulse characteristic value for indicating whether a pulse signal is generated at each time.

[0113] The pulse sequence in the embodiments of the present disclosure is also called a pulse stream or a pulse signal stream. The pulse characteristic value is used to indicate whether a pulse signal is generated. Two preset values ​​can be used to respectively indicate whether a pulse signal is generated. For example, in some implementations, binary symbols 1 and 0 can be used as pulse characteristic values, respectively indicating whether a pulse signal is generated at the corresponding moment or not.

[0114] 106 , sampling the pulse sequence generated by operation 104 according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment.

[0115] 108. In response to the end of a data frame output cycle, the data to be encoded in the data frame output cycle is obtained according to the current encoding mode and the pulse characteristic values ​​at each sampling time in the data frame output cycle.

[0116] The data frame output period is determined based on a preset data frame output frequency.

[0117] 110 , based on the current encoding mode, encode the data to be encoded within the data frame output period to obtain encoding information corresponding to the data frame output period.

[0118] like Figure 2As shown in FIG. 1 , an example diagram of a pulse sequence in an embodiment of the present disclosure is shown. Figure 2 As shown, x and y are the two coordinate axes of the pixel coordinate system, and t is the time coordinate axis. The pulse camera represents visual information in the form of an H*W*T pulse array, where H*W is the spatial resolution of the pulse camera and T is the number of sampling times of the pulse camera. The pulse signal sequence corresponding to each spatial region is a pulse sequence, and the cross section of the pulse array at a certain moment is a pulse matrix. The pulse array consists of two symbols 1 and 0, where 1 ( Figure 2 The solid point in the middle indicates that a pulse signal is generated at the pixel (corresponding to a spatial area) at the sampling moment, where the pixel corresponds to the pixel point in the image; 0 indicates that no pulse signal is generated at the pixel. The pulse camera can record the change of the instantaneous light intensity value of each pixel, without the concept of frame rate and exposure time, breaking through the limitations of traditional cameras.

[0119] Based on this embodiment, continuous recording and digital expression of the light process is achieved through a pulse sequence. The pulse sequence generated by each pixel can be used to estimate the light intensity at any time and obtain the image at any time, thereby realizing continuous imaging. It has high time sensitivity, high-frequency expression capability and super expression capability for high-speed motion processes. It can effectively record high-speed motion, such as rotating wheels, high-speed table tennis and even football, and can capture motion details in the monitoring scene, and can support special needs such as scientific research and high-precision detection. In addition, after generating a pulse sequence, this embodiment does not output the pulse sequence in real time, but encodes it at a lower frequency according to the data frame output period and then outputs it, thereby reducing the pressure of data storage and transmission.

[0120] In the embodiment of the present disclosure, the optical signal of the spatial region can be collected by a signal collector, such as a photosensitive device in a camera, a pixel of a CMOS sensor, etc. The optical signal collected by the signal collector may include, but is not limited to, laser, infrared light, or visible light, etc. That is, in the embodiment of the present disclosure, each of the multiple signal collectors can be used to collect the optical signal of each spatial region in the observation scene and accumulate them for subsequent processing.

[0121] Optionally, in some possible implementations, in operation 102, a signal accumulation value can be obtained by collecting optical signals in a spatial region and accumulating the optical signals in the spatial region; in response to the signal accumulation value reaching a preset threshold, a pulse signal is generated, and the signal accumulation value is reset (e.g., to zero) for re-accumulation.

[0122] Each signal collector can perform time-domain sampling on the optical signal at a specified spatial region in the observation scene. In the collector array formed by arranging multiple signal collectors, each signal collector collects the optical signal of a spatial region in the observation scene. The spatial regions corresponding to different signal collectors do not overlap. The signal collectors in the collector array cooperate with each other to cover the entire observation scene, thereby realizing the sampling of the optical signal in the entire observation scene. The pulse sequences corresponding to all spatial regions in the observation scene are arranged according to the spatial position relationship of the spatial regions to form a pulse array. Figure 1 The embodiment shown is a processing process for the optical signals of the spatial region collected by each signal collector. The optical signals of the spatial region collected by each signal collector in the collector array are respectively processed according to the above Figure 1 The processes of the illustrated embodiment are processed independently, so that the light signal in the entire observation scene can be processed to obtain the coding information of the pulse signal corresponding to the area of ​​the entire observation scene. The observation scene in the disclosed embodiment is a scene whose visual information needs to be recorded, such as a traffic intersection including a signal light, a scene including a rotating wheel, a scene including a high-speed moving object, etc.

[0123] Optionally, in some of the possible implementations, in operation 102, each signal collector can be used to convert the instantaneous light intensity value of the optical signal collected each time into an electrical signal (such as a current signal, a voltage signal, etc.) that is positively correlated with the instantaneous light intensity value, and the signal intensity value of the electrical signal is positively correlated with the collected light intensity value. After that, the signal intensity value of the electrical signal is passed to the connected signal accumulator, and then, the received signal intensity values ​​are accumulated by the signal accumulator to obtain a signal accumulation value, and the signal accumulation amount is passed to the connected filter. The function of the signal collector is to realize optical signal acquisition and photoelectric conversion, that is, to collect optical signals and convert the collected optical signals into electrical signals (such as current signals, voltage signals, etc.). In the disclosed embodiment, the electrical signal has a signal intensity, and accordingly, the optical signals in the spatial region are accumulated, that is, the signal intensity of the corresponding electrical signal is accumulated, and the obtained signal accumulation value represents the accumulated signal intensity.

[0124] Accordingly, the filter may filter the received signal accumulation value based on a preset threshold value, and in response to the signal accumulation value reaching the preset threshold value, the filter generates a pulse signal and resets the signal accumulation value on the connected signal accumulator (eg, to 0).

[0125] The signal collector is a high-sensitivity photosensor, and the signal accumulator is a high-sensitivity conversion device. The two can work together to accurately measure the number of collected photons. The pulse sequence output by the filter represents the amount of photon irradiation in the corresponding spatial area.

[0126] Optionally, in some of the possible implementations, in operation 102, after accumulating the optical signal of the spatial region to obtain the signal accumulation value, in response to the end of a data frame output cycle, if the signal accumulation value in the data frame output cycle does not reach a preset threshold, the signal accumulation value is output. Thereafter, operation 108 is performed to obtain the data to be encoded in the data frame output cycle according to the current encoding mode and the pulse characteristic value and / or the signal accumulation value at each sampling moment in the data frame output cycle.

[0127] Further optionally, after the signal accumulation value is output, the signal accumulation value may be reset to zero to re-accumulate.

[0128] Based on the embodiments of the present disclosure, when the signal accumulation value does not reach a preset threshold value within a data frame output period, the signal accumulation value may be directly output.

[0129] Optionally, in some possible implementations, the preset sampling frequency in operation 106 is 2 n Hertz (Hz), where n is an integer greater than or equal to 0. Then in operation 106, according to 2 n The sampling frequency of Hz is used to sample the pulse sequence generated by operation 104, and the corresponding time period from 1 / 2 before the sampling time period is obtained. n The pulse characteristic value within the time unit, where 1 / 2 n The pulse characteristic value within the time unit is used to characterize 1 / 2 n Whether there is a pulse signal in the time unit, which can be 0 or 1, 0 means that 1 / 2 n No pulse signal is generated within the time unit, 1 means that 1 / 2 n A pulse signal is generated within the time unit.

[0130] For example, in a specific example, assuming n = 2, in a unit time, each sampling moment corresponds to 1 / 2 n The time units are: pulse generation, pulse generation, no pulse generation, and pulse generation. Then, the 1 / 2 of each sampling moment in the unit time obtained by operation 106 is n The pulse characteristic values ​​in the time unit are 1101 respectively. In a specific implementation, the unit time can be 1 second (s), 1 millisecond (ms), 1 microsecond (us), etc., and the embodiment of the present disclosure does not limit this. Then the corresponding 1 / 2 n The time unit is 1 / 2 n s, 1 / 2 n ms, 1 / 2 nus, etc. Unless otherwise stated, the disclosed embodiment is described with the unit time of 1s as an example, but is also applicable to other unit times. For example, when the unit time is 1s, based on this specific example, the pulse characteristic values ​​within 250ms corresponding to each sampling moment within 1s are 1101 respectively.

[0131] Accordingly, in some possible implementations, the preset data frame output frequency in operation 108 is 2 m The data frame output frequency is Hz; where m is an integer greater than or equal to 0, and m is less than n. In a specific implementation, m can be much smaller than n. For example, when n is 20, m is 5. The sampling time length between two adjacent data frames is 1 / 2 n-m The unit time is called the data frame period, and the bit depth of the data frame is nm bits (called: multi-bit). This configuration can be called the (n,m) synchronous coding system.

[0132] When recording the color of a digital image, the computer actually uses the bit depth required for each pixel to represent it. The reason why computers can display colors is that they use a counting unit called a bit to record the data of the color represented. When these data are recorded in the computer in a certain arrangement, they constitute a computer file of a digital image. A bit is the smallest unit in the computer memory, which is used to record the color value of each pixel. The richer the color of the image, the more bits there are. The number of bits used for each pixel in the computer is the bit depth.

[0133] Optionally, in the embodiment of the present disclosure, before operation 108, a current coding mode may be obtained, wherein the current coding mode may include at least one of the following: a pulse number coding mode, an instant pulse coding mode, a strongest pulse coding mode, and the like.

[0134] For example, the encoding mode information may be pre-set, and in this embodiment, the encoding mode corresponding to the pre-set encoding mode information may be obtained as the current encoding mode. Alternatively, the encoding mode information may be input by the user, and in this embodiment, the encoding mode corresponding to the received encoding mode information may be determined as the current encoding mode. Alternatively, multiple encoding mode information may be pre-set, and in this embodiment, a encoding mode may be selected from the pre-set encoding mode information according to a preset selection method (such as sequential selection, random selection, reverse selection, etc.) as the current encoding mode. Alternatively, the encoding mode corresponding to each application scenario may be pre-set, and in this embodiment, the current application scenario may be determined according to the currently set application scenario information or the application scenario information input by the user, and the encoding mode corresponding to the current application scenario may be obtained as the current encoding mode.

[0135] Figure 3 FIG. 1 is a flow chart of another embodiment of the pulse signal coding processing method disclosed in the present invention. Figure 3 As shown, when the current encoding mode is the pulse number encoding mode, the encoding processing method of this embodiment includes:

[0136] 302 , collecting optical signals in a spatial region, and accumulating the optical signals in the spatial region to obtain a signal accumulation value.

[0137] 304 , in response to the signal accumulation value reaching a preset threshold, a pulse signal is generated, and the signal accumulation value is reset (eg, reset to zero) to restart accumulation.

[0138] 306. Based on the generation time of the pulse signal corresponding to the above-mentioned spatial region, a pulse sequence for expressing the optical signal of the spatial region and its change process is generated according to the time sequence relationship. The pulse sequence includes pulse characteristic values ​​for indicating whether a pulse signal is generated at each time.

[0139] 308 , sampling the pulse sequence generated by operation 306 according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment.

[0140] 310, in response to the end of a data frame output cycle, based on the pulse characteristic values ​​at each sampling moment in the ended data frame output cycle, determine the number of pulse signals generated in the data frame output cycle as the data to be encoded in the data frame output cycle.

[0141] The data frame output period is determined based on the preset data frame output frequency. The number of pulse signals generated in each data frame output period is in the range of [0,2 n-m -1].

[0142] 312, encode the number of pulse signals generated within the data frame output period into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

[0143] Figure 4 FIG. 1 is a flow chart of another embodiment of the pulse signal coding processing method disclosed in the present invention. Figure 4 As shown, when the current encoding mode is the instantaneous pulse encoding mode, the encoding processing method of this embodiment includes:

[0144] 402 , collecting optical signals in a spatial region, and accumulating the optical signals in the spatial region to obtain a signal accumulation value.

[0145] 404 , in response to the signal accumulation value reaching a preset threshold, a pulse signal is generated, and the signal accumulation value is reset (ie, set to zero) to allow for re-accumulation.

[0146] 406. Based on the generation time of the pulse signal corresponding to the above-mentioned spatial region, a pulse sequence for expressing the optical signal of the spatial region and its change process is generated according to the time sequence relationship. The pulse sequence includes pulse characteristic values ​​for indicating whether a pulse signal is generated at each time.

[0147] 408 , sampling the pulse sequence generated by operation 406 according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment.

[0148] 410. In response to the end of a data frame output cycle, based on the pulse characteristic values ​​at each sampling moment in the ended data frame output cycle, the pulse width of the pulse signal generated in the data frame output cycle and closest to the current moment is obtained as the data to be encoded in the data frame output cycle.

[0149] The data frame output period is determined based on a preset data frame output frequency.

[0150] 412, encode the pulse width of the most recent pulse signal into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

[0151] In a specific application, if no pulse signal appears within the data frame output period, the coded information with a bit depth of nm bits can be represented as all 0s (i.e., nm 0s); if a pulse signal appears within the data frame output period but its starting point is before the data frame period, the pulse width of the pulse signal is obtained and encoded into coded information with a bit depth of nm bits.

[0152] Figure 5 FIG. 1 is a flow chart of another embodiment of the pulse signal coding processing method disclosed in the present invention. Figure 5 As shown, when the current coding mode is the strongest pulse coding mode, the coding processing method of this embodiment includes:

[0153] 502 , collecting optical signals in a spatial region, and accumulating the optical signals in the spatial region to obtain a signal accumulation value.

[0154] 504 , in response to the signal accumulation value reaching a preset threshold, a pulse signal is generated, and the signal accumulation value is reset (ie, set to zero) to restart accumulation.

[0155] 506. Based on the generation time of the pulse signal corresponding to the above-mentioned spatial region, a pulse sequence for expressing the optical signal of the spatial region and its change process is generated according to the time sequence relationship. The pulse sequence includes pulse characteristic values ​​for indicating whether a pulse signal is generated at each time.

[0156] 508 , sampling the pulse sequence generated by operation 506 according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment.

[0157] 510. In response to the end of a data frame output cycle, based on the pulse characteristic values ​​at each sampling moment in the ended data frame output cycle, the shortest pulse width of the pulse signal generated in the data frame output cycle is obtained as the data to be encoded in the data frame output cycle.

[0158] The data frame output period is determined based on a preset data frame output frequency.

[0159] 512, encode the shortest pulse width into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

[0160] In a specific application, if no pulse signal appears within the data frame output period, the coded information with a bit depth of nm bits can be represented as all 0s (i.e., nm 0s); if a pulse signal appears within the data frame output period but its starting point is before the data frame period, the pulse width of the pulse signal is obtained and encoded into coded information with a bit depth of nm bits.

[0161] Optionally, in some of the possible implementations, the code rate of the encoding information may also be determined based on a preset sampling frequency, a preset data frame output frequency, and a bit depth of the encoding information.

[0162] Among them, the code rate is the bit rate, that is, the number of data bits (i.e. the number of bits) transmitted per unit time when the coded information is transmitted, and the unit is bit / s (bit / s or bps), kilobit / s (kbit / s or kbps) or megabit / s (Mbps). The higher the code rate, the more data is stored and transmitted per unit time, and the better the recording and expression of visual information. However, the larger the amount of coded information per unit time, the higher the corresponding sampling frequency and data frame output frequency, the larger the storage space occupied, and the larger the transmission bandwidth required for external transmission; if the code rate is lower, the situation is just the opposite.

[0163] Based on the above embodiment, the sampling frequency is 2 n Hz, data frame output frequency is 2 m Hz, and the bit depth of the coded information is nm bits, the bit rate of the coded information corresponding to each signal collector is determined to be 2m Similarly, based on the coding information corresponding to all spatial regions in the observation scene, the bit rate of the coding information corresponding to the observation scene can be determined to be S*2 m (nm) bps. Wherein, S is the number of signal collectors in the collector array covering the entire observation scene area, and the value of S is an integer greater than 1.

[0164] Optionally, the pulse signal coding processing method of the embodiment of the present invention can be implemented by a pulse signal coding processing device or a pulse camera. After the coding processing device or the pulse camera obtains the coding information, it can store the coding information in a local storage space in a first-in-first-out (FIFO) manner, and output the coding information in the local storage space according to a certain data frame output frequency for subsequent analysis, target retrieval or image reconstruction and other applications.

[0165] In a specific implementation, when the coded information corresponding to the area of ​​the entire observation scene is stored in the local storage space, the storage area of ​​the local storage space can be divided into multiple storage areas according to the positional relationship of the spatial areas corresponding to each signal collector in the collector array in the entire observation scene, and the coded information corresponding to each spatial area can be stored in the corresponding storage area in the local storage space in a FIFO manner. When the coded information in the local storage space is output to the outside, the coded information in the same time period in multiple storage areas can be output according to the time sequence relationship. The disclosed embodiment does not limit the storage method and output method of the coded information in the local storage space.

[0166] Optionally, in some of the possible implementations, a sampling frequency may be predetermined as the preset sampling frequency, or the predetermined sampling frequency may be adjusted according to actual requirements, and the adjusted sampling frequency may be used as the preset sampling frequency.

[0167] For example, in some specific examples, the preset sampling frequency can be determined based on the size of the local storage space, the bit depth of the coding information, and the number of pixels (i.e., signal collectors) included in the coding processing device for implementing the coding processing method of the pulse signal, or the preset sampling frequency can be adjusted so that the local storage space can simultaneously meet the storage requirements of the coding information and the requirements for outputting the coding information. The local storage space is the size of the storage space of the coding processing device or the pulse camera itself.

[0168] Alternatively, in some other specific examples, the preset sampling frequency can be determined or adjusted based on the size of the local storage space, the code rate of the coded information, and the transmission bandwidth, wherein the transmission bandwidth is the bandwidth between the coding processing device or the pulse camera and the peer device when transmitting data externally. Thus, the coded information obtained by encoding based on the above embodiment of the present disclosure can simultaneously meet the size of the local storage space and the requirement of outputting the coded information externally.

[0169] Alternatively, in some specific examples, the preset sampling frequency may be determined or adjusted according to the requirements of the current application scenario. For example, the sampling frequency may be determined according to the different movement speeds of the observed object (located in the monitoring scene). When the movement speed of the observed object is high or increased, in order to achieve continuous recording and expression of the high-speed movement process of the observed object, it is necessary to increase the sampling frequency. On the contrary, when the movement speed of the observed object is low or decreased, the sampling frequency may be reduced to reduce the pressure of storage, transmission and subsequent calculation and processing of the coded information.

[0170] In addition, in some possible implementations, the data frame output frequency can be predetermined as the above-mentioned preset data frame output frequency, or the predetermined data frame output frequency can be adjusted according to actual needs, and the adjusted data frame output frequency can be used as the above-mentioned preset data frame output frequency.

[0171] For example, in some specific examples, the above-mentioned preset data frame output frequency can be determined or the preset data frame output frequency can be adjusted based on the size of the local storage space, the preset sampling frequency, the bit depth of the encoding information, and the number of pixels included in the encoding processing device, so that the local storage space can simultaneously meet the storage requirements of the encoding information and the requirements for outputting the encoding information to the outside.

[0172] Alternatively, in other specific examples, the above-mentioned preset data frame output frequency can be determined or the preset data frame output frequency can be adjusted based on the size of the local storage space, the bit rate of the encoded information and the transmission bandwidth, so that the encoded information encoded based on the above-mentioned embodiments of the present disclosure can simultaneously meet the size of the local storage space and the requirement of outputting the encoded information to the outside.

[0173] Alternatively, in some specific examples, the preset data frame output frequency may be determined or adjusted according to the requirements of the current application scenario to meet the real-time observation requirements of the observed object. For example, when the observed object needs to be detected in real time, the data frame output frequency needs to be increased. Conversely, when the real-time detection requirements of the observed object are low, the data frame output frequency may be reduced.

[0174] In the embodiment of the present disclosure, 2 n Hz high-speed sampling frequency to synchronously record 1 / 2 of each sample n Whether there is a pulse within a time unit, and use 2 m The data frame output frequency of Hz encodes the pulse characteristic value in each data frame output period into coding information with a bit depth of nm bits. This coding method can be called multi-bit synchronous coding, and the corresponding pulse visual model can also be called a multi-bit synchronous pulse model.

[0175] The disclosed embodiment proposes multi-bit synchronous coding, which can take into account both the microsecond pulse expression accuracy required for high dynamic range and the millisecond accuracy required for high-speed motion expression. Specifically, the light process is continuously recorded and synchronously expressed by a pulse sequence using a high frequency to achieve high dynamics, but the pulse sequence is not output in real time, but a multi-bit coded data frame is synchronously output using a lower frequency.

[0176] Optionally, in some possible implementations, after obtaining the coding information corresponding to the data frame output period based on the embodiments of the present disclosure, a preset compression coding algorithm, such as predictive coding, run-length coding, entropy coding (for example, regular variable-length codes such as exponential Golomb codes, equal-length coding, or non-equal-length coding based on probability distribution, etc.), etc., can also be used to compress and encode the coding information corresponding to the data frame output period to form a more efficient compressed bit stream and obtain compressed coding information.

[0177] By compressing the encoded information based on a preset compression encoding algorithm, the encoded information can be compressed several times, which helps to save local storage space and improve storage efficiency. It also helps to improve the external transmission efficiency of the encoded information and improve the utilization of bandwidth resources.

[0178] Accordingly, in some possible implementations, after the compressed coded information is stored or transmitted externally, when the coded information is needed, the decoding method corresponding to the above-mentioned preset compression coding algorithm can be used to decode the compressed coded information so as to decode the compressed coded information and restore it to the coded information.

[0179] Optionally, in some possible implementations, after obtaining the coding information corresponding to the data frame output period based on the embodiment of the present disclosure, or after decoding the compressed coding information to obtain the coding information corresponding to the data frame output period, it is also possible to determine the light intensity value corresponding to the coding information based on the current coding method and the coding information corresponding to the data frame output period.

[0180] In a specific implementation, when the current encoding method is a pulse number encoding method, since the encoding information corresponding to the data frame output period records the number of pulse signals generated within the data frame output period, the number of pulse signals generated within the data frame output period can be determined based on the encoding information corresponding to the data frame output period, and then, based on the data frame output period and the number of pulse signals generated within the data frame output period, the average light intensity value within the data frame output period can be obtained.

[0181] In a specific implementation, the shorter the pulse width, the stronger the light; conversely, the longer the pulse width, the weaker the light, that is, the light intensity value is inversely proportional to the pulse width; and within a certain time period, the more pulses there are, the stronger the light is; conversely, the fewer pulses there are, the weaker the light is, that is, the light intensity value is inversely proportional to the number of pulses. Therefore, the light intensity value within the corresponding time period can be estimated based on the pulse width and / or the number of pulses.

[0182] For example, when the current encoding method is the pulse number encoding method, the light intensity value i of any pixel at any time can be obtained in the following way: i = k*C / T. Among them, T is the duration of the data frame output cycle, C represents the number of pulse signals generated in the data frame output cycle, k is a constant greater than zero, and the specific value of k can be set according to actual needs.

[0183] By using pulse number encoding, a high dynamic range image can be reconstructed in a timing mode.

[0184] When the current encoding method is the instantaneous pulse encoding method, since the encoding information corresponding to the data frame output period records the pulse width of the pulse signal generated within the data frame output period and closest to the current moment, the pulse width of the pulse signal generated within the data frame output period and closest to the current moment can be determined based on the encoding information corresponding to the data frame output period, and then, based on the pulse width of the most recent pulse signal, the instantaneous light intensity value corresponding to the pulse width of the most recent pulse signal can be obtained, so that the instantaneous light intensity at the current moment can be obtained.

[0185] For example, when the current encoding mode is the instantaneous pulse encoding mode, the instantaneous light intensity value i corresponding to the pulse width of the most recent pulse signal can be obtained in the following manner: i=k*1 / τ. Wherein, τ is the pulse width of the pulse signal generated within the data frame output period and closest to the current moment, k is a constant greater than zero, and the specific value of k can be set according to actual needs.

[0186] By adopting the instantaneous pulse coding method, the light intensity index at the data frame output moment can be accurately recorded. By comparing the light intensity values ​​at two adjacent output moments to detect whether the light intensity has changed, it can effectively carry out machine vision tasks such as target detection that require detecting light changes at any time, and improve the execution effect of such machine vision tasks.

[0187] When the current encoding method is the strongest pulse encoding method, since the encoding information corresponding to the data frame output period records the shortest pulse width in the pulse width of the pulse signal generated within the data frame output period, the shortest pulse width in the pulse width of the pulse signal generated within the data frame output period can be determined based on the encoding information corresponding to the data frame output period, and then, based on the shortest pulse width, the maximum light intensity value corresponding to the shortest pulse width can be obtained, so that the maximum light intensity within the data frame output period can be obtained.

[0188] For example, when the current encoding mode is the strongest pulse encoding mode, the instantaneous light intensity value i corresponding to the maximum light intensity value corresponding to the shortest pulse width can be obtained in the following manner: i=k*1 / τ. Among them, τ is the shortest pulse width among the pulse widths of the pulse signal generated within the data frame output period, k is a constant greater than zero, and the specific value of k can be set according to actual needs.

[0189] The strongest pulse coding method can record the appearance of the strongest light, even if the pulse width is much shorter than the data frame period, thus ensuring the maximum dynamic range. In contrast, the pulse number coding method will average it into the entire data frame output period, and the instant pulse synchronization coding method may miss such short-term strong pulses. The advantages of the three coding methods, pulse number coding method, instant pulse coding method, and strongest pulse coding method, are the shortcomings of the other two schemes. In actual coding, any one or multiple coding methods can be used at the same time.

[0190] In a specific application, the pulse width is 1 / 2 n The pulse of the time unit corresponds to the maximum value of the photocurrent intensity 2 n ×Q×1.602176634×10 -19 A, where Q is the full well charge. When the pulse width is 1s (2 n time unit), the corresponding photocurrent intensity is Q×1.602176634×10 -19 A. If the photocurrent is accumulated and imaged in 1 second, the dynamic range of the image is 20log2 n When n=20, the shortest pulse width is about 1us and the dynamic range is about 120dB; when n=24, the shortest pulse width is about 62.5ns and the dynamic range is about 144dB.

[0191] The above dynamic range performance estimation assumes that the light intensity and pulse width are linearly related. If nonlinear mapping, such as logarithmic circuits, is used, the dynamic range can be greatly improved.

[0192] Optionally, in some of the possible implementations, after determining the light intensity value corresponding to the coded information, target detection can also be performed based on the light intensity value corresponding to the coded information corresponding to one or more spatial regions in the observation scene. Since the visual information of a specific target is basically unchanged in a specific observation scene, target tracking can be achieved based on the light intensity value of one or more spatial regions in the observation scene in the time domain.

[0193] Based on this embodiment, the motion information of the target contained in the coded information of the observed scene can be obtained, and the description of the position and motion process of the target can be obtained.

[0194] Optionally, in some of the possible implementations, after determining the light intensity value corresponding to the coded information, a preset pulse reconstruction algorithm can be used to generate a reconstructed image of the observed scene based on the light intensity values ​​corresponding to the coded information corresponding to all spatial regions in the observed scene. Each pixel in the reconstructed image corresponds to a signal collector, and the grayscale value of each pixel in the reconstructed image represents the light intensity value of the spatial region corresponding to each pixel. That is, in the reconstructed image, the pixel value of each pixel (i, j) is the light intensity value of the spatial region corresponding to the pixel.

[0195] The above-mentioned preset pulse reconstruction algorithm can be, for example, a pulse reconstruction algorithm (TFI) based on peak potential interval (ISI), a pulse reconstruction algorithm (TFP) based on fixed window sliding, a pulse reconstruction algorithm based on convolutional neural network (CNN), etc. The embodiments of the present disclosure do not limit the specific pulse reconstruction algorithm used.

[0196] Based on this embodiment, the light intensity value of each pixel at any time can be estimated by the pulse width of the pulse in which it is located, so the image at any time can be estimated. By performing target detection on the reconstructed image of the observation scene at a certain time, the position and shape of the target contained in the coded information of the observation scene can be obtained. By performing target detection and tracking on the reconstructed image at each time, the motion information of the target contained in the coded information of the observation scene can be obtained, thereby realizing the recording of the position and motion process of the target.

[0197] In addition, when the current encoding method is the pulse number encoding method, since the encoding information corresponding to the data frame output period records the number of pulse signals generated in the data frame output period, the number of pulse signals generated in the data frame output period can be determined based on the encoding information corresponding to the data frame output period, and the number of pulse signals generated in the data frame output period can be directly converted into a digital image with a bit depth of nm bits, without obtaining the average light intensity value in the data frame output period and generating a reconstructed image based on the light intensity value, and the digital image can be directly obtained. By accumulating the digital images corresponding to k data frame output periods, a k×(nm)-bit high dynamic image can be obtained.

[0198] A digital image is an image expressed in digital form. Since digital images are easier to process, transmit and store, they have become the preferred method for expressing dynamic images such as videos. Based on this embodiment, when the current encoding method is the pulse number encoding method, the digital image can be directly obtained from the encoding information without the need for light intensity estimation and image reconstruction, thereby improving the image generation efficiency and the processing, transmission and storage effects.

[0199] After obtaining the digital image corresponding to the entire observation scene based on the digital image corresponding to the collector array covering the area of ​​the entire observation scene, target detection and tracking can be performed based on the digital image, thereby realizing the recording of the target's position and movement process.

[0200] Any pulse signal coding processing method provided in the embodiments of the present disclosure can be executed by any appropriate device with data processing capabilities, including but not limited to: camera equipment (such as pulse cameras), terminal equipment and servers, etc. Alternatively, any pulse signal coding processing method provided in the embodiments of the present disclosure can be executed by a processor, such as the processor executing any pulse signal coding processing method mentioned in the embodiments of the present disclosure by calling the corresponding instructions stored in the memory. No further details will be given below.

[0201] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0202] Figure 6 FIG. 1 is a schematic diagram of the structure of an embodiment of a pulse signal coding processing device of the present disclosure. The pulse signal coding processing device of this embodiment can be used to implement the above-mentioned pulse signal coding processing method embodiments of the present disclosure. Figure 6As shown, the encoding processing device of this embodiment includes: a conversion unit 602, a generation unit 604, a sampling unit 606, an acquisition unit 608 and an encoding unit 610. Among them:

[0203] The conversion unit 602 is used to convert the received photon flow into a pulse signal representing the light intensity change process.

[0204] The generating unit 604 is used to generate a pulse sequence for expressing the optical signal of the spatial region and its changing process based on the generation time of the pulse signal corresponding to the spatial region and in accordance with the timing relationship, and the pulse sequence includes a pulse characteristic value for indicating whether a pulse signal is generated at each time.

[0205] In a specific implementation, the encoding processing device may include M generating units 604 , and the M generating units 604 are connected to the M filters 606 in a one-to-one correspondence.

[0206] The sampling unit 606 is used to sample the pulse sequence generated by the generating unit 604 according to a preset sampling frequency to obtain the pulse characteristic value at each sampling moment.

[0207] In a specific implementation, the encoding processing device may include L sampling units 606, and the L sampling units 606 are connected to the M generating units 604 in a one-to-one correspondence or a one-to-many manner. Wherein, L is an integer greater than zero, and the value of M is greater than or equal to L.

[0208] The acquisition unit 608 is used to respond to the end of a data frame output cycle and acquire the data to be encoded in the data frame output cycle according to the current encoding method and the pulse characteristic values ​​at each sampling moment in the data frame output cycle; wherein the data frame output cycle is determined based on a preset data frame output frequency.

[0209] In a specific implementation, the encoding processing device may include: P acquisition units 608, which are connected to L sampling units 606 in a one-to-one correspondence or a one-to-many manner. P is an integer greater than zero, and the value of L is greater than or equal to P.

[0210] The encoding unit 610 is used to encode the data to be encoded within the data frame output period based on the current encoding mode to obtain encoding information corresponding to the data frame output period.

[0211] In a specific implementation, the encoding processing device may include: Q encoding units 610, which are connected to P acquisition units 608 in a one-to-one correspondence or a one-to-many manner. Q is an integer greater than zero, and the value of P is greater than or equal to Q.

[0212] Based on this embodiment, continuous recording and digital expression of the light process is achieved through a pulse sequence. The pulse sequence generated by each pixel can be used to estimate the light intensity at any time and obtain the image at any time, thereby realizing continuous imaging. It has high time sensitivity, high-frequency expression capability and super expression capability for high-speed motion processes. It can effectively record high-speed motion, such as rotating wheels, high-speed table tennis and even football, and can capture motion details in the monitoring scene, and can support special needs such as scientific research and high-precision detection. In addition, after generating a pulse sequence, this embodiment does not output the pulse sequence in real time, but encodes it at a lower frequency according to the data frame output period and then outputs it, thereby reducing the pressure of data storage and transmission.

[0213] Optionally, in some of the possible implementations, the conversion unit 602 may include a signal collector, a signal accumulator and a filter. The signal collector is used to collect optical signals in a spatial area. The signal accumulator is used to accumulate the optical signals in the spatial area collected by the signal collector to obtain a signal accumulation value. The filter is used to generate a pulse signal in response to the signal accumulation value obtained by the signal accumulator reaching a preset threshold, and reset the signal accumulation value for re-accumulation. In a specific implementation, the encoding processing device may include: N signal collectors, M signal accumulators and M filters, each signal collector can be connected to one or more signal accumulators, that is, the two are connected in a one-to-one correspondence or a one-to-many manner; M signal accumulators and M filters are connected in a one-to-one correspondence. M and N are integers greater than zero, respectively, and the value of M is greater than or equal to N.

[0214] Optionally, in some of the possible implementations, the pulse signal encoding and processing device of the embodiment of the present disclosure may include multiple signal collectors and multiple signal accumulators. Each of the multiple signal collectors is used to collect light signals from each spatial region in the observation scene, convert the instantaneous light intensity value of the light signal collected each time into an electrical signal that is positively correlated with the instantaneous light intensity value, and transmit the signal strength value of the electrical signal to the connected signal accumulator. Among them, each signal collector realizes the collection of light signals from a spatial region in the observation scene, the spatial regions corresponding to different signal collectors in the multiple signal collectors do not overlap, and the array formed by the multiple signal collectors covers the collection of light signals from the entire observation scene; the pulse sequences corresponding to all spatial regions in the observation scene are arranged according to the spatial position relationship of the spatial regions to form a pulse array.

[0215] Optionally, in some of the possible implementations, the pulse signal encoding processing device of the embodiment of the present disclosure may include multiple filters. In this embodiment, the signal accumulator is specifically used to: accumulate the received signal strength value to obtain the signal accumulation value, and pass the signal accumulation amount to the connected filter. Accordingly, the filter is specifically used to: filter the received signal accumulation value based on a preset threshold; in response to the signal accumulation value reaching the preset threshold, generate a pulse signal, and reset the signal accumulation value on the connected signal accumulator.

[0216] Optionally, in some possible implementations, the sampling unit 606 is specifically configured to: n The sampling frequency of Hertz is used to sample the pulse sequence and obtain the 1 / 2 n The pulse characteristic value within the time unit, the 1 / 2 n The pulse characteristic value within the time unit is used to characterize 1 / 2 n The flag indicating whether a pulse signal exists in a time unit can be 0 or 1; wherein the value of n is an integer greater than or equal to 1.

[0217] Optionally, in some of the possible implementations, the preset data frame output frequency is 2 m The data frame output frequency in Hertz; wherein the value of m is an integer greater than or equal to 0, and the value of m is less than the value of n.

[0218] Optionally, in some possible implementations, the acquisition unit 608 is further used to: obtain a current encoding mode; wherein the current encoding mode may, for example, include but is not limited to at least one of the following: a pulse quantity encoding mode, an instantaneous pulse encoding mode, and a strongest pulse encoding mode.

[0219] Optionally, in some possible implementations, the acquisition unit 608 is specifically used to obtain a coding mode corresponding to pre-set coding mode information as the current coding mode; or, determine a coding mode corresponding to the received coding mode information as the current coding mode; or, from the pre-set coding mode information, select a coding mode according to a preset selection method as the current coding mode; or, obtain a coding mode corresponding to the current application scenario as the current coding mode.

[0220] Optionally, in some of the possible implementations, when the current encoding mode is a pulse number encoding mode, the acquisition unit 608 is specifically used to: in response to the end of a data frame output cycle, determine the number of pulse signals generated in the data frame output cycle based on the pulse characteristic values ​​at each sampling time in the data frame output cycle as the data to be encoded in the data frame output cycle; wherein the value range of the number of pulse signals generated in the data frame output cycle is [0,2 n-m -1].

[0221] Accordingly, in some possible implementations, the encoding unit 610 is specifically used to encode the number of pulse signals generated within the data frame output period into encoding information with a bit depth of nm bits as the encoding information corresponding to the data frame output period.

[0222] Optionally, in some possible implementations, when the current encoding method is an instantaneous pulse encoding method, the acquisition unit 608 is specifically used to: in response to the end of a data frame output cycle, based on the pulse characteristic values ​​at each sampling moment in the data frame output cycle, obtain the pulse width of the pulse signal generated in the data frame output cycle and closest to the current moment as the data to be encoded in the data frame output cycle.

[0223] Accordingly, in some possible implementations, the encoding unit 610 is specifically configured to: encode the pulse width of the most recent pulse signal into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

[0224] Optionally, in some possible implementations, when the current encoding method is the strongest pulse encoding method, the acquisition unit 608 is specifically used to: in response to the end of a data frame output cycle, based on the pulse characteristic values ​​at each sampling moment in the data frame output cycle, obtain the shortest pulse width in the pulse width of the pulse signal generated in the data frame output cycle as the data to be encoded in the data frame output cycle.

[0225] Accordingly, in some possible implementations, the encoding unit 610 is specifically configured to: encode the shortest pulse width into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

[0226] Figure 7 FIG. 1 is a schematic diagram of another embodiment of the pulse signal coding processing device disclosed in the present invention. Figure 7 As shown, in Figure 6 On the basis of the illustrated embodiment, the encoding processing device of this embodiment further includes a first determining unit 702, which is used to determine the bit rate of the encoding information based on a preset sampling frequency, a preset data frame output frequency, and a bit depth of the encoding information.

[0227] In addition, see Figure 7 The encoding processing device of the embodiment of the present disclosure may also include a compression unit 704, which is used to use a preset compression encoding algorithm to compress and encode the encoding information corresponding to the data frame output period to obtain compressed encoding information.

[0228] In addition, see Figure 7 The encoding processing device of the embodiment of the present disclosure may also include a decompression unit 706, which is used to use the preset compression encoding algorithm to decode the compressed encoding information to obtain the encoding information corresponding to the data frame output period.

[0229] In addition, see Figure 7 The encoding processing device of the embodiment of the present disclosure may also include a second determining unit 708, which is used to determine a preset sampling frequency based on the size of the local storage space, the bit depth of the encoding information, and the number of pixels included in the encoding processing device, or adjust the preset sampling frequency; or, based on the size of the local storage space, the bit rate of the encoding information and the transmission bandwidth, determine a preset data frame output frequency, or adjust the preset data frame output frequency; or, according to the requirements of the current application scenario, determine the preset sampling frequency, or adjust the preset sampling frequency.

[0230] In addition, see Figure 7 The encoding processing device of the embodiment of the present disclosure may also include: a third determination unit 710, used to determine the number of pulse signals generated within the data frame output period based on the encoding information corresponding to the data frame output period; a conversion unit 712, used to convert the number of pulse signals generated within the data frame output period into a digital image with a bit depth of nm bits.

[0231] In addition, see Figure 7 The encoding processing device of the embodiment of the present disclosure may further include a fourth determining unit 714, which is used to determine the light intensity value corresponding to the encoding information based on the current encoding mode and the encoding information corresponding to the data frame output period.

[0232] Optionally, in some of the possible implementations, when the current encoding mode is a pulse number encoding mode, the encoding processing device of the embodiment of the present disclosure may further include a third determination unit 710, which is used to determine the number of pulse signals generated in the data frame output period based on the encoding information corresponding to the data frame output period. Accordingly, the fourth determination unit 714 is specifically used to: obtain the average light intensity value in the data frame output period based on the data frame output period and the number of pulse signals generated in the data frame output period.

[0233] Optionally, see Figure 7In some of the possible implementations, when the current encoding mode is an instantaneous pulse encoding mode, the encoding processing device of the embodiment of the present disclosure may further include a fifth determining unit 716, which is used to determine the pulse width of the pulse signal generated within the data frame output period and closest to the current moment based on the encoding information corresponding to the data frame output period. Accordingly, the fourth determining unit 714 is specifically used to obtain the instantaneous light intensity value corresponding to the pulse width of the latest pulse signal based on the pulse width of the latest pulse signal.

[0234] Optionally, see Figure 7 In some possible implementations, when the current encoding mode is the strongest pulse encoding mode, the encoding processing device of the embodiment of the present disclosure may further include a sixth determining unit 718, which is used to determine the shortest pulse width among the pulse widths of the pulse signal generated within the data frame output period based on the encoding information corresponding to the data frame output period. Accordingly, the fourth determining unit 714 is specifically used to obtain the maximum light intensity value corresponding to the shortest pulse width based on the shortest pulse width.

[0235] In addition, see Figure 7 The encoding processing device of the embodiment of the present disclosure may also include a target detection unit 720, which is used to perform target detection based on the light intensity value corresponding to the encoding information corresponding to one or more spatial regions in the observation scene.

[0236] In addition, see Figure 7 The encoding processing device of the embodiment of the present disclosure may also have a reconstruction unit 722, which is used to generate a reconstructed image of the observation scene based on the light intensity values ​​corresponding to the encoding information corresponding to all spatial regions in the observation scene by using a preset pulse reconstruction algorithm; wherein the grayscale value of each pixel in the reconstructed image represents the light intensity value of the spatial region corresponding to each pixel.

[0237] In addition, an embodiment of the present disclosure further provides a pulse camera, including a pulse signal encoding and processing device provided by any of the above embodiments of the present disclosure.

[0238] In addition, an embodiment of the present disclosure further provides an electronic device, including:

[0239] Memory for storing computer programs;

[0240] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the pulse signal encoding processing method described in any of the above embodiments of the present disclosure is implemented.

[0241] Optionally, in some of the possible implementations, the electronic device in the embodiments of the present disclosure may include, but is not limited to, any of the following: an integrated circuit, a sensor, and the like.

[0242] Optionally, in some other possible implementations, the electronic device in the embodiments of the present disclosure may include, but is not limited to, any of the following: cameras, webcams, audio / video players, navigation devices, fixed-location terminals, entertainment devices, smart phones, communication devices, mobile devices, transportation vehicles or facilities, industrial equipment, medical equipment, security equipment, flight equipment, household appliances, etc.

[0243] Figure 8 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Figure 8 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0244] like Figure 8 As shown, the electronic device includes one or more processors 802 and memory 804 .

[0245] The processor 802 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0246] The memory 804 may store one or more computer program products, and the memory 804 may include various forms of computer-readable storage media, such as volatile memory 804 and / or non-volatile memory 804. The volatile memory 804 may include, for example, a random access memory 804 (RAM) and / or a cache memory 804 (cache), etc. The non-volatile memory 804 may include, for example, a read-only memory 804 (ROM), a hard disk, a flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor 802 may run the computer program products to implement the pulse signal encoding processing method of each embodiment of the present disclosure described above and / or other desired functions.

[0247] In one example, the electronic device may further include: an input device 806 and an output device 808 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0248] In addition, the input device 806 may also include, for example, a keyboard, a mouse, etc.

[0249] The output device 808 can output various information to the outside, including the determined distance information, direction information, etc. The output device 808 can include, for example, a display, a speaker, a printer, a communication network and a remote output device 808 connected thereto, and the like.

[0250] Of course, to simplify, Figure 8 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0251] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by the processor 802, enable the processor 802 to execute the steps of the encoding processing method of the pulse signal according to various embodiments of the present disclosure described in the above part of this specification.

[0252] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0253] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by the processor 802, the processor 802 executes the steps of the pulse signal encoding processing method according to various embodiments of the present disclosure described in the above part of this specification.

[0254] The computer readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory 804 (RAM), a read-only memory 804 (ROM), an erasable programmable read-only memory 804 (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory 804 (CD-ROM), an optical storage 804 piece, a magnetic storage 804 piece, or any suitable combination of the above.

[0255] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0256] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0257] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0258] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0259] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0260] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0261] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A pulse signal coding processing method, characterized in that: include: Convert the received photon stream into a pulse signal representing the light intensity change process; Based on the generation time of the pulse signal corresponding to the spatial region, a pulse sequence for expressing the optical signal of the spatial region and its change process is generated according to the time sequence relationship, wherein the pulse sequence includes a pulse characteristic value for indicating whether a pulse signal is generated at each time; Sampling the pulse sequence according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment; In response to the end of a data frame output cycle, the data to be encoded in the data frame output cycle is obtained according to the current encoding mode and the pulse characteristic value at each sampling moment in the data frame output cycle; wherein the data frame output cycle is determined based on a preset data frame output frequency, and the preset data frame output frequency is less than the preset sampling frequency; Based on the current encoding mode, the data to be encoded within the data frame output period is encoded to obtain encoding information corresponding to the data frame output period.

2. The method according to claim 1, characterized in that The step of sampling the pulse sequence according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment includes: According to 2 n The sampling frequency of Hz is used to sample the pulse sequence to obtain the 1 / 2 n The pulse characteristic value within the time unit, the 1 / 2 n The pulse characteristic value within the time unit is used to characterize 1 / 2 n An indicator of whether there is a pulse signal in the time unit; where n is an integer greater than or equal to 1.

3. The method according to claim 2, characterized in that The preset data frame output frequency is 2 m The data frame output frequency in Hertz; wherein the value of m is an integer greater than or equal to 0, and the value of m is less than the value of n.

4. The method according to claim 3, characterized in that Also includes: Obtaining the current encoding mode; The current coding method includes at least one of the following: a pulse quantity coding method, an instantaneous pulse coding method, and a strongest pulse coding method.

5. The method according to claim 4, characterized in that When the current encoding mode is a pulse number encoding mode, obtaining the data to be encoded in the data frame output period according to the current encoding mode and the pulse characteristic values ​​at each sampling moment in the data frame output period includes: Based on the pulse characteristic values ​​at each sampling moment in the data frame output period, determining the number of pulse signals generated in the data frame output period as the data to be encoded in the data frame output period; The value range of the number of pulse signals generated within the data frame output period is [0,2 n-m -1].

6. The method according to claim 5, characterized in that The step of encoding the data to be encoded within the data frame output period based on the current encoding mode to obtain encoding information corresponding to the data frame output period includes: The number of pulse signals generated within the data frame output period is encoded into encoding information with a bit depth of nm bits as the encoding information corresponding to the data frame output period.

7. The method according to claim 4, characterized in that When the current encoding mode is an instantaneous pulse encoding mode, obtaining the data to be encoded in the data frame output period according to the current encoding mode and the pulse characteristic values ​​at each sampling moment in the data frame output period includes: Based on the pulse characteristic values ​​at each sampling moment in the data frame output period, the pulse width of the pulse signal generated in the data frame output period and closest to the current moment is obtained as the data to be encoded in the data frame output period.

8. The method according to claim 7, characterized in that The step of encoding the data to be encoded within the data frame output period based on the current encoding mode to obtain encoding information corresponding to the data frame output period includes: The pulse width of the most recent pulse signal is encoded into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

9. The method according to claim 4, characterized in that When the current encoding mode is the strongest pulse encoding mode, obtaining the data to be encoded in the data frame output period according to the current encoding mode and the pulse characteristic values ​​at each sampling time in the data frame output period includes: Based on the pulse characteristic values ​​at each sampling moment in the data frame output period, the shortest pulse width among the pulse widths of the pulse signal generated in the data frame output period is obtained as the data to be encoded in the data frame output period.

10. The method according to claim 9, characterized in that The step of encoding the data to be encoded within the data frame output period based on the current encoding mode to obtain encoding information corresponding to the data frame output period includes: The shortest pulse width is encoded into encoding information with a bit depth of nm bits as encoding information corresponding to the data frame output period.

11. The method according to any one of claims 4 to 10, characterized in that: The obtaining of the current encoding mode comprises: Obtaining a coding mode corresponding to the preset coding mode information as the current coding mode; or, Determine the encoding mode corresponding to the received encoding mode information as the current encoding mode; or, Selecting a coding mode as the current coding mode from the preset coding mode information according to a preset selection method; or, Obtain an encoding mode corresponding to the current application scenario as the current encoding mode.

12. The method according to any one of claims 1 to 10, characterized in that: Also includes: The bit rate of the encoding information is determined based on the preset sampling frequency, the preset data frame output frequency, and the bit depth of the encoding information.

13. The method according to any one of claims 1 to 10, characterized in that: After obtaining the coding information corresponding to the data frame output period, the method further includes: A preset compression coding algorithm is used to compress and encode the coding information corresponding to the data frame output period to obtain compressed coding information.

14. The method according to claim 13, characterized in that After obtaining the compressed coding information, the method further includes: The preset compression coding algorithm is used to decode the compression coding information to obtain the coding information corresponding to the data frame output period.

15. The method according to any one of claims 1 to 10, characterized in that: Also includes: Determine the preset sampling frequency or adjust the preset sampling frequency based on the size of the local storage space, the bit depth of the encoding information, and the number of pixels included in the device for implementing the encoding processing method of the pulse signal; or, Determine the preset data frame output frequency or adjust the preset data frame output frequency based on the size of the local storage space, the code rate of the encoding information and the transmission bandwidth; or, The preset sampling frequency is determined or adjusted according to the requirements of the current application scenario.

16. The method according to any one of claims 1 to 10, characterized in that: Convert the received photon stream into a pulse signal that represents the light intensity change process, including: Collecting optical signals in a spatial region, and accumulating the optical signals in the spatial region to obtain a signal accumulation value; In response to the signal accumulation value reaching a preset threshold, a pulse signal is generated, and the signal accumulation value is reset so as to be accumulated again.

17. The method according to claim 16, characterized in that After accumulating the optical signals in the spatial region to obtain the signal accumulation value, the method further includes: In response to the end of a data frame output cycle, if the signal accumulation value within the data frame output cycle does not reach a preset threshold, the signal accumulation value is output.

18. The method according to claim 17, characterized in that The method further comprises: After the signal accumulation value is output, the signal accumulation value is reset to zero to re-accumulate.

19. The method according to any one of claims 5-6, characterized in that: After obtaining the coding information corresponding to the data frame output period, the method further includes: Determining the number of pulse signals generated within the data frame output period based on the coding information corresponding to the data frame output period; The number of pulse signals generated within the data frame output period is converted into a digital image with a bit depth of nm bits.

20. The method according to any one of claims 4 to 10, characterized in that: After obtaining the coding information corresponding to the data frame output period, the method further includes: Based on the current encoding mode and the encoding information corresponding to the data frame output period, a light intensity value corresponding to the encoding information is determined.

21. The method according to claim 20, characterized in that When the current encoding mode is a pulse number encoding mode, determining the light intensity value corresponding to the encoding information based on the current encoding mode and the encoding information corresponding to the data frame output period includes: Determining the number of pulse signals generated within the data frame output period based on the coding information corresponding to the data frame output period; Based on the data frame output period and the number of pulse signals generated within the data frame output period, an average light intensity value within the data frame output period is acquired.

22. The method according to claim 20, characterized in that When the current encoding mode is an instantaneous pulse encoding mode, determining the light intensity value corresponding to the encoding information based on the current encoding mode and the encoding information corresponding to the data frame output period includes: Determine the pulse width of the pulse signal generated within the data frame output period and closest to the current moment based on the coding information corresponding to the data frame output period; Based on the pulse width of the most recent pulse signal, an instantaneous light intensity value corresponding to the pulse width of the most recent pulse signal is obtained.

23. The method according to claim 20, characterized in that When the current coding mode is the strongest pulse coding mode, determining the light intensity value corresponding to the coding information based on the current coding mode and the coding information corresponding to the data frame output period includes: Determine the shortest pulse width among the pulse widths of the pulse signals generated within the data frame output period based on the coding information corresponding to the data frame output period; Based on the shortest pulse width, a maximum light intensity value corresponding to the shortest pulse width is acquired.

24. The method according to claim 20, characterized in that After determining the light intensity value corresponding to the coding information based on the current coding mode and the coding information corresponding to the data frame output period, the method further includes: Target detection is performed based on light intensity values ​​corresponding to the coded information corresponding to one or more spatial regions in the observation scene.

25. The method according to claim 20, characterized in that After determining the light intensity value corresponding to the coding information based on the current coding mode and the coding information corresponding to the data frame output period, the method further includes: Using a preset pulse reconstruction algorithm, a reconstructed image of the observation scene is generated based on the light intensity values ​​corresponding to the encoded information corresponding to all spatial areas in the observation scene; wherein the grayscale value of each pixel in the reconstructed image represents the light intensity value of the spatial area corresponding to each pixel.

26. A pulse signal coding processing device, characterized in that: include: A conversion unit, used for converting the received photon flow into a pulse signal representing a light intensity variation process; A generating unit, configured to generate a pulse sequence for expressing the optical signal of the spatial region and its changing process according to a time sequence relationship based on the generation time of the pulse signal corresponding to the spatial region, wherein the pulse sequence includes a pulse characteristic value for indicating whether a pulse signal is generated at each time; A sampling unit, used to sample the pulse sequence according to a preset sampling frequency to obtain a pulse characteristic value at each sampling moment; An acquisition unit, configured to, in response to the end of a data frame output cycle, acquire the data to be encoded in the data frame output cycle according to the current encoding mode and the pulse characteristic values ​​at each sampling moment in the data frame output cycle; wherein the data frame output cycle is determined based on a preset data frame output frequency, and the preset data frame output frequency is less than the preset sampling frequency; The encoding unit is used to encode the data to be encoded within the data frame output period based on the current encoding method to obtain encoding information corresponding to the data frame output period.

27. A pulse camera, characterized in that: Comprising the device of claim 26.

28. An electronic device, characterized in that: include: A memory for storing a computer program product; A processor is used to execute a computer program product stored in the memory, and when the computer program product is executed, it implements the method described in any one of claims 1 to 25.

29. The electronic device according to claim 28, characterized in that The electronic device includes any one of the following: an integrated circuit, a sensor.

30. The electronic device according to claim 28, characterized in that The electronic device includes any one of the following: a camera, a webcam, an audio / video player, a navigation device, a fixed-position terminal, an entertainment device, a smart phone, a communication device, a mobile device, a transportation vehicle or facility, an industrial device, a medical device, a security device, a flight device, and a household appliance.

31. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 25 is implemented.